It Was in the Manual — Eight People Still Died

In 2003, a radiation treatment planning system called Multidata was in use at Panama's National Oncology Institute. It calculated the dose a cancer patient should receive during therapy. It had been tested. It had regulatory approval. It was working correctly for the vast majority of cases.

But there was a specific configuration — a particular arrangement of radiation shielding blocks that oncologists drew directly into the software — that the system misinterpreted. When technicians drew the blocks in a certain way, the software produced a number that looked completely normal. Confident. Precise. And between three and five times higher than what a human body could survive.

Seventeen patients were overdosed. Eight of them died.

The software had a known limitation. It was documented. The vendor could point to the page in the manual.

The Responsibility Shuffle

Nobody in this story woke up and decided to hurt anyone. The developers built what was specified. The testers tested what was in scope. The vendor documented the limitation. The regulators certified what was in front of them. The oncologists trusted a system that had been approved.

Every individual action, examined on its own, was reasonable.

And yet.

The developer said: I built the spec. The tester said: I tested the documented requirements. The vendor said: it's in the manual. The regulator said: it passed review at the time.

Nobody lied. Nobody was obviously negligent. There was just a series of individually reasonable actions that, assembled together, produced an outcome nobody would have chosen if they'd seen it whole. Everyone did their part. Nobody held the complete picture.

This is the most dangerous kind of failure in software — not malice, not carelessness, but incompleteness.

Documentation Is Not a Safety Net

Here is the thing about writing down a known limitation: it only works if it reaches the right person, at the right moment, before the dangerous thing happens.

An oncologist drawing shielding blocks in the middle of a busy clinical day is not consulting the manual. They are trusting the software to stop them if something is wrong. The software didn't stop them. It produced a confident number and said nothing.

Writing down a dangerous edge case is not the same as making the system safe. Closing the ticket is not the same as eliminating the risk. And "we disclosed it" is not the same as "people are protected."

That distance — between documented and safe — is exactly where the Multidata patients ended up.

Three things worth carrying into your next test cycle

Test for the confident wrong answer. The Multidata system didn't crash. It didn't throw an error. It produced a number that looked completely normal. The most dangerous failures aren't the visible ones — they're the ones where your system gives a calm, precise, completely wrong output and nothing flags it.

Translate bug-report language into human-impact language. "Known limitation" sounds almost respectable. But in plain English, the Panama limitation was this: if an operator draws the blocks in a specific way, the software silently calculates the wrong dose and presents it as correct. That's not a limitation. That is a trap with a manual page.

Name the gap explicitly. If multiple parties — vendor, hospital, operator, regulator — each assume someone else owns a specific safety scenario, write it down. "I have tested X. I have not tested Y. I don't know who is responsible for Y." That sentence, in writing, makes a collective blind spot impossible to pretend away.

You won't prevent every failure of this kind. But you can be the person who asks out loud: who is actually verifying this is safe in the real world — not just on paper? And you can make the answer to that question impossible to ignore.

That is what it means to test with a human cost in mind.