What "in production" means in a hospital

In short: in a hospital, a model is in production when a clinician can see its output during real care, a named group has approved that, a named person owns it when it breaks, and the ward runs fine the moment you switch it off. The deploy is the easy part.

In a tech company, "in production" usually means the code is live and users hit it. I spent several years building machine learning inside a hospital network: decision support tools, federated learning across a research consortium in several countries, and work on keeping models and the data behind them secure. Most of what I learned there was about the distance between "it works" and "it is allowed to be used".

That distance is not bureaucracy for its own sake. Every step exists because a model in a hospital touches a patient through a person, and the hospital is responsible for both.

Production is a permission, not a deploy

The order varies between institutions, but the shape is roughly this:

1. Research approvalAn ethics board and a data access agreement say which data you may use, for what, and for how long.
2. Privacy and security reviewWhere the data sits, who can see it, how it is encrypted, and whether it ever leaves the network.
3. Silent runThe model runs on live data, nobody sees its output, and you compare it with what clinicians actually decided.
4. Clinical sign-offA clinical lead agrees on what the output looks like, how it is worded, and which decisions it may inform.
5. Limited go-liveOne unit, one use, a named owner, and a date to review whether it should continue.
6. Ongoing reviewRegular checks of performance and drift, and a written rule for when to switch it off.

The silent run is the step engineers skip in their heads and should not. It is the only time you see the model on real, current data with no risk to anyone. Data in a research extract is cleaner and older than data arriving live, and the gap between the two shows up here, not in your validation set.

Who is on call?

Ask this before go-live, and write the answer down. There are usually three different people, and none of them is you by default:

• IT on call looks after servers and the network. They can restart a service. They cannot judge whether a prediction is wrong.
• The clinician on duty looks after the patient. They can ignore the model. They cannot fix it.
• The model owner, usually a small team or one engineer, works office hours and is often not on any rota.

That last point changes the design. If nobody who understands the model is awake at 3am, the model has to fail in a way that the two people who are awake can handle without you.

When the model is wrong at 3am

It will be wrong. The question is what happens next. In a well-designed setup, the answer is: very little, because the model never acted alone.

Three properties make that true:

1. Decision support, not decision. The output informs a clinician who can see the same patient. It never triggers an action by itself.
2. Visible failure. When input data is missing or out of range, the model shows "no result" rather than a confident guess. A blank is safer than a wrong number that looks right.
3. A tested off switch. Turning the model off is one action that IT on call can take, and the clinical workflow does not notice.

In a hospital, the model is never the last line. A person is, and your job is to make their night easier, not harder.

What this means for the engineering

Knowing all of the above changes what you build long before go-live:

• Every prediction is logged with the model version and the inputs it saw, so a strange result can be explained the next morning.
• Input checks run before the model, with plain-language reasons when they reject a case.
• The output is worded with the clinicians who will read it. "Higher than usual risk, see factors" lands very differently from a bare 0.73.
• Monitoring watches the input distribution, not just latency and errors.
• The review date is in the calendar from day one. A model nobody reviews is a model nobody owns.

None of this is specific to one hospital, and none of it needs a large team. It needs the questions asked early.

Before your next go-live

Write a one-page note titled "If this is wrong". Name who sees the output, who can switch it off and how, what the ward does without it, and which input changes would make it silently wrong. Then ask a clinician and someone from IT to read it. If either of them finds a step they did not know about, you are not in production yet.