Executive takeaways
- No model change can still mean a new evidence question. A hospital-controlled change to inputs or workflow may affect whether prior validation supports current use.
- Impact assessment, not automatic invalidation. A changed dependency triggers proportionate review, not an automatic declaration that all prior evidence is void.
- The decision needs an owner. Who decides proceed / revise / defer when the dependency map changes?
The scenario — synthetic, not a real hospital
Hospital B — Fictional Memorial (synthetic example)
Hospital B deploys a sepsis alert model (Version X.Y-L1) with a documented site release: locked model, sequestered test data, prespecified criteria, joint CMO/CIO approval.
Three months later, three hospital-controlled changes occur in one quarter:
- Nursing documentation: The hospital standardises mental-status assessment from a free-text note to a structured drop-down to improve charting compliance. The model in Part 1 relied on structured nursing-documentation inputs within a bounded range. The drop-down changes distribution and completeness.
- Laboratory interface: A laboratory information system upgrade changes the reference range encoding for lactate and the timing of result availability. For illustration, assume that results become available 45 minutes later than in the evaluated workflow; no actual timing data were measured.
- Alert routing: To reduce alarm fatigue, the escalation route changes from bedside nurse + attending to bedside nurse only, with hypothetical 30-minute batching.
By assumption, no manufacturer software update occurs and the model version remains X.Y-L1. The scenario does not establish what a real manufacturer, hospital or regulator would require.
The distinct decision: who reopens acceptance when the hospital changes?
Part 1 examined release evidence at initial deployment. This example adds a separate proposition: a hospital-controlled operational change can require reassessment of an earlier acceptance decision even when the manufacturer has not issued a new model version. The decision pathway therefore needs to connect hospital change authority with clinical release authority.
Why this creates an evidence question
Under the principles in Part 1, the released configuration assumed:
- A defined input distribution for nursing documentation
- A defined timeliness and encoding for laboratory inputs
- A defined clinical response pathway for monitoring
Each assumed dependency may now differ from the evaluated conditions; this example does not establish that performance degraded. The question for release authority is not "did we change the model?" but "does the evidence that supported X.Y-L1 still support its use with these new operating conditions?"
This is the distinction Part 1 introduced: manufacturer-controlled adaptation vs hospital-controlled dependency change. Both require governance, but the second is often missed if IT change control does not trigger clinical AI impact review.
What management should be able to show — at conceptual level only
A complete assessment would require device-specific review. At conceptual level, the questions are:
| Question | Conceptual check | Why it matters |
|---|---|---|
| Is the input still within evaluated boundary? | Compare new documentation distribution to validation distribution | If distribution shifted materially, prior sensitivity/PPV may not hold |
| Is timeliness still within boundary? | Compare new lab result timing to validation timing | Delayed lactate may delay alert |
| Is monitoring still valid? | Compare alert routing and batching to monitoring plan | Batching may affect response metric |
| Is fallback operable? | Confirm global model and manual override still available | Continuity of care |
The questions illustrate a governance decision, not a prescribed clinical testing protocol. They cannot resolve a particular device’s validation or authorisation status.
Decision consequences
Three proportionate options:
Proceed with documented rationale: If competent impact review substantiates that the change remains within the evaluated boundary and monitoring is adequate, record rationale and continue with enhanced monitoring.
Revise with targeted re-evaluation: If the change may exceed the evaluated boundary, determine with the manufacturer and clinical governance team what targeted re-evaluation is required before authorising continued use.
Defer and revert: If the evidence does not support continued use under the changed conditions, consider deferral or a clinically reviewed fallback, with patient-continuity implications assessed first.
Each option requires named clinical and technical approval. The choice should be recorded with exceptions, not assumed.
What remains unknown
- Whether the synthetic changes actually degraded performance — no live data examined
- Whether any particular PCCP permits these hospital-side changes — device-specific determination required
- Whether monitoring would have detected change quickly enough — effectiveness of monitoring not tested here
- How a court, notified body or insurer would view the change — separate determination
The Board test for Part 2
For a hospital that has not yet encountered this scenario:
- Do we have a dependency map linking clinical AI inputs to source systems and documentation standards?
- Does our IT change process trigger clinical AI impact review?
- Can we retrieve the current dependency map and last release decision within 24 hours?
- Has a clinical authority exercised stop/defer authority in the past 12 months?
A missing or untested answer would identify a governance question requiring follow-up; it would not by itself establish an unsafe deployment or a loss.
Notably Absent
No real hospital, real EHR data, real model, real patient outcomes, real device dossier, real PCCP authorisation, or real insurer decision was examined for this synthetic scenario. No device-specific examination method or clinical acceptance threshold is established by this scenario. This article does not provide medical, legal or insurance advice.
Close
A model update is not the only event that creates a release question. A hospital’s own operational changes can create one. Recognising those dependencies in a controlled release record allows proportionate review before clinicians rely on an assumed capability.
ODA3 Institute would welcome discussion with hospitals that have faced similar dependency changes: what triggered review, what evidence was available, and what decision was taken? Contact: contact@oda3.org.
Related reading: Part 1 — Hospital AI Release Evidence Must Follow the Local Configuration, including its illustrative synthetic practitioner annex. The link becomes available after Part 1 is published.
Methodology note
Hospital dependency change → impact assessment → clinical authority decision → documented proceed/revise/defer outcome
