prodrome-clock

Hourly early warning for sepsis in intensive care, from the Prodrome pipeline. Trained on hospital system A of the PhysioNet/CinC 2019 challenge corpus and evaluated on system B, which it never saw.

This is a research artefact. It is not a medical device, it has not been cleared or approved by any regulator, and it must not be used to inform clinical decisions.

Read this before the numbers

• It predicts a recorded clinical suspicion, not a biological event. The label turns on six hours before a clinician documented suspicion of infection alongside a rise in SOFA score. Every label in this field is a proxy.
• A three-feature clock reaches 100% of this model's utility score. A baseline that alerts on nothing but how long the patient has been in the unit scores 0.2413 against this model's 0.2413. Septic stays in this corpus run substantially longer than the rest, and the challenge utility metric rewards that. What this model adds is visible in the ranking, the in-window detection rate and the calibration, not in the headline score.
• The model leans heavily on the workup and the clock. An ablation fitting the same model on measured values alone reaches roughly half the utility, while the ordering and staleness columns alone reach most of it. Nobody orders a lactate for a patient they are relaxed about, so the test-ordering pattern encodes clinician suspicion, and the label is a record of clinician suspicion.
• Two hospital systems in one country is not the world. The only supported claim is the transfer measured below.
• The reported lead times are generous. A long median largely reflects patients flagged early in a long stay, which is why the stricter in-window detection rate is reported beside it.

Results

Measured by prodrome eval on corpus ef46d3e6ec24, split seed 7. External results first, because that is the number that means something.

tested on	utility	AUPRC	AUROC	detected (ever)	detected (in window)	median lead	alerts per patient-day	precision among alerted	calibration error
site B (external, never trained on)	0.2413	0.047	0.628	0.347	0.065	45.5 h	0.04	0.346	0.097
site A (internal)	0.2938	0.080	0.642	0.369	0.101	30.5 h	0.04	0.506	0.090

utility is the published challenge scoring function, where alerting through the whole beneficial window scores 1 and never alerting scores 0. precision among alerted is the share of alerted patients who go on to become septic, against a base rate of about 7 percent.

What it promises

With probability at least 95% over the calibration draw, at most 4.0 alerts are raised per patient-day and at least 30% of alerted patients go on to develop sepsis (observed on the calibration slice: 0.04 alerts per patient-day, 45% precision).

Checked on the external test patients, that promise held: precision among alerted patients was 0.346 against a floor of 0.30, and the alert rate was 0.04 against a budget of 4.0 per patient-day.

The threshold is not a default. It is chosen on held-out calibration patients as the lowest cut whose precision, at its exact Clopper-Pearson lower bound, clears the operator's floor and whose alert rate stays inside the operator's budget. When no threshold qualifies, the policy alerts on nothing rather than implying a guarantee it cannot support.

What the model is reading

ablation	features	utility	share of full	auroc	auprc	in window	alerts/pt-day
full	205	0.2468	100%	0.8016	0.0724	0.153	0.11
physiology_only	125	0.1159	47%	0.7697	0.0604	0.117	0.08
process_only	75	0.2159	87%	0.7263	0.0523	0.109	0.09
clock_only	3	0.2428	98%	0.6656	0.0488	0.074	0.06
workup_only	72	0.2119	86%	0.7539	0.0520	0.115	0.10
no_time_index	202	0.2316	94%	0.7937	0.0707	0.156	0.14

The process and clock columns alone reach 87% of the full model's utility, more than physiology alone at 47%. This model is substantially reading the workup rather than the patient, which is worth knowing before anyone calls it an early warning.

Measured on site B (external).

Subgroups on the external site

A model that works on average and fails on the over-eighties is a model that fails, so the breakdown is here rather than on request.

dimension	group	patients	septic	utility	AUROC	detection rate
unit	micu	6,923	390	0.1658	0.542	0.251
age_band	80_plus	2,322	129	0.1949	0.598	0.279
age_band	50_to_64	6,278	351	0.2268	0.612	0.316
age_band	under_50	4,818	276	0.2351	0.610	0.341
sex	male	10,732	647	0.2357	0.628	0.342
sex	female	9,268	495	0.2486	0.627	0.354
unit	sicu	6,982	428	0.2720	0.689	0.393
age_band	65_to_79	6,582	386	0.2736	0.665	0.402
unit	unknown	6,095	324	0.2888	0.649	0.401

Intended use

Research and education: reproducing the reported numbers, comparing a new method against a measured baseline on an openly licensed corpus, and studying calibrated abstention and cross-hospital transfer.

Out of scope: any clinical use, any deployment that informs patient care, any triage or resource allocation, and any claim about a population unlike the two hospital systems in this corpus.

How to use it

pip install prodrome
prodrome init
prodrome data fetch      # the corpus, under its own licence, 42 MB
prodrome data prepare
prodrome train && prodrome calibrate && prodrome eval

The model file here is loadable by prodrome.core.registry through the configuration recorded alongside it. A clinical model without its feature pipeline is not a model, so the preprocessing travels with the weights.

Training data

corpus	PhysioNet/CinC Challenge 2019, version 1.0.0
patients	40,336 across two hospital systems
rows	1,552,210, one per ICU hour
septic	2,932 patients, 7.3 percent
trained on	system A, 4067 held-out patients at test
corpus digest	ef46d3e6ec2497f7279ef352acdc32521cf54d42bb785ba2f3df4f2e179d854f

Licence and attribution

The model weights and the Prodrome source are Apache-2.0.

This model was produced from the PhysioNet/Computing in Cardiology Challenge 2019 database, which is available under the Open Database License (ODbL) v1.0. Under section 4.3 of that licence this notice accompanies the model as a Produced Work. The database itself is neither redistributed with this model nor included in the source repository. Cite Reyna et al., Early Prediction of Sepsis from Clinical Data: The PhysioNet/Computing in Cardiology Challenge 2019, Critical Care Medicine 48(2), 2020.

Reproducing this card

Every number above is generated from the committed evaluation reports by prodrome publish card, so the card cannot drift from what the harness measured.