Verified figures
Every published figure has been independently re-extracted from MIMIC-IV and reproduced across five or more separate pipeline runs. Experiment 2's action-recoverability gap (0.900 → 0.791) survived a 15× swing in underlying row counts almost untouched — strong evidence it's a robust, ratio-based effect, not an artifact of the bugs below. Full before/after table: Evidence page.
What was found and fixed
Three real pipeline bugs were found, diagnosed, and fixed in the open: a BigQuery write-disposition gap that silently duplicated rows (~15×) on re-run, a schema-autodetect type mismatch that broke downstream joins, and a window-overlap sum-vs-union error that double-counted patients on simultaneous multi-drug vasopressor therapy (34.8% of the cohort). Full diagnostic trail, including the false starts: Evidence page.
Project phases
Phase 0 — Foundation
Cohort extraction, interactive demo, manuscript draft, repo scaffold.
Phase 1–3 — Extraction, verification, bug fixes
Full pipeline re-run five+ times end to end. Cohort, Experiment 1, Experiment 2, and Experiment 3 all independently reproduced. Two real bugs found, fixed, and confirmed against the original source run recovered from an earlier commit.
Reviewer response — Mortality-predictive validity per variant (Experiment 6, 2026-09-03)
Operationalizes the clinical reviewer's task-specificity point: does a variant's lower action-recoverability also mean lower mortality-predictive validity? Action-recoverability gap (A minus E) = 0.109 — large. Mortality-AUROC gap (A minus E) = 0.011 — about 3% to 10% of the action gap (paired ratio 0.065, 95% CI [0.036, 0.095]), not "roughly a tenth." Reported as the actual result: sanitizing the state costs almost nothing in mortality-prediction accuracy despite costing a lot in next-action predictability. Full write-up: evidence page.
Reviewer response — Respiratory SOFA, a second mechanism instance (Experiment 7 / Proposition 3, 2026-09-04)
Tests whether the cardiovascular non-identifiability pattern generalizes to a different SOFA subscore. It doesn't, in the same form — respiratory SOFA shows a narrower score-shift collision instead: for PF in [100,200), two patients with identical PF get different official scores (3 vs. 2) purely as a function of ventilation status. A genuinely different mathematical shape, deliberately isolated from Experiments 1/2/3/6. Full write-up: evidence page, proof: FORMAL_ANALYSIS.md §4.5.
Reviewer response — Variant F disentanglement test (Experiment 8, 2026-09-08)
Tests a specific alternative to Variant D: keep severity untouched, add explicit treatment-history features (duration on vasopressor, time since last dose-tier change) alongside it. The pre-registered success criterion was falsified — F's action-recoverability AUROC (0.914) exceeded even the full state's (0.900), rather than staying near D's (0.792). Reported as the actual result: two purely temporal features are more predictive of the next action than the entire physiology-plus-severity state, sharpening the treatment-persistence account below rather than avoiding it. Full write-up: evidence page, proof: FORMAL_ANALYSIS.md §7.5.
Phase 4 — Experiment 5 (negative controls) and H3
All four negative controls have run: shuffled labels (AUROC ≈0.5, as expected), a 5-offset
temporal decay curve (0.874 at 0h down to 0.703 at 32h), a cross-reference against the variant
with no explicit treatment features, and prevalence-matched placebo features (matches variant
E's own number, confirming the placebo adds nothing). Results:
notebook/results/experiment5_negative_controls.json.
H3 has been run. The pre-registered paired-bootstrap test on per-patient predictions
gives Delta = AUROC(0) minus AUROC(8) = 0.147, 95% CI [0.140, 0.154] (bin-index offsets, common
rows, logreg) — Prediction A, per
docs/DECISIONS_H3.md.
An exploratory synthetic ground-truth check (exploratory work in review, exp-simulation) finds this rule cannot, on
its own, separate the SOFA-construction mechanism from ordinary treatment persistence — see
Evidence for the exploratory
analyses and the exact test and
why.
Phase 5 — Statistical rigor pass
Clustered bootstrap CIs for mutual information, multiple-comparisons correction, the matched 4-hour window-overlap statistic.
Phase 6 — Literature completion
Verified citation table of related sepsis-RL work, causal-inference and window-bias references, the Appendix A literature audit.
Phase 7 — Manuscript finalization
Sweep every remaining placeholder, rewrite the Abstract and Results with the verified figures above, remove the internal-draft watermark.
Phase 8 — Thesis-scale extensions
Cross-dataset replication on eICU, fluid administration (the other half of the action space), reward-function leakage, a formal policy-sensitivity study.