The two mechanisms

Both are provable — see the formal proof page for the full theorems. This page is the intuitive deep-dive: what each mechanism is, why it happens, and how much it actually measures out to.

Terminology: this is confounding, not ML-sense leakage

Both mechanisms below were originally described as "leakage." A reviewer correctly pushed back — the two diagrams below are the actual distinction, made visual. Full statement: Terminology, formal proof page.

What this project actually found (ordinary treatment history) Prior action A(t-1) State S(t) Next action A(t) Legitimate — H1 and H2 (below) are both this pattern. Verified: this project's own action label has zero overlap with the state window it follows. True (ML-sense) leakage — not found anywhere in this pipeline Action A(t) State S(t) Prediction of A(t) itself

This isn't a retreat from the project's findings — it's a stronger, more precise claim. H1 and H2 are real, provable, and still a genuine problem for offline-RL state design (a state that's a near-deterministic function of the prior action still threatens the causal assumptions policy learning relies on) — they're just a different, causal-inference-flavored pathology (conditioning on a variable affected by treatment) than the ML-flavored "leakage" charge, and the fix is different too: disentangling the treatment-history term, not correcting an indexing bug. Full derivation: FORMAL_ANALYSIS.md §1.1.

One more distinction worth stating plainly: whether a feature counts as "confounded" is itself task-specific — a feature confounded for action-prediction can be exactly right for predicting mortality. See Evidence for the full scope note.

Mechanism 1 — Treatment-confounded severity construction

The cardiovascular SOFA subscore is defined as a function of two things: mean arterial pressure, and whichever vasopressor drug and dose is currently running. For any nonzero dose — not merely above some threshold — the scoring rule's branches stop referencing blood pressure entirely: the moment a vasopressor is active at all, MAP drops out of the formula completely. Two patients with opposite blood pressure, same dose, get the identical score.

100%
of already-treated timesteps have a dose-determined score, not a MAP-determined one
32.3%
of all eligible timesteps affected, 4h window
28.6%
of all eligible timesteps affected, 24h window

Removing the subscore alone isn't sufficient: the overall SOFA total is a deterministic sum of all six organ subscores, so the cardiovascular contribution is exactly reconstructible from the total minus the other five — this is why variant C (subscore removed, total kept) barely drops in predictability (0.885 vs. 0.900), while variant D (total recomputed from a treatment-independent proxy) drops close to the floor set by the variant with no explicit treatment features (0.792 vs. 0.791).

$$\sigma_{cardio} = \text{sofa\_total} - \sum_{i \ne cardio}\text{subscore}_i$$

Full statement and proof: Theorem 1 and Theorem 2.

What happened after this was proven: does fixing it cost anything clinically (Experiment 6 — almost nothing in mortality-prediction accuracy, 0.011 AUROC vs. a 0.109 action-recoverability gap)? Does the same pattern hold for a different SOFA subscore (Experiment 7 / Proposition 3 — yes, but in a structurally different form: a score-shift collision, not a constancy)? Does adding explicit treatment-history features back recover what fixing it loses (Experiment 8 / Variant F — no; the pre-registered test for this was falsified)?

Mechanism 2 — Treatment overlap

A "current state" snapshot is built by aggregating physiology over a lookback window. If that window overlaps a period when treatment was already running, the snapshot carries treatment history into it — even with no drug-dose term in any scoring formula at all.

31.1%
mean window overlap, 24h window (verified, union-corrected)
28.7%
of decision points >50% overlapping
12.4%
median overlap — most windows are partly, not entirely, affected

The counterintuitive part: a naive "wider window, more overlap" intuition is false in general. Absolute overlap hours are provably non-decreasing in window length (windows nest), but the fraction is a ratio of a growing numerator to a growing denominator — it rises exactly when marginal treatment density exceeds average density accumulated so far, the same relationship as a marginal-cost curve crossing an average-cost curve. This is why a 4-hour window measured a higher >50%-overlap rate than the 24-hour window in this project's own data.

Full statement and proof: Proposition 1, treatment overlap.

Try both yourself

The live SOFA calculator and window-overlap timeline are on the guided walkthrough. The full synchronized model — causal graph, state vector, and live predictability gauge moving together — is on the interactive demo.

Guided walkthrough → Live mechanism visualizer →