A prediction records a choice

A predictor answers a conditional question. Given features observed under some data-generating regime, what outcome is likely? Let X be the features and Y the outcome. A score such as P(Y = 1 | X = x) can be excellent for prioritising inspection, forecasting demand or detecting change. Its success requires the future prediction setting to preserve enough of the relationship learned from the past.

An action question is different. Let T be a treatment, policy or system action. We now want to compare outcomes if T were deliberately set to one value rather than another. Structural causal models write this as P(Y | do(T = t)). The do notation means that the ordinary causes of T are replaced by a specified assignment. Potential-outcome notation expresses the same target as a contrast between Y(1) and Y(0), the outcomes a unit would have under the two treatment states. These formalisms differ in emphasis, but both force the analyst to specify a counterfactual contrast rather than read it from association alone.¹²

The two quantities coincide only when the treated and untreated groups are exchangeable for the causal question, either by design or under defensible assumptions. In the opening case, underlying risk affects both support assignment and deadline failure. Conditioning on contact therefore retains a path from contact backwards through risk and forwards to failure. Setting contact externally removes the historical assignment path while preserving the path from contact to outcome.

The assignment mechanism is part of the data

Machine-learning datasets often present treatment as another feature column. That representation hides how the value arrived. A clinician chose it, a queue threshold fired, a customer self-selected, a policy excluded ineligible cases, or an earlier model routed the case. This assignment process is not background decoration. It determines which potential outcomes become observable and which comparisons are credible.

In a graph, an intervention on T is represented by cutting incoming arrows to T and installing an external setter. It does not mean filtering records where T happened to equal the desired value. A customer who historically received support is evidence about the joint process “risk influenced support, then support and risk influenced outcome”. A customer assigned support by a random setter is evidence about a different process.

What prediction still earns

None of this demotes prediction. Baseline-risk models can define useful strata, improve precision, reveal heterogeneity worth testing, detect drift and allocate scarce measurement. Propensity models estimate how historical actions were assigned. Outcome models can serve as nuisance components inside doubly robust estimators. A predictor can therefore contribute materially to causal analysis without its feature coefficients becoming causal claims.

The distinction is about licence. A high-risk score may justify collecting more evidence or offering a choice when those actions are low consequence and independently governed. It does not establish that decreasing the score’s inputs will decrease risk. Some inputs are immutable causes, some are proxies, some are consequences, and some exist only because an earlier policy measured them. Feature attribution explains the predictor’s local dependence on its inputs. It does not, without a causal bridge, explain what changing the world would do.

This creates a useful division of labour. Prediction asks where the outcome is likely under the current regime. Causal inference asks which specified change would alter it. Decision theory asks whether the expected alteration is worth its cost and risk. A mature system preserves all three questions instead of forcing one score to answer them.

Worked example 1 · A reversal you can audit by hand

Consider 100,000 synthetic cases. Thirty per cent are high risk and 70 per cent low risk. Historical staff contact 80 per cent of high-risk cases and 20 per cent of low-risk cases. Among low-risk cases, failure is 20 per cent without support and 10 per cent with it. Among high-risk cases, failure is 60 per cent without support and 40 per cent with it.

Contact helps in both strata. The population average treatment effect is 0.7 × (−0.10) + 0.3 × (−0.20) = −0.13. Yet contacted cases fail at 28.9 per cent and uncontacted cases at 23.9 per cent. The pooled observational contrast is therefore about +5.1 points. The sign reverses because risk changes both contact assignment and failure.

If risk blocks every common-cause path from contact to outcome, the back-door adjustment is:

This formula is not a ritual instruction to control for every available column. It is valid because a causal story says risk is a pre-treatment common cause and is sufficiently measured. If contact changes a later variable and that variable helps transmit the benefit, controlling for it would erase part of the effect. If two causes both feed a selection variable, conditioning on that collider can create an association that did not exist. Graphical criteria make these distinctions explicit.⁴

Identification before estimation

Teams often start with an estimator: matching, inverse-propensity weighting, a causal forest, double machine learning, or a library function named estimate_effect. The order should be reversed. Identification asks whether the desired causal quantity is recoverable from the available design and assumptions. Estimation asks how accurately to compute an identified expression from finite, noisy data.

A crisp estimand for the running case might be: among cases eligible for proactive support, what is the 14-day average effect on missed-deadline probability of assigning one defined contact protocol rather than no contact? This sentence fixes the target population, two treatment versions, outcome, horizon and aggregation. “Does support work?” leaves each of them open.

For the simple average treatment effect, four assumptions carry most of the burden. Consistency requires the recorded treatment to correspond to the treatment named in the estimand. Exchangeability requires no unblocked common causes of assignment and outcome after conditioning on the selected covariates. Positivity requires each relevant covariate stratum to have a non-zero chance of each treatment. A no-interference assumption says one unit’s treatment does not alter another unit’s outcome, unless the estimand explicitly models that interaction. Hernán and Robins develop these conditions as part of the design of causal questions, not as estimator settings.³

Write the trial you wish had been run

A practical way to expose ambiguity is to specify the target trial before touching the observational table. Write the eligibility rules, treatment strategies, assignment time, follow-up start, outcome window, causal contrast and analysis. Then map each field to available data. The exercise does not turn observational records into a randomised trial. It reveals where the record fails to represent the desired experiment.

Timing is often the decisive defect. A treatment label may be recorded after risk has already evolved. Eligibility may be assessed using information that became available only after treatment. Cases that must survive long enough to enter a treated group receive an artificial advantage. A before-and-after comparison may combine the intervention with seasonality, staffing changes or a new measurement system. These are not tuning issues. They alter who is compared and which counterfactual is being estimated.

For the support example, time zero is the moment an eligible case first becomes assignable. Both strategies must be feasible at that moment. Follow-up begins immediately for both groups, even if contact is delivered later. Cases are analysed according to the specified assignment contrast, while treatment receipt is recorded separately. Emulating the trial disciplines the observational analysis by making every mismatch visible.

Randomisation is a design, not a magic word

Random assignment can make treatment independent of pre-treatment causes in expectation. In the synthetic support example, assigning contact with probability 0.5 in both risk groups makes the unadjusted difference converge to the −13-point causal effect. This is the article’s negative control: when selection no longer differs by risk, the predictive group contrast and the intervention contrast align.

Randomisation does not rescue an ill-defined treatment. “Contact” may mean a reminder text, a 20-minute call, a promise of assistance, or an action bundled with priority processing. Non-compliance changes the effect of assignment into something different from the effect of receipt. Spillovers violate unit independence. Attrition can recreate selection after randomisation. A sound trial protects assignment, but the causal claim still needs a treatment contract, outcome measurement and analysis plan.

Observational studies can identify effects under strong, inspectable conditions. Quasi-experiments exploit thresholds, instruments, policy changes or temporal discontinuities. Longitudinal methods handle time-varying treatment and confounding. The decisive point is that the study contains an identification argument. Randomisation is often the clearest such argument, not the only possible one.²

A powerful learner cannot infer a missing counterfactual

Suppose every high-risk case historically receives support and every low-risk case does not. The dataset has no untreated high-risk cases and no treated low-risk cases. A flexible neural network can fit observed outcomes perfectly, but it cannot learn the missing within-stratum contrasts from those data. Extrapolation will produce numbers because software must return something. The numbers are supported by model shape, priors or external knowledge, not by overlap in the observed sample.

Modern causal machine learning is valuable after this boundary is respected. Double or debiased machine learning uses flexible learners for nuisance functions, then combines orthogonal scores and cross-fitting to reduce regularisation bias in target-parameter estimation. Causal forests estimate treatment-effect heterogeneity while preserving inferential structure under their assumptions. Neither method identifies a causal effect merely by being sophisticated.⁸⁹

The practical sequence used by DoWhy is useful precisely because it keeps these stages apart: model assumptions, identify the estimand, estimate it, then attempt to refute the result with robustness tests.¹⁰ Refutation cannot prove that hidden confounding is absent. It can expose estimates that are fragile to placebo treatments, random common causes, subsets or alternative specifications.

flowchart LR
  A["The helpful action that<br/>prediction tells us to stop"] --> B["A prediction records a choice"]
  B --> C["Identification before estimation"]
  C --> D["Robustness is evidence, not<br/>licence"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#7B4CD8,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Robustness is evidence, not licence

A tempting reply is that causal machinery can be replaced by robust prediction. Train across many environments, retain only features whose relationships persist, and prefer models that survive distribution shift. This programme contains an important causal intuition. Peters, Bühlmann and Meinshausen showed that predictions based on causal parents are generally expected to remain valid across interventions and environments under stated structural assumptions, while non-causal predictors can fail when their inputs are manipulated.⁵

But invariance is evidence about a candidate mechanism, not a universal permission to act. Several non-causal relationships may remain stable across the environments sampled. A causal relationship can appear unstable if the treatment version, measurement process or effect modifiers change. Practical invariant-risk objectives can miss natural invariances, learn suboptimal predictors or become fragile under sampling, as negative analyses of IRM demonstrate.⁶⁷

Observation alone often leaves several graphs alive

Conditional independences can rule out causal structures, but observational data frequently identify only an equivalence class of graphs. Direction may require temporal knowledge, interventions, functional assumptions, non-Gaussian noise or external evidence. Causal discovery is therefore a proposal generator whose outputs inherit the assumptions of its search space and tests. It should not be treated as a machine that converts a dataset into ground-truth arrows.

The competing position matters here. A decision-theoretic or potential-outcome analysis may avoid claiming a complete structural graph and focus on a narrowly defined treatment contrast. A structural model may expose transport, mediation and policy counterfactuals more directly. The article does not require allegiance to one formal school. It requires that the data regime, counterfactual question and identifying assumptions be explicit enough to fail.

When prediction is exactly the right tool

Causal analysis is unnecessary for many valuable decisions. If the system only forecasts tomorrow’s workload and does not manipulate the predictors, a stable conditional model may be sufficient. If a policy has already been randomised in the target population and the operational action exactly matches the trial treatment, the difference in outcomes may estimate the effect without an elaborate adjustment model. If the cost of an action is tiny, reversible and measured through rapid feedback, an online experiment may dominate retrospective causal modelling.

This counterexample sets a useful boundary. Causation is required when the decision depends on changing the world, not whenever a model supports any decision. Forecasting can inform staffing without claiming that weather alerts cause demand. Classification can prioritise human review without claiming that the score changes the underlying case. The mistake begins when a variable that predicts an outcome is treated as an action whose modification is expected to produce that outcome.

An effect has an address

A causal estimate belongs to a population, treatment version, comparator, outcome definition, horizon and operating context. Moving it to another population is a transport claim. Bareinboim and Pearl formalise how experiments and observational data from heterogeneous sources can be combined under assumptions represented in causal structures.¹¹ In production terms, a validated effect under a staff-assigned workflow may not transport to model-triggered contact, a different channel, constrained capacity or a customer population that has adapted to the intervention.

Transport is not a final paperwork step. It can reverse the architecture decision. A treatment may help high-risk cases and waste attention on low-risk cases. A model may accurately estimate heterogeneous effects but deploy into a queue with insufficient capacity, causing delayed support for everyone. The policy effect then includes congestion created by the policy itself.

Release an intervention, not a score

The production failure is rarely a single statistical error. It is a chain in which a predictive score silently acquires authority. The model proposes that contacted cases have higher risk. A workflow interprets the proposal as “contact is harmful”. A policy suppresses contact. The released system changes assignment, world state and later training data. No component records that the deployed action differs from the historical variable whose coefficient motivated it.

A governed causal decision separates ten objects that compound systems often blur:

The causal model proposal belongs inside this chain; it is not the chain. A score can estimate baseline risk, treatment effect, expected utility or uncertainty. The deterministic authority kernel still decides whether the action is permitted, fresh and within release bounds. The outcome observer then measures what actually happened. A causal estimate without effect verification remains a hypothesis about a released system.

Worked example 2 · A production-shaped support release

Suppose the team now wants to automate support assignment. It defines treatment as a specific call protocol delivered within two hours by a trained service queue. The comparator is ordinary digital reminders. The target is eligible customers not already in a hardship or complaints pathway. The primary outcome is a missed deadline within 14 days; guard outcomes include complaints, repeat contact and queue delay.

Historical analysis suggests a −13-point average effect after risk adjustment. The team records why it believes the measured risk variables block relevant common-cause paths. It tests overlap and finds that some extreme-risk cases were always contacted. Rather than extrapolate, it excludes that region from automated policy and routes it to human review. This changes the claim from “support works for everyone” to “the effect is identified for the overlap population”.

The release uses a randomised canary within the eligible overlap population. Ten per cent of cases receive policy-assigned contact, ten per cent receive the comparator, and the rest stay on the existing human process. The action service binds the assignment to customer, purpose, protocol version and outcome window. If a request times out, a readback checks whether a contact task was created before retrying, preventing duplicate calls.

After assignment, the system records an effect receipt: task created, accepted, attempted and completed. Outcome measurement remains independent of the model that recommended contact. Guard thresholds stop expansion if queue delay, complaints or missed-deadline risk cross their limits. The next model version trains on data labelled by assignment policy, so historical staff selection and randomised policy assignment are not pooled as though they were one treatment regime.

This architecture does more than improve estimation. It preserves the difference between a model proposal and a world-changing action. It also makes unknown outcome a causal problem: if the system cannot tell whether contact occurred, it cannot interpret subsequent outcomes as treated or untreated without bias.

An average effect is not yet a case policy

The average benefit does not prove that every eligible customer benefits, nor does it reveal each person’s unobservable individual effect. A policy must decide who receives treatment under uncertainty. It may use estimated conditional effects, baseline risk, treatment cost, capacity and asymmetric harm. The resulting object is a policy, not a list of personalised causal facts.

A treatment-effect model can rank cases correctly yet produce a poor policy if it ignores queue congestion. Conversely, a coarse rule can have higher policy value when its eligible region has strong overlap and reliable execution. Offline policy evaluation also inherits the historical assignment problem: if the proposed policy chooses actions rarely or never observed for some cases, its value estimate depends on extrapolation.

This is why the release card asks whether the deployment setter lies inside the evidence claim. The relevant benchmark is not whether a treatment-effect model beats a predictive model on a proxy metric. It is whether the complete policy improves verified outcomes against a defined baseline while respecting authority, capacity and recovery limits.

The assignment-substitution audit

The card changes an architecture review. A model card can establish predictive performance. An assignment-substitution card establishes whether the proposed action is the action studied, whether the target population is supported, and whether the released system can create operating evidence. Both may be necessary. They answer different questions.

stateDiagram-v2
  [*] --> Claim
  state "Thought experiment 1 · The<br/>warning lamp" as Claim
  state "World A · direct effect" as Test
  state "World B · common cause" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Where this mechanism stops helping

The simple adjustment and playground cease to justify action when their causal structure is wrong. Hidden common causes can leave residual bias. Measurement error can turn an adequate variable into an inadequate proxy. Positivity failure leaves required counterfactual regions unsupported. Treatment versions can differ so much that consistency becomes fiction. Interference can make one unit’s outcome depend on other assignments. Dynamic policies can create time-varying confounding affected by prior treatment.

Production systems add further breaks. Capacity constraints mean treating more cases changes treatment quality. Users adapt to repeated interventions. The policy changes which outcomes become observed. Outcome labels arrive late or only for selected cases. Model-driven assignment can feed its own predictions back into the next training set. In these settings, a static average treatment effect may answer a valid but insufficient question.

The failure trigger is not “the estimate has uncertainty”. Every estimate has uncertainty. The trigger is that the available evidence no longer identifies the deployed policy effect under assumptions the team is willing to defend. The remedy may be a cluster trial, sequential experiment, instrumental design, explicit congestion model, narrower population, stronger measurement or abstention.

Compact glossary

Assignment mechanism

The process that determines which unit receives which action, including human judgement, rules, capacity and prior models.

Causal estimand

The exact counterfactual contrast sought, with treatment, comparator, population, outcome and horizon.

Exchangeability

A condition under which compared treatment groups can stand in for each other’s counterfactual outcomes, often after justified adjustment.

Intervention

A specified replacement of the ordinary process that sets a variable or action.

Overlap or positivity

Sufficient probability of each treatment within the covariate regions for which an effect is claimed.

Transportability

The argument that an effect identified in one population or system applies to another.

Approve the intervention, not the predictor

Prediction is disciplined compression of an observed regime. Causal inference begins when a decision proposes to replace part of that regime. The move from score to action therefore needs more than an accuracy threshold. It needs an intervention contract, an account of historical and deployed assignment, an identified estimand, tests that can expose failure, and operating evidence that the intended effect occurred.

The synthetic support case makes the consequence concrete. The historical predictor was correct about who failed. It was wrong as a guide to suppressing support because it carried the imprint of staff selection. Adjustment recovered the effect only because the common cause was measured and treatment overlap existed. Randomisation showed what happened when assignment was actually replaced. The no-overlap case showed where estimation had to stop.

The changed decision is simple: no predictive feature becomes an action merely because it is stable, important or well calibrated. Before release, compare the evidence-generating setter with the deployment setter. Where they differ, earn a causal bridge through design, assumptions and experiment. Where that bridge cannot be defended, keep the score in its proper role: prediction, triage or a prompt to gather better evidence.

A system that acts should record the whole causal wager. What was set, for whom, by whose authority, under which evidence, compared with what, and with which verified outcome? That record turns causation from a philosophical label into an engineering discipline. It also makes the system capable of the most important causal response: stopping when the world refuses to behave as predicted.