Observational comparisons begin with unlike groups

A treatment group and a control group are rarely assembled by the clean random mechanism of an experiment. Patients offered surgery can be sicker, younger or more suitable for intervention than patients given drugs. Households that adopt a programme can differ in income, motivation and access from households that do not. A raw difference in outcomes therefore mixes the effect of treatment with differences that were present before treatment began.

Rosenbaum and Rubin begin with the potential-outcome structure used in Paper 0017. Unit i has a response R_1i under treatment 1 and R_0i under treatment 0. The average treatment effect is E(R₁) − E(R₀). Only one response is observed for each unit because assignment Z selects one treatment. Causal inference is therefore a missing-data problem. The unobserved response cannot be recovered merely by comparing each treated unit with any convenient control.

Let X be the vector of observed pretreatment measurements used to describe units before assignment. If treated and control groups had the same distribution of X, comparisons would not be confounded by those observed measurements. Exact matching on a high-dimensional X is conceptually direct but operationally brittle. With p binary covariates there are already 2^p exact cells; continuous covariates make exact repetition rarer still. The paper’s central move is to find a one-dimensional function that preserves the balance needed for design.

The scaffold and its exclusions

The original analysis treats the N study units as a simple random sample from a population and assumes that the unit index carries no further information once X is known. X contains observed measurements made before treatment assignment. It need not contain every variable used in real assignment, which is precisely why the later ignorability condition is an assumption rather than an empirical result.

The response R_ti is taken to be unique for unit i under treatment t. That excludes interference, such as one person’s treatment changing another person’s outcome, and excludes multiple hidden versions of “the same” treatment. Rubin had called this the stable unit-treatment value assumption. The paper states the limitation instead of pretending the two-potential-outcome notation covers every causal system.

The propensity score is a probability, not a personality

For binary treatment, the propensity score is

It is the probability that a unit with observed covariates x receives treatment 1 under the assignment mechanism being modelled. It is not the probability that treatment will work, a risk of the outcome, a moral entitlement to treatment or an intrinsic tendency carried unchanged across institutions. If hospital policy, eligibility or time changes, the assignment probability can change even for an otherwise identical patient.

A balancing score b(X) is any function of X such that, conditional on b(X), treated and control units have the same distribution of X. In Dawid’s notation from Paper 0019, X is conditionally independent of Z given b(X). The identity function b(X)=X is a trivial balancing score: exact covariate agreement balances covariates. The valuable question is whether a many-to-one function can do the same job.

The propensity score does. To see why, take all units with e(X)=.4. Every covariate configuration in this set has the same probability .4 of treatment. Assignment therefore cannot favour one of those configurations over another within the set. The mix of configurations among treated units must match the mix among controls in the population model. Units need not share the same X; they share the same assignment probability.

This is a statement about distributions, not about pairwise clones. A treated unit aged 30 and a control aged 70 can share a score because other covariates offset age in the assignment mechanism. Balance means that across all units at that score, the joint distribution of X agrees between treatment groups. Checking only the mean of each coordinate is weaker than checking the joint distribution, though mean and variance diagnostics are useful finite-sample summaries.

The first theorem balances; the second orders every balancing score

Theorem 1 states that treatment assignment and observed covariates are conditionally independent given the propensity score:

The result does not require multivariate normal covariates. It follows from the definition of e(X). Within a score fibre, Pr(Z=1 | X)=e(X) is constant, so knowing the remaining detail of X cannot further change treatment probability. By the symmetry of ordinary conditional independence, treatment status also supplies no further information about X once the score is fixed.

Theorem 2 characterises all balancing scores. A function b(X) is balancing if and only if it is finer than e(X), meaning there is a function f such that e(X)=f(b(X)). “Finer” means b retains enough information to recover the propensity score. Full X is finest; e(X) is coarsest. A refinement that combines e(X) with sex, region or another pretreatment feature remains balancing because it never mixes units that have different treatment probabilities.

The converse is just as important. Suppose b combines two covariate profiles with different propensities. Inside their shared b-cell, treatment favours the higher-propensity profile. Treated and control units then have different X mixtures, contradicting balance. Any valid balancing score must separate all profiles that e separates. This is the precise sense in which the propensity score is the smallest assignment-relevant compression.

Balance becomes causal only through strong ignorability

A balancing theorem says how observed covariates distribute. It does not yet say that outcomes under treatment and control are comparable. Rosenbaum and Rubin define treatment assignment as strongly ignorable given a covariate vector v when two conditions hold:

The independence component says that, within v, assignment carries no information about the two potential outcomes. The inequality is overlap or positivity: every covariate profile in scope has some chance of each treatment. Randomisation supplies both by design when implemented and recorded correctly. An observational study must defend them with subject-matter knowledge and design evidence.

Theorem 3 says that if assignment is strongly ignorable given X, it is strongly ignorable given any balancing score b(X), including e(X). The result joins two conditional-independence claims. Balance removes the association between X and Z within b; ignorability removes the association between potential outcomes and Z within X. Because b retains the assignment probability, integrating over X within b does not reintroduce assignment bias.

Theorem 4 then connects design to estimation. At a value b, the expected observed difference between treated and control responses equals the conditional average treatment effect:

Averaging these differences with the distribution of b in the target population yields E(R₁−R₀). The weights matter. Weighting by treated units targets the average effect among the treated; weighting by the full population targets the population average. Trimming regions of poor overlap can create a more credible comparison but changes the population to which the effect applies.

The population score is a theorem; the estimated score is a design object

The large-sample argument treats e(X) as known. In an observational study the assignment mechanism is usually unknown. Rosenbaum and Rubin therefore examine an empirical score when X takes only finitely many repeated values: the proportion treated among units with X=a. For any intermediate empirical proportion, treated and control units are exactly balanced on the repeated X value by construction. As samples grow, these proportions approach population balance under the stated sampling conditions.

With many or continuous covariates, most exact X patterns occur once. Their empirical treatment proportions are then zero or one, which is useless for adjustment. Some model is required. The paper points to logit and discriminant models. If X conditional on treatment is multivariate normal with a common covariance matrix, the propensity score is a monotone function of the linear discriminant. In a polynomial exponential family, the log-odds of treatment is a polynomial in X.

Those observations motivate models; they do not establish that a fitted logistic regression is correct. The score model should be revised by checking whether it achieves covariate balance in the intended design. A high predictive likelihood or classification accuracy answers how well treatment can be predicted. A causal design instead asks whether observed X is balanced after matching, weighting or subclassification and whether sufficient overlap remains. A model that predicts near-perfectly may expose the absence of counterfactual comparators rather than rescue the study.

Estimated scores also create uncertainty and finite-sample quirks. Two close estimates need not correspond to equal true propensities. Coarse subclasses contain score variation. Greedy matching depends on order; matching without replacement spends controls. A treatment model selected after looking at outcomes contaminates the design stage. The paper’s small-sample results are narrower than the modern slogan that “estimated propensity scores balance automatically”.

Three applications turn the score into a design

1. Matched sampling from a control reservoir

Matching selects controls whose observed covariate distribution resembles the treated group. It can reduce the cost of collecting outcomes from a large control reservoir, make comparisons legible and reduce reliance on response-model extrapolation. Exact matching on a balancing score has the population property required by Corollary 4.1 when strong ignorability holds.

Exact scalar matches are often unavailable, so practical procedures use nearest or caliper matches. Rosenbaum and Rubin analyse equal-per-cent bias reduction: a matching rule has this property when it reduces the bias in every coordinate of X by the same percentage. Under a condition that E(X | b) follows a common vector direction as a scalar function of b, matching on b alone has that property. This is a statement about bias geometry, not a guarantee for arbitrary algorithms or arbitrary outcome surfaces.

The paper reports a simulation inherited from Rubin’s earlier work. Fifty treated units were matched to fifty controls from reservoirs two, three or four times as large. With bivariate normal covariates, common covariance and parallel linear response surfaces, sample and population propensity scores removed similar percentages of bias. Larger reservoirs generally enabled better matches. The authors explicitly distinguish these approximate finite matches from Theorem 1’s exact population balance.

2. Subclassification without the 2^p explosion

Subclassification divides units into strata and compares outcomes within each. Direct adjustment then combines the stratum differences using target-population weights. If each of p binary confounders defined its own split, the design would require 2^p cells, many containing only one treatment. Stratifying a scalar score avoids this combinatorial explosion.

The paper demonstrates the idea in an observational study comparing coronary-artery bypass surgery with drug therapy. A logit score used clinical, haemodynamic and demographic pretreatment measurements. Five equal-sized score subclasses contained 303 patients each. From highest to lowest score, the numbers of surgical patients were 234, 164, 98, 68 and 26. Treatment prevalence varied greatly across strata, which is exactly what the score records.

Across 74 covariates, the distribution of F statistics showed dramatic improvement. Before subclassification the quartiles for the treatment main-effect statistics were 6.8, 10.9 and 16.8, with maximum 51.8. After subclassification they were .1, .2 and .6, with maximum 3.6. The treatment-by-subclass interaction statistics were also small. The authors caution that practical subclasses are not homogeneous in the score, so residual X bias can remain.

3. Covariance adjustment and the danger of the wrong axis

With a common sample covariance matrix, the point estimate from multivariate covariance adjustment for X equals an estimate based on the sample linear discriminant. This exposes a useful diagnostic: plot outcomes or residuals against the discriminant, or preferably against e(X), and inspect nonlinear response surfaces, nonparallel slopes and extrapolation.

If treated and control covariates are multivariate normal with a common covariance matrix, the discriminant is a monotone function of the propensity score. Outside that case it can be a poor approximation. With unequal variances, extreme low and high values of a covariate may both belong mostly to controls, so a linear discriminant cannot represent treatment probability. Linear covariance adjustment can then increase or overcorrect bias when response surfaces are nonlinear.

The lesson is broader than a contest between old estimators. Dimension reduction is safe for design only when it preserves assignment-relevant information. A visually tidy one-dimensional projection can conceal separation, nonlinearity or crossed response surfaces. The score is not sacred, either: it must be estimated, diagnosed and connected to a defensible causal structure.

A modern audit keeps the theorem and narrows the slogan

The enduring result is not “propensity-score matching makes observational data random”. It is a conditional theorem. The true propensity score balances observed X. Strong ignorability given X plus overlap transfers to a balancing score. A particular estimate, matching algorithm and outcome estimator must still approximate that population logic.

Measured balance comes first. Austin’s diagnostic work recommends comparing covariate distributions in the matched sample rather than relying on the treatment model’s fit. Standardised differences, variance ratios, empirical distributions and important interactions probe different failures. A non-significant covariate test can simply reflect a smaller matched sample; balance is a design property, not a null-hypothesis trophy.

Variable choice is causal. Brookhart and colleagues show that variables related to outcomes can improve precision even when weakly related to treatment, while variables strongly related only to treatment can worsen precision by pushing scores towards extremes. Post-treatment variables and colliders can introduce bias. “Include everything” and automated prediction are not substitutes for a temporal and causal covariate protocol.

Overlap constrains the estimand. Crump and colleagues formalise trimming when treatment probabilities approach zero or one. D’Amour and colleagues show why high-dimensional covariate sets make strict overlap increasingly demanding. Adding predictors can make unconfoundedness sound more plausible while simultaneously erasing credible treated-control comparisons. Trimming may improve precision and credibility, but the resulting effect pertains to the overlap population, not automatically to everyone.

One-to-one propensity-score matching is not the theorem. King and Nielsen criticise a common algorithm that greedily prunes pairs by scalar score distance. After it approximates complete randomisation, further pruning can increase imbalance on X, a pattern they call the propensity-score matching paradox. Their argument does not refute the balancing identity or productive uses such as stratification, weighting and diagnosis. It warns against treating a scalar distance and repeated pruning as universally optimal design.

Unmeasured confounding survives. The score is built from recorded X. If clinician judgement, latent severity or self-selection affects both assignment and outcomes but is absent from X, conditional exchangeability can fail even when every measured balance plot is immaculate. Negative controls, sensitivity analyses, quasi-experimental variation and domain evidence address parts of this debt; none is delivered by the score definition itself.

Balance does not identify an individual effect. The theorem supports average comparisons in populations and score strata under its assumptions. Two people with the same score may have different potential outcomes and treatment effects. The score organises comparability; it is not a personalised treatment rule. Using it to rank who “deserves” treatment mistakes an assignment model for a utility or benefit model.

An executable balancing-score laboratory

The laboratory uses the four equally common covariate cells from the worked examples. Twenty units occupy each cell. The true treatment probabilities are .2, .5, .5 and .8, producing forty treated and forty control units. Untreated means are 20, 50, 40 and 70; treatment adds exactly 10 in every cell.

Select the correct observed score, a misspecified age-only score, or a hidden-confounding case. The chart compares absolute percentage-point imbalances in age-risk and severity before and within score strata, then compares the raw and adjusted effect estimates with the true ten-point effect. The hidden case keeps perfect observed balance but shifts treated baselines by an unrecorded factor, demonstrating the boundary between balance and ignorability.

How to inspect it. Begin with the correct score and balance view. Raw age-risk and severity differences are both 30 percentage points, while exact score stratification reduces both to zero. Switch to effect view: 25 raw becomes 10 adjusted. Omit severity and observe residual within-score imbalance and an adjusted estimate near 19.89. Finally choose the hidden factor: observed balance still reads zero, but the adjusted effect is 22 because strong ignorability was deliberately violated.

The laboratory is exact arithmetic, not an estimator benchmark. It avoids sampling error and model-fitting noise so the logical roles remain separate. The correct case demonstrates the balancing and identification chain. The omitted case demonstrates score-model failure visible in X. The hidden case demonstrates causal failure invisible in X. Real studies face all three layers at once.

Memory studio

What survives the compression

The 1983 paper earns a result that is both narrower and more powerful than its popular shorthand. A scalar conditional assignment probability can carry exactly the part of observed X needed for population balance. That statement converts high-dimensional exact matching from an impossible ideal into a family of feasible designs. It also explains why methods that looked different, including discriminant matching, subclassification and covariance adjustment, can be understood through a common assignment axis.

The score’s power comes from a disciplined division of labour. The assignment model describes who receives treatment. The balancing theorem describes observed pretreatment comparability. Strong ignorability connects that comparability to potential outcomes. Overlap says both counterfactual arms are represented. Population weights specify whose average effect is being estimated. Removing any layer changes the claim.

This division is particularly useful in intelligence, agency and consciousness research, where observational assignment is rarely innocent. Systems are selected for evaluations, people self-select into interfaces, clinicians allocate interventions, and model policies route cases using latent and observed information. A score can help reconstruct the recorded assignment mechanism and expose where comparison is missing. It cannot establish that a behavioural report, architecture or intervention is causally decisive when the relevant latent states were never measured.

The deepest lesson is therefore not that many variables become one number. It is that compression is valid only relative to a mechanism and a target. The propensity score forgets distinctions that do not alter treatment probability. Whether those distinctions are also irrelevant to causal inference depends on the additional ignorability claim. The algebra tells us what is preserved; scientific judgement must justify why preservation is enough.