A world model is a wager about consequences

The useful idea behind a world model predates the current phrase. Sutton's Dyna architecture combined direct experience with updates generated from a learned model, allowing an agent to plan with hypothetical transitions while continuing to act.^[1] Ha and Schmidhuber later showed a compact controller trained inside a learned generative environment and transferred back to the task.^[2] PlaNet learned latent dynamics from pixels and planned over imagined action sequences.^[3] Dreamer trains behaviour from trajectories imagined by a recurrent state-space model, while MuZero demonstrates that a planning model need not reconstruct the whole observation if it predicts reward, value and policy-relevant quantities.^[4][5]

These systems differ substantially, but they share a functional commitment: some internal state is updated under possible actions and used to choose what to do. A minimal notation is:

z is the model state, a an action, e an environmental condition or regime and θ the learned parameters. Changing a should change only the descendants warranted by the transition mechanism.

This equation does not demand pixels, objects or an explicit causal graph. It demands a state sufficient for the decision and a transition rule that behaves correctly over the proposed choices. A chess model may ignore wood grain. A warehouse model cannot ignore whether a gripper is open, whether the object is supported and whether the arm is physically able to reach it. Representation is relative to the intervention and outcome, not to visual completeness.

Thought experiment: repaint the puck

Place two visually distinct pucks on a frictionless table. The blue puck is heavy and the red puck is light. Every training example preserves that pairing. A fixed impulse therefore sends red pucks farther than blue pucks. A sufficiently accurate video predictor can use colour as a compact substitute for mass and never represent the physical property separately.

Now run two matched interventions. First, repaint the heavy puck red without changing its mass, then apply the same impulse. The correct trajectory should remain the heavy-puck trajectory even though appearance changes. Second, replace the heavy puck with a light, visually identical shell. The correct trajectory should now change even though the initial pixels barely do. One intervention changes a correlate while preserving the mechanism-relevant property; the other changes the property while preserving the correlate.

A model that follows colour in both cases can remain impressive on ordinary held-out videos. It fails because its state does not support the comparison the planner needs. A model that follows mass across the pair has stronger evidence of intervention survival, but only for this family of impulses, masses and surfaces. The experiment does not prove that the latent variable literally means mass. It establishes a behavioural distinction: the predicted effect tracks the independently varied physical factor rather than its familiar visual proxy.

The boundary is equally important. If the operational task is only to sort pucks by visible colour, forcing the model to identify mass would add no decision value. The right latent factor is the one whose independent variation can change the permitted decision. Intervention design should begin from that decision, not from a wish to recover every cause in the scene.

Prediction and intervention answer different questions

A predictor estimates what usually follows under the process that generated its data. An intervention asks what follows after part of that process is deliberately replaced. The distinction can be written compactly:

The first conditions on an action that happened naturally. The second replaces the action-assignment mechanism while holding the stated background x fixed. They coincide only under identification assumptions.

In a fully observed Markov environment where actions are externally assigned and the training data covers the relevant state-action pairs, learning p(s' | s, a) can be enough. The action is effectively an intervention because the current state blocks relevant common causes and the assignment mechanism does not leak hidden information. This is one reason controlled simulators are powerful.

Real logs are less polite. A driver brakes because a hazard is present. A robot closes its gripper mainly when an object is centred. A clinician chooses a treatment after reading information unavailable to the model. A trading system changes its quote when latent order-book pressure rises. The action now carries information about hidden state. Conditioning on it is not equivalent to setting it.

World-model language sometimes hides this distinction by treating every action token as a causal handle. It is only a handle if the model has learned what changes when that handle is moved independently of its usual context. Causal representation research makes the related point that high-dimensional observations do not arrive with their decision-relevant variables already identified.^[8] Interventional data can add identifying information that observational dependence alone lacks.^[9]

The architectural conclusion is deliberately modest. A causal graph is neither necessary nor sufficient for intervention survival. A black-box latent model can produce correct paired effects over a bounded intervention set. An elegant structural model can fail if its variables are wrong, its graph is not identified, or the physical mechanism changes. The release criterion should therefore be behavioural and decision-linked, while architecture supplies hypotheses about why the behaviour may transfer.

flowchart TD
  A["The brake that exists only in<br/>familiar futures"] --> B["A world model is a wager about<br/>consequences"]
  B --> C["Why accurate predictors invent<br/>obedient worlds"]
  C --> D["Intervention survival as the<br/>evaluation object"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#D95545,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Why accurate predictors invent obedient worlds

Consider a robot trained from successful warehouse demonstrations. The gripper closes when the object is centred, the wrist is aligned and the operator expects contact. The camera sequence therefore contains a strong pattern: closing is followed by lifting. A model can minimise next-frame error by letting the action token stand for the whole precondition bundle. It may never represent contact, support, gripper width or object mass separately.

At deployment, the planner asks what happens if the gripper closes five centimetres to the left. The model has little evidence for that action-state combination. It may still generate the familiar lift because “close” was a reliable summary of successful picks in the logs. The imagined future is coherent, but the action has inherited causes that were present when humans chose it. The model has learned the policy's habits as if they were the world's dynamics.

Three ways the shortcut enters

First, action coverage is selective. The data-collection policy visits a narrow region of the state-action space. The model can be excellent on held-out samples from the same policy and undefined where a new planner wants to go. Random train-test splitting preserves the selection mechanism, so it measures interpolation inside the policy rather than response to changed choice.

Second, the latent state can discard causal variables. Representation learning preserves what lowers the training loss. If colour predicts mass in every training environment, a latent model may encode colour and omit mass. Reconstruction does not necessarily rescue it because colour is enough to reproduce the image. Reward prediction may also preserve the shortcut if the policy never breaks the correlation. The missing variable becomes visible only when colour and mass are independently varied.

Third, planning changes the input distribution adversarially. A planner searches for trajectories with high predicted return. Small model errors that are harmless under logged behaviour can become attractive opportunities. The planner repeatedly chooses actions that exploit them. Work on model-based policy optimisation formalises the trade between cheap synthetic rollouts and model bias, and finds value in short rollouts branched from real data rather than trusting long imagined trajectories indiscriminately.^[6] Recent cross-benchmark evidence also shows that adding Dyna-style synthetic rollouts can degrade performance across many environments, so more imagination is not automatically better.^[13]

Thought experiment: two identical films

Imagine two sealed laboratories that produce the same training videos. In Laboratory A, a robot arm lifts blocks through ordinary contact mechanics. In Laboratory B, the arm is disconnected; a hidden gantry moves each block in synchrony with the arm. Every observed frame, command and reward is identical. No learner can infer from those data alone which mechanism produced the films.

Now intervene by immobilising the arm while issuing the lift command. In A, the block remains still. In B, the hidden gantry may continue moving it. The observational distribution permitted both worlds. The intervention separates them. This is not a complaint that neural networks are insufficiently symbolic. It is an information limit: multiple generators agree on what was observed and disagree on what would happen after the assignment mechanism changes.

The baseline that should sometimes win

A serious test needs a case where the intervention thesis adds no practical value. Suppose a forecasting service predicts the next five minutes of temperature inside a tightly controlled chamber. The controller is fixed, the operating range is narrow, the forecast is advisory, the chamber has independent hard limits and no downstream optimiser can query novel controls. A simple autoregressive model may outperform a structured world model on cost, calibration and maintenance while meeting the decision need.

This negative control matters because “causal” can become prestige vocabulary. If the deployment policy will not change the action-assignment regime, consequences are low and out-of-support choices are mechanically blocked, intervention capability may be unnecessary. The observational baseline should remain in the benchmark. If it matches closed-loop outcomes across the pre-registered perturbation set, the heavier design has not earned its complexity.

The observation that would weaken this paper's central claim is therefore clear: a simple predictive model maintains low decision regret when the policy, initial conditions and relevant environment parameters are deliberately varied, while an intervention-oriented model adds no reliable gain. In that domain, the simple model has behaviourally survived the interventions, whatever its internal representation. The thesis concerns evidence, not architectural branding.

Intervention survival as the evaluation object

An ordinary benchmark samples cases and scores predictions. An intervention benchmark samples paired worlds. Begin with a base state and a proposed change. Generate or observe two trajectories that differ only through the intervention and its legitimate descendants. Then ask four separate questions:

1. Adherence: did the model actually apply the requested action or environmental change?
2. Invariant preservation: did non-target background conditions remain fixed within tolerance?
3. Differential correctness: did the outcome change in the right direction and magnitude?
4. Decision validity: does using the model improve policy choice or reduce regret when actions are executed?

The first three diagnose simulation behaviour. The fourth checks whether those gains matter to the agent. A model can pass local paired tests and still fail planning because uncertainty compounds, rewards are mis-specified or the planner searches outside the tested region. Conversely, a task-sufficient model may support excellent decisions without producing human-readable videos.

The score is a vector, not a beauty average

Let the control branch predict outcome ŷ0 and the intervention branch predict ŷI. Let the matched reference pair provide y0 and yI. The quantity that matters first is the effect error:

The subtraction removes shared conditions and exposes whether the predicted change matches the observed or simulated change. The norm must be chosen for the outcome: metres for stopping distance, degrees for pose, probability distance for a distribution, or task loss for a decision.

Effect error cannot stand alone. An invariant error should separately measure drift in variables declared unchanged, such as road geometry, object identity or background agents. Adherence checks that the requested intervention occurred. A support or calibration term records whether the model knew the pair lay beyond its evidence. Closed-loop regret then tests whether the planner chose better actions when these predictions were used.

Collapsing the components into one smooth score can recreate the original problem. A video model might receive a high visual-quality score that compensates for ignoring the action. Another might execute the intervention but redraw the entire scene. A third might predict the correct local difference yet remain confidently wrong on an unsupported branch. Release should use a conjunctive gate: adherence, invariants, effect, uncertainty and decision outcome must each clear a declared threshold. A composite number is useful for ranking only after the safety-relevant floors are met.

Suppose a manipulation prompt moves a target block five centimetres left. The generated branch moves it by exactly five centimetres, but also shifts the camera, changes the neighbouring block and rotates the table texture. A single semantic score may call the result responsive. The paired vector says something sharper: adherence passed, target effect passed, invariant preservation failed. That diagnosis tells the engineering team whether to improve conditioning, state factorisation, data collection or the evaluator itself.

Name the intervention family

“Robust to intervention” is too broad to test. At least four families should be separated. An action intervention changes the agent's command while keeping the environment mechanism fixed. An initial-state intervention moves an object, changes a starting speed or alters inventory. A parameter intervention changes friction, mass, latency or sensor noise inside the same structural mechanism. A mechanism intervention replaces the transition law itself, such as swapping a gripper, changing traffic rules or disabling an actuator.

The distinction matters because success need not transfer upward. A model trained on varied actions may still fail when friction changes. A model trained across friction values may fail when the direction of actuator response reverses. Mechanism changes require adaptation, model selection or abstention rather than confident extrapolation.

CausalWorld was designed to expose this granularity in robotic manipulation by allowing interventions on factors such as object mass, colour and size, enabling controlled transfer distributions.^[7] Causal Triplet similarly emphasises actionable counterfactual settings and interventional downstream evaluation for high-dimensional representations.^[12] These environments do not solve real-world causality, but they make the evaluation contract inspectable.

A particularly direct recent test is What-If World, a 2026 preprint that builds 319 paired prompts from driving and robotic-manipulation frames, varying one physical detail and scoring adherence, physics, environment preservation and outcome. Across nine tested video models, none exceeded 52 per cent on the paired score; visually subtle interventions were especially difficult.^[11] The result is preliminary and benchmark-specific, but it demonstrates why independent single-video scoring can miss intervention failure.

A minimal benchmark that exposes the gap

Now reduce the problem to three numbers. A toy vehicle has current velocity v, commanded action a and terrain t. Its next velocity is generated by:

ε is zero-mean noise with standard deviation 0.08. The action accelerates the vehicle; terrain adds an opposing effect.

The logged behaviour policy always sets a = t. Action and terrain are therefore perfectly collinear. In the log, their combined contribution is only 0.5a. A least-squares model using both features can fit the held-out data while having no information about how to divide that observed effect between action and terrain. The minimum-norm solution assigns roughly 0.25 to each.

A second model receives the same amount of data, except that 25 per cent of actions are randomised. Those interventions break the collinearity. The fitted coefficients recover approximately 1.40 for action and -0.90 for terrain. On ordinary held-out logs, both models obtain 0.082 root-mean-square error. The familiar benchmark declares a tie.

Reverse the action assignment so that a = -t. The observational model's error rises to 2.301. The intervention-trained model remains at 0.081. Randomise actions more broadly and the errors are 1.650 versus 0.080. The observational model did not become worse; the test finally asked a question its data had never identified.

The final row is essential. The mechanism-changing test alters the coefficients themselves, including reversing the action effect. The intervention-trained model now performs poorly too. It had learned the original mechanism across varied assignments, not a law guaranteed under arbitrary redesign. This prevents a common overclaim: intervention data extends the identified envelope; it does not abolish the envelope.

Theory provides a complementary, carefully bounded result. Richens and Everitt show that, under their assumptions and over a sufficiently large family of local distributional shifts, an agent achieving regret bounds must encode an approximate causal model of the data-generating process.^[10] The theorem supports a connection between broad robust adaptation and causal structure. It does not prove that any successful benchmark agent contains a readable causal graph, nor does it directly cover every mediated sequential-control setting. Behavioural evidence and assumptions remain part of the claim.

A release gate for world models

Consider a production-shaped robotic picking system. A vision encoder proposes a latent state. A world model rolls out candidate gripper motions. A planner selects a trajectory. A deterministic controller enforces joint limits and collision envelopes. Cameras and force sensors read back the effect. The model is not the authority to move; it is a proposal engine inside a larger control system.

The evaluation should begin from the decisions the model is permitted to influence. Suppose it may rank grasps but cannot exceed a low-speed envelope. The intervention suite should include action changes, object displacement, weight changes, partial occlusion, slippery surfaces, delayed sensor frames, failed suction and altered lighting. Each case needs a control branch, an intervention branch, expected invariants, acceptable uncertainty and a physical postcondition.

The intervention readiness card

The following instrument turns “good world model” into a reviewable release claim. It should be completed for a model version, task and operating envelope, not for a foundation model in the abstract.

Four release levels

A useful release vocabulary is more honest than a binary “world model” label. A renderer generates plausible continuations but is not used to compare actions. A replay predictor forecasts under the existing policy and is evaluated only inside its logged regime. A decision simulator passes paired interventions for a named family and may compare bounded choices. A planning model additionally demonstrates closed-loop value under search, uncertainty and replanning.

Promotion should depend on the highest level needed. A synthetic-data generator may remain a renderer. An anomaly forecaster may remain a replay predictor. A robot planner needs the stronger levels. Do not make every model prove more than its job requires, and do not let a lower-level success borrow the language of a higher one.

The failure boundary: the mechanism can move

Matched intervention tests assume that a meaningful part of the world can be held fixed while one assignment changes. This modularity is often a useful approximation, but it is not guaranteed. A human driver may react to the ego vehicle's braking. A flexible object may change contact geometry as it deforms. A market participant may adapt after detecting the agent's policy. The intervention then changes the environment through channels omitted from the model, so the reference pair itself needs a wider boundary.

Time creates another break. Wear, software updates, seasonal conditions and new operating rules can alter a transition mechanism while surface observations remain familiar. A model validated across several friction values has evidence for parameter variation inside one law. It does not automatically cover a tyre replacement that changes how braking force is transmitted, or a controller update that reverses an actuator convention. Parameter robustness must not be reported as mechanism robustness.

The release envelope therefore needs invalidation triggers, not only an accuracy monitor. Useful triggers include a new hardware or policy version, persistent residual structure after an action, changed actuator-response timing, disagreement between independent sensors, a rise in unknown outcomes, or failure of an invariant that previously held. When a trigger fires, the system should shorten its horizon, gather discriminating evidence, select another model, fall back to a bounded controller or abstain. Continuing to produce fluent futures is not a recovery strategy.

Some interventions cannot be validated safely or often enough. A catastrophic collision, a rare medical complication or a large market move may have too little direct evidence for strong claims. Simulation and digital twins can organise hypotheses, but their own mechanisms require validation. In those regions, the model may support reversible probes or human analysis without acquiring authority over the irreversible choice. The correct release decision can be a narrower action set rather than a more confident model.

Operational controls that follow from the argument

First, retain real observations as anchors. Replan frequently from fresh state rather than extending an unverified latent narrative. Second, cap horizon and action novelty according to measured error, not a fixed aesthetic preference. Third, use ensembles or other uncertainty estimates to detect unsupported branches, while remembering that agreement among similarly trained models can remain wrong. Fourth, inject failures into sensors, actions, state estimation and effect readback. Fifth, record which model version, intervention manifest and authority boundary supported each consequential proposal.

For irreversible outcomes, the planner should not gain authority merely because its simulated return is high. A deterministic authority kernel can constrain identities, preconditions, effect limits and permitted actions. The live system must then verify the physical postcondition. A command accepted by a robot controller is not evidence that the object was grasped. An API response is not evidence that the world changed as intended.

The competing design may be a reactive policy, a model-free controller, a rules engine or a human operator. Compare them on the same outcome and perturbation suite. World models earn their cost when safe interaction is scarce, look-ahead materially improves decisions, the intervention space can be covered and model uncertainty changes behaviour. They lose when dynamics change faster than validation, action effects are not identifiable, consequences are too high for imagined evidence, or a simpler controller already satisfies the operating objective.

stateDiagram-v2
  [*] --> Claim
  state "Thought experiment, repaint<br/>the puck" as Claim
  state "Prediction and intervention<br/>answer different questions" as Test
  state "Three ways the shortcut enters" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Paired intervention benchmark

The following Python programme reproduces the minimal worked example. It trains one linear transition model on perfectly confounded logs and another with 25 per cent randomised actions. It then evaluates a familiar-policy negative control, two positive intervention cases and a mechanism-changing failure boundary.

Run with Python 3.10 or later and NumPy. The expected output includes a 0.082 versus 0.082 tie on held-out logged behaviour, followed by 2.301 versus 0.081 RMSE on the reversed-action intervention. Small last-digit differences can occur across NumPy versions.

#!/usr/bin/env python3
"""
Paired intervention benchmark for a tiny learned world model.

The environment contains:
  v: current velocity
  a: commanded action
  t: terrain indicator
  y: next velocity

True training-regime dynamics:
  y = 0.8*v + 1.4*a - 0.9*t + noise

In observational logs, the behaviour policy always chooses a == t.
That makes action and terrain perfectly collinear. A least-squares
transition model can fit held-out logs while being unable to identify
what action itself changes.

A second model receives 25% randomised actions. This small intervention
breaks the collinearity and makes the action effect identifiable.

Requirements: Python 3.10+ and NumPy.
"""

from __future__ import annotations

from dataclasses import dataclass
from typing import Literal

import numpy as np

SEED = 7
NOISE_STD = 0.08


@dataclass(frozen=True)
class Batch:
    x: np.ndarray
    y: np.ndarray


def sample_batch(
    rng: np.random.Generator,
    n: int,
    *,
    intervention_rate: float = 0.0,
    policy: Literal["logged", "reversed", "random"] = "logged",
    coefficients: tuple[float, float, float] = (0.8, 1.4, -0.9),
) -> Batch:
    """Generate synthetic transitions under a chosen action-assignment regime."""
    terrain = rng.choice(np.array([-1.0, 1.0]), size=n)
    velocity = rng.normal(0.0, 1.0, size=n)

    if policy == "logged":
        action = terrain.copy()
        mask = rng.random(n) < intervention_rate
        action[mask] = rng.choice(np.array([-1.0, 1.0]), size=int(mask.sum()))
    elif policy == "reversed":
        action = -terrain
    elif policy == "random":
        action = rng.choice(np.array([-1.0, 1.0]), size=n)
    else:  # defensive guard for future edits
        raise ValueError(f"Unknown policy: {policy}")

    beta_v, beta_a, beta_t = coefficients
    noise = rng.normal(0.0, NOISE_STD, size=n)
    next_velocity = (
        beta_v * velocity
        + beta_a * action
        + beta_t * terrain
        + noise
    )

    # The final column is an intercept.
    features = np.column_stack(
        [velocity, action, terrain, np.ones(n, dtype=float)]
    )
    return Batch(features, next_velocity)


def fit_transition(batch: Batch) -> np.ndarray:
    """Fit y ~= beta_v*v + beta_a*a + beta_t*t + intercept."""
    beta, *_ = np.linalg.lstsq(batch.x, batch.y, rcond=None)
    return beta


def rmse(beta: np.ndarray, batch: Batch) -> float:
    residual = batch.x @ beta - batch.y
    return float(np.sqrt(np.mean(residual**2)))


def main() -> None:
    rng = np.random.default_rng(SEED)

    observational_train = sample_batch(rng, 3_000, intervention_rate=0.0)
    interventional_train = sample_batch(rng, 3_000, intervention_rate=0.25)

    observational_model = fit_transition(observational_train)
    interventional_model = fit_transition(interventional_train)

    tests = {
        # Negative control: the familiar behaviour policy.
        "held-out logged policy": sample_batch(rng, 2_000, policy="logged"),
        # Positive intervention: action assignment is deliberately reversed.
        "reversed action intervention": sample_batch(
            rng, 2_000, policy="reversed"
        ),
        # Broader action coverage under the same mechanism.
        "random action intervention": sample_batch(rng, 2_000, policy="random"),
        # Failure boundary: the physical mechanism itself has changed.
        "mechanism-changing shift": sample_batch(
            rng,
            2_000,
            policy="random",
            coefficients=(0.5, -0.4, -1.2),
        ),
    }

    print("coefficients [velocity, action, terrain, intercept]")
    print(
        "observational:",
        np.array2string(observational_model, precision=3, floatmode="fixed"),
    )
    print(
        "interventional:",
        np.array2string(interventional_model, precision=3, floatmode="fixed"),
    )
    print()
    print(f"{'test':32s} {'observational':>14s} {'interventional':>15s}")
    for name, batch in tests.items():
        print(
            f"{name:32s} "
            f"{rmse(observational_model, batch):14.3f} "
            f"{rmse(interventional_model, batch):15.3f}"
        )


if __name__ == "__main__":
    main()