A failure belongs to a loop

The phrase “the AI failed” compresses several causal claims into one sentence. It may mean that a learned policy selected a poor action. It may mean that the goal was translated into the wrong operational objective. It may mean that the world presented inputs outside the stated operating conditions, or that an interface reported stale state. It may mean that the scoring procedure looked at the wrong evidence. These are different engineering problems. They require different repairs, owners and release decisions.

The four cuts used here are deliberately diagnostic:

1. The system is the implemented decision mechanism: model weights, prompt, rules, planner, tools, memory, wrappers, compute limits and stochastic settings.
2. The task is the normative contract: what outcome counts, which constraints apply, what actions are allowed, and when the attempt ends.
3. The environment is the causal process with which the system interacts: initial states, dynamics, observations, action semantics, exogenous events and interface behaviour.
4. The evaluator is the measurement process: sampled cases, labels or oracle, instrumentation, aggregation, thresholds and the moment at which evidence is observed.

A problem sits further upstream. “Reduce late deliveries” is a concern. It becomes a task only after someone chooses a target, constraints, population, horizon and acceptable trade-offs. A task might be “deliver 99 per cent of eligible parcels by 18:00 without entering restricted zones or exceeding a battery reserve”. That task can be instantiated in many environments: different maps, weather, traffic rules, sensors and command protocols. Several systems can attempt it. Several evaluators can judge the traces.

A small formalism makes the ambiguity visible

Let S denote the configured system, T the operational task, E the environment and ξ uncontrolled variation such as a seed or exogenous event. A run produces a trajectory τ. The evaluator V converts that trajectory into a verdict y:

Here τ contains the observations, proposals, actions, state transitions and effects that occurred. V may inspect all of that evidence or only a thin projection, such as final answer accuracy. A failure indicator might be F = 1 when y falls below a threshold or when any hard constraint is violated.

The causal problem is simple: many different tuples (S,T,E,V) can produce the same value of F. One failed run is therefore compatible with several explanations. More data from the identical configuration estimates how often the configuration fails, but it does not by itself localise the cause. Frequency is not attribution. Localisation requires a contrast in which one cut changes and the remaining cuts are held fixed or independently checked.

This paper’s contribution is a four-cut attribution method. It treats the cuts as intervention sites, not as claims that the world naturally arrives in four boxes. The method asks what must be specified, replayed and varied before a team spends money changing the learner.

Why the task-environment line moves

Reinforcement-learning formalisms often package transition dynamics, observations, rewards and terminal states inside an MDP. That is mathematically coherent and useful for algorithms [1]. The diagnostic cut in this article serves a different purpose. It separates what people require from what the world causes, because those two changes normally have different owners and repairs.

Three counterfactual questions locate the boundary. First, would the quantity change because a policy owner changed what ought to count? A delivery deadline tightened from 18:00 to 16:00 is a task change. Second, would it change because the physical or digital world behaved differently? A road closure or a service returning stale state is an environment change. Third, would only the recorded verdict change while the trajectory remained fixed? That points to the evaluator.

Some variables can move between cuts. A time limit may express a real business deadline, making it part of the task, or an experimental compute budget, making it external truncation. A human reviewer may be part of the environment when their response changes the next state, yet part of the evaluator when they judge a frozen trace. The boundary need not be universal, but it must be explicit and stable for the incident being diagnosed. Otherwise a team can “repair” a failure by silently redefining the task or moving an inconvenient case outside the environment envelope.

Four cuts through one result

The cuts are easiest to understand as four different kinds of question. The system cut asks, “Given the stated task and interface, did the implementation select an unsuitable action?” The task cut asks, “Did we operationalise the intended outcome and constraints correctly?” The environment cut asks, “Did the world and interface behave within the declared conditions?” The evaluator cut asks, “Did the evidence and scoring rule support the verdict we recorded?”

The system cut: did the implementation choose badly?

A system fault follows the configured decision mechanism. In the courier example, a constraint-blind planner chooses the shortest path through the restricted cell even when the task and action contract state the restriction correctly. Replacing the planner while keeping task, environment and evaluator fixed repairs the trajectory. That intervention is evidence for a system attribution.

The system boundary must include more than model weights. A prompt can drop a condition. A tool wrapper can coerce a decimal into an integer. A context compiler can omit the freshest document. A retry policy can duplicate an effect. A small model can be sound while the configured compound system fails. The diagnostic question is whether the fault follows the executable configuration.

The task cut: did we ask for the wrong thing?

A task is an executable interpretation of intent. It chooses the target, constraints, population, time horizon and permitted actions. Those choices are partly normative. No amount of model accuracy can recover a constraint that was never represented. If the intended rule says “reach the destination without entering restricted cells” but the operational task says only “reach the destination”, a direct path through the restriction may be perfectly optimal for the stated task.

Reward misspecification research makes this distinction concrete. AI Safety Gridworlds separates the reward visible to the agent from a hidden performance function used by the experimenter, allowing an agent to score well while violating the intended objective [6]. Inverse reward design treats a designed reward as evidence about a designer’s intent rather than a literal and complete objective [7]. Pan and colleagues further show that the effects of reward misspecification vary with model and environment conditions [8]. These are bounded research environments, but the lesson travels: the objective encoded for optimisation is not identical to the purpose that justified the system.

The environment cut: did the world honour the contract?

An environment is not just “the data”. It includes the transition process, observation semantics, action semantics, reset conditions, clocks, external actors and interface failures. The environment can change while the system remains byte-for-byte identical. A new traffic pattern can alter a controller’s state distribution. A document service can begin returning stale replicas. A simulator can map an action to the wrong actuator. A timeout can end an episode without the task reaching a terminal state.

This last distinction is operationally important. Gymnasium’s environment API returns separate terminated and truncated signals: termination belongs to the task’s terminal condition, whereas truncation can arise from an external limit such as a time cap [2]. Conflating the two changes bootstrapping logic and can therefore change learning results [3]. A single Boolean called done can hide whether the task ended or the experimental apparatus stopped observing it.

The evaluator cut: did the instrument deserve its verdict?

An evaluator selects evidence, encodes an oracle, chooses a denominator and aggregates cases. It can miss a real failure or manufacture one. A final-state scorer cannot detect a forbidden intermediate action. A stale answer key can reject a correct response. A benchmark that samples only easy environments can reverse the apparent ranking of methods. An aggregate score can hide a critical slice whose denominator is small but consequence is high.

Behavioural testing methods such as CheckList were motivated by the fact that aggregate test accuracy can conceal basic capability failures [12]. The Arcade Learning Environment’s revised evaluation protocols similarly show that sticky actions, episode handling and reporting conventions affect what an agent score means [13]. An evaluator is an instrument with failure modes, not a transparent window onto capability.

The evaluator contains a population and an oracle

Evaluation has two separable jobs. The sampling frame decides which environments and cases stand for the claim. The oracle decides what counts as success on each sampled case. A perfect oracle applied to an unrepresentative sample can still support the wrong release decision. A representative sample scored by a stale or shallow oracle can do the same.

Jayawardana and colleagues expose the first problem: a few chosen MDP instances can rank methods differently from the wider family [5]. Henderson and colleagues document sensitivity to implementation, seeds and reporting choices in deep RL [14]. These findings motivate two distinct evaluator interventions. Resampling changes the population while preserving the per-case judge. Rescoring changes the judge while preserving the cases and traces.

This separation prevents an easy mistake. When a method’s score falls after harder cases are added, the system may not have regressed at all; the claim’s denominator became more honest. When a trace-level constraint check replaces endpoint accuracy, the measured pass rate may fall because the instrument now sees more. A release gate should record whether a score change came from behaviour, population or oracle.

flowchart LR
  A["The courier that failed<br/>without changing"] --> B["A failure belongs to a loop"]
  B --> C["Four cuts through one result"]
  C --> D["Two separating experiments"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#007D68,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Two separating experiments

A useful thought experiment changes one causal feature at a time. The purpose is not to prove that a chosen cut is always independent. It is to make clear what evidence would distinguish one explanation from another.

Thought experiment one: the same courier, a different interface

Freeze the planner binary, task object, start state, random seed and evaluator. Run it in Environment A, where E=(+1,0) and S=(0,+1). Then run it in Environment B, where a versioning error swaps those commands. The system emits the same sequence in both environments. In A, it succeeds. In B, it stops in the wrong cell.

The variation is sufficient to make the failure follow the environment. It does not prove that the planner is ideal. It establishes a narrower fact: this observed difference does not require a change in model competence. A system repair that memorises the swapped semantics may restore the score, but the diagnostic repair is to restore or version the interface contract.

Thought experiment two: two judges, one trajectory

Freeze the system, task, environment and resulting trajectory. The robot crosses the restricted cell and reaches its destination. Evaluator A checks only the final state. Evaluator B checks the whole trace against the hard constraint. A returns pass; B returns fail. Nothing about the robot’s action changed. Only the evidence aperture changed.

This experiment separates evaluator quality from system behaviour. A stronger evaluator does not improve the trajectory; it improves what the organisation knows about the trajectory. That distinction matters in production because a better test can initially make a system look worse. The apparent regression is often a measurement correction.

The negative control: when the preferred explanation looks unnecessary

A critic might say that a capable system should infer the real intention, adapt to interface changes and compensate for weak evaluation. Construct the strongest case for that view. Give a frontier model the natural-language purpose, action documentation, examples of past interface migrations and permission to inspect its own trace. It may detect the mismatch and avoid the restricted state even when the formal task is incomplete.

That result would show that system capability can mask faults in other cuts. It would not make those cuts unnecessary. The same system might silently infer a different intention on another case, or use an undocumented workaround that the evaluator cannot see. More importantly, a compensating model does not repair the shared contract for other systems. Robust intelligence can reduce observed failures while increasing ambiguity about why the loop worked. The negative control therefore tests whether the four-cut method adds diagnostic value beyond raw task success.

Intervene before retraining

Attribution becomes defensible when a team can produce an intervention fingerprint. The idea is familiar from fault isolation: rerun the same case after one controlled repair. Replace only the system. Repair only the task. Restore only the environment contract. Rescore the unchanged trace with an independent evaluator. Record which changes flip the verdict.

This is stronger than asking people to vote on the root cause. It is also more modest than claiming a unique cause from one success. A one-cut repair can show that a change is sufficient for the case. It does not always show that the changed component was the sole cause. Two faults may compensate for each other, and several repairs may work.

For stochastic language-model agents, a one-cut replay must become distributional. Causal Agent Replay resamples a selected step under the same policy, runs the remainder repeatedly and measures the change in outcome with confidence intervals. Its point-of-commitment rule matters because an early intervention also re-rolls every stochastic step downstream: a large early effect need not locate the decision that committed the failure. This sharpens the protocol here. Compare outcome distributions, not one lucky counterfactual trace. The method also marks a hard boundary: noisy model judges and live tools with irreversible side effects can make faithful replay impossible.

1. Freeze the evidence. Preserve system version, task version, environment version, evaluator version, seed, inputs, observations, actions, timestamps and effects. Do not debug from a prose recollection.
2. Restate intent independently. Ask the accountable owner to write the required outcome and hard constraints without copying the implemented reward or existing test. This exposes circular validation.
3. Replay against the declared environment. Check observation units, action semantics, reset state, time, exogenous events, termination and truncation. Compare the live interface with the versioned contract.
4. Rescore the frozen trace. Use a second oracle, direct postcondition readback or trace-level property tests. Keep the system and environment untouched.
5. Run one-cut repairs. Change one cut at a time. Where replay is impossible, simulate the smallest counterfactual that preserves the other evidence.
6. Record the attribution set. Mark causes as supported, contradicted or unresolved. Assign a repair owner and a release consequence to each supported cut.

The four-cut attribution laboratory

The accompanying Python program implements the courier as a deterministic 3 × 3 environment. It uses standard-library dataclasses for the task, environment, trace and verdict. The baseline passes. Four single faults are then injected, one per cut. Each matching one-cut repair restores the pass. A negative control sends a constraint-violating trace to a weak endpoint evaluator, which falsely approves it.

The artefact assumes that one fault is injected at a time and that the intended task is available to the strict evaluator. Those assumptions make the causal logic visible. They are not guarantees about live incidents. The expected output is:

scenario            observed  one-cut repair         repaired  observed path
------------------------------------------------------------------------------------------------
baseline            PASS      none                   PASS      0,0 -> 0,1 -> 1,1 -> 2,1
system_fault        FAIL      replace system         PASS      0,0 -> 1,0 -> 2,0 -> 2,1
task_fault          FAIL      repair task            PASS      0,0 -> 1,0 -> 2,0 -> 2,1
environment_fault   FAIL      restore environment    PASS      0,0 -> 1,0 -> 1,1 -> 1,2
evaluator_fault     FAIL      replace evaluator      PASS      0,0 -> 0,1 -> 1,1 -> 2,1

negative control
weak evaluator:   PASS
strict evaluator: FAIL ('forbidden_state_visited',)

When the diagonal bends: interaction tests

The laboratory injects one fault at a time and verifies only the matching repair, so the displayed diagonal is evidence of sufficiency for those four cases, not evidence that cross-repairs fail. Live failures often arise from interactions. A brittle planner may succeed under the declared action map and fail only after an interface change. An incomplete task may appear harmless while a rich evaluator infers the missing constraint, then become visible when that evaluator is replaced. Either component can be repaired, yet the vulnerability belongs to the pair.

A small factorial experiment detects this. Choose two versions of the system, S₀ and S₁, and two versions of the environment, E₀ and E₁. Run all four combinations under the same task and evaluator. Let q(S,E) be the pass rate or loss. The interaction is the difference of differences:

If ISE is near zero within uncertainty, the changes act approximately additively. A large value means the effect of one cut depends on the other. The next decision is then joint: repair the interface contract and test the planner across the supported environment family. Do not force an interaction into a single-owner root cause simply because the incident system requires one dropdown value.

The same discipline applies to evaluator interactions. Hold the recorded trajectory fixed, score it with the original evaluator and a blinded replacement, then repeat after changing only the task statement. If the replacement reverses the verdict only for one wording, the failure cannot be assigned to either task or evaluator alone. Preserve all four cells, the scorer versions and disagreements rather than averaging them into one benchmark number.

Uncertainty belongs in this matrix. Repeat stochastic policies, use paired cases where possible and report the interval for each repair effect and interaction. A visually strongest cell is not a diagnosis when ordinary run-to-run variation spans the apparent gain. The permitted outcome is therefore sometimes “insufficient evidence”: freeze broad retraining, improve the experimental seam and rerun the discriminating intervention.

The strongest objection: everything can be called “the system”

A broad systems view can absorb prompts, task parsers, environment adapters and evaluators into one socio-technical system. That boundary is useful for accountability: someone must own the whole outcome. It is less useful for diagnosis. “The system failed” does not tell an engineer whether to retrain a policy, repair a requirement, roll back an interface or replace a label set.

The four cuts therefore answer an instrumental question. They create seams at which controlled variation, ownership and evidence differ. They do not deny interaction. A task compiler may itself be software. An evaluator may use a model. An environment adapter may be maintained by the same team as the agent. The relevant test is whether the seam enables a discriminating intervention and a different repair decision.

The explicit failure boundary

The method breaks when the cuts cannot be varied or checked independently enough to distinguish explanations. That happens when intent is genuinely contested, historical state cannot be reconstructed, the system adapts during replay, environment versions are unknown, evaluator labels were produced by the same model, or several components changed together. It also happens when a live human response becomes part of the environment and cannot ethically be replayed.

In those cases, attribution should remain set-valued. The incident record can say “supported: task or evaluator; unresolved between them” rather than forcing one root cause. Causal humility is operationally useful. It prevents a confident model patch from closing an incident whose measurement process is still untrusted.

From benchmark to operating contract

The four cuts matter most when a benchmark becomes a live workflow. A benchmark often freezes the dataset, action space and scoring script. Production reintroduces effective time, partial state, external services, authority, irreversible effects and people who can change the case while the system is reasoning. Without versioned boundaries, every incident becomes a story about the model.

Production-shaped worked scenario: the invoice exception

Consider a synthetic accounts-payable assistant. It reads an invoice, purchase order and goods-receipt record. It may propose pay, hold or review. A deterministic authority service decides whether the proposal can be executed. The intended outcome is to pay valid invoices promptly while preventing duplicates and payments without receipt evidence.

At 09:00, the source system records a receipt. At 09:04, a replica still reports “not received”. At 09:05, the assistant proposes hold. The evaluator uses the case’s final human disposition at 17:00 as the label and marks the proposal wrong because the invoice was eventually paid. An aggregate model report now contains a false negative.

Four explanations remain possible. The system may have ignored a fresh event already present in context. The task may have omitted the rule that stale receipt state should trigger review rather than hold. The environment may have breached a declared five-minute freshness guarantee. The evaluator may have compared a 09:05 proposal with a 17:00 label that incorporated later evidence. Each explanation implies a different control and owner.

The correct evaluation object is therefore not merely the final disposition. It is the time-indexed decision: “Given evidence legally and operationally available at 09:05, did the proposal follow the task and authority constraints?” The trace needs source versions, observation times, effective times, proposal, policy decision and eventual effect. World state, evidence history and remembered belief must remain separate.

Distribution-shift evidence reinforces this concern. Freshly collected ImageNet-like test data produced lower classifier accuracy than the original test set even after efforts to match the original collection process [11]. WILDS assembled naturally occurring shifts across domains precisely because an unchanged model can encounter a changed data-generating process [10]. Goal-misgeneralisation experiments show another boundary: an agent can receive the correct training reward and still pursue an unintended goal out of distribution [9]. Together, these results argue against treating every production miss as evidence that more in-domain optimisation is the first repair.

The environment specification

An environment specification makes the causal seam inspectable. It should be versioned beside the system and evaluator, not left in a design diagram. The specification need not model the entire world. It must state the parts of the world whose behaviour the task and system rely upon.

The four-cut attribution card

The following decision instrument is intended for an incident review, experiment plan or architecture decision record. A “yes” is not a final verdict. It identifies the next discriminating test. The card should be completed against a frozen trace and attached evidence, not from memory.

The card changes architecture as well as incident handling. It requires task versions, environment versions and evaluator versions to travel with the system release. It encourages direct effect readback rather than trusting an accepted command. It makes an unknown outcome distinct from a known failure. It also separates authority from capability: a model may propose an action, but task constraints and environment preconditions should be mechanically enforceable where consequence warrants it.

Diagnose the loop before replacing the learner

Machine-intelligence work often begins with a model and ends with a score. The causal object in between is larger. A configured system receives a task, acts in an environment and is judged by an evaluator. The same failure can be produced by any of those cuts, and the same apparent success can conceal a breach that the evaluator never observed.

The durable intuition is the courier’s unchanged command sequence. When the interface changes, the path changes without a change in policy. When the judge changes, the verdict changes without a change in path. Those contrasts show why a score alone cannot locate a cause.

The changed decision is simple: do not authorise retraining from an unattributed failure. First freeze the trace, restate intent, verify the environment contract, rescore independently and run the smallest one-cut repairs. Retrain when the failure follows the executable system after the other cuts survive. Repair the task when the optimisation target misstates the purpose. Restore or narrow the environment when live conditions breach the contract. Replace the evaluator when its evidence cannot support the verdict.

Some incidents will remain unresolved. That is not a failure of discipline. An explicit attribution set is more useful than a fictional root cause. It preserves what is known, identifies the next experiment and keeps the release decision tied to evidence rather than to the cultural habit of blaming the model.