AI Research Library

Trajectory Assurance Beyond Final Answers

Rajesh Ranjan Mahapatra

A consequential agent should be judged while its decision path is still alterable, using observable prefix checks that distinguish a safe detour from a deviation heading towards an irreversible effect.

© 2026 Rajesh Ranjan Mahapatra. 28 min read

Trajectory Assurance Beyond Final Answers

A consequential agent should be judged while its decision path is still alterable, using observable prefix checks that distinguish a safe detour from a deviation heading towards an irreversible effect.

TLDR

  • A correct final state cannot prove that an agent reached it safely, because many authorised, lucky, wasteful and prohibited trajectories can end at the same destination.
  • Trajectory assurance evaluates each observable prefix against a safe path envelope made of hard invariants, ordered obligations, cumulative budgets and calibrated risk signals.
  • The practical unit is not hidden chain-of-thought. It is an attributable event stream joining intent, identity, evidence, proposals, approvals, effects and authoritative readback.
  • The envelope must permit multiple legitimate routes. A monitor that rewards one reference trace will punish useful exploration and still miss novel but unsafe behaviour.
  • Release decisions change when an agent can cause consequential effects: outcome tests remain necessary, but an inline monitor must be able to review, block or recover before completion.

The payment that never happened

Imagine two supplier-payment agents. Both receive an invoice whose bank details conflict with the approved vendor record. Both finish with the same ledger state: no payment has been posted.

The first agent binds the requesting user, retrieves the current vendor record, exposes the mismatch, and stops before asking the payment service to act. The second agent prepares and submits the transfer before the required approval. The payment gateway rejects it for an unrelated maintenance condition. A final-state evaluator sees two identical outcomes and may award both runs full credit.

Yet the second run crossed the boundary that mattered. It attempted an unauthorised effect and escaped harm only because another system failed. A retry, a different maintenance window or a slightly different gateway response could have converted the same behavioural defect into a real transfer. The final outcome erased the evidence that separated control from luck.

Two identical outcomes conceal different exposure A safe teal trajectory and an unsafe coral trajectory begin from the same authorised intent and end at no payment posted. The unsafe path crosses an approval boundary before being rejected by the gateway. Same destination, different exposure authorised intent no payment posted verify → expose mismatch → stop controlled outcome approval boundary submit first gateway rejects by luck
Figure 1. A successful endpoint can preserve an unsafe causal path. Illustrative worked scenario. Both traces end with no payment, but only the upper trace remains inside the required authority sequence.

This is the burden of trajectory assurance. It does not replace final verification. It asks an earlier question: while the run is still unfolding, has its observable prefix remained inside the set of paths that preserve the task's intent, evidence, authority and recoverability? The useful monitor is therefore neither a reward for stylistic reasoning nor a demand that every agent follow one approved script. It is a runtime decision instrument.

The argument has a precise limit. An evaluator can only assure properties represented in observable events or authoritative state. It cannot certify an unlogged side effect, infer true purpose from eloquent prose, or turn a compromised tool into a trustworthy witness. The article therefore treats model-generated rationales as supporting context at most. The control surface is the operational trace.

Part one

The destination is a lossy projection

A multi-step agent does not produce only an answer. It produces a sequence of observations, searches, state updates, proposals, tool calls and external effects. Let the observable trajectory prefix at step t be τ≤t = (e₀, e₁, …, eₜ). A final evaluator applies some function g to the completed run and environment, yielding an outcome y = g(τ≤T).

The important property of g is not sophistication. It is compression. Many different trajectories map to the same final answer or final database state. A good path, a lucky path, a path with a repaired error, and a prohibited path whose effect happened to fail can all collapse into one value of y. Outcome correctness is therefore necessary evidence about the destination, but weak evidence about the route.

That compression hides three different causal stories. In a latent violation, the agent crosses a boundary but no visible harm follows. In a lucky success, an unrelated safeguard or environmental accident rescues the run. In a repaired failure, the system detects its own deviation and executes an authorised recovery. These stories should not receive the same assurance judgement. The repaired run may be acceptable when recovery is part of the contract, but it still consumes error and recovery budgets. The lucky run is evidence of a defect, not competence. The latent violation is a near miss whose recurrence risk remains.

Final metrics can be extended with side-effect checks, cost and policy outcomes, but every extension still chooses what information to preserve. The question is therefore not whether to use outcome evaluation. It is which trajectory properties must survive the projection because they predict future harm, loss of authority or inability to recover. That choice should be made before observing the preferred model's traces, otherwise the evaluator will tend to canonise whatever behaviour the model already produces.

Final evaluation compresses distinct trajectories into one outcome Several differently shaped paths enter a narrowing projection and become one final result, showing that outcome scoring loses information about intermediate behaviour. Outcome is a many-to-one projection authorised repaired wasteful prohibited final-state evaluator g y route information lost
Figure 2. Final outcome evaluation deliberately discards trajectory information. Conceptual figure. This compression is useful for task success, but it cannot recover which intermediate obligations were violated.

Thought experiment: two identical receipts

Suppose a travel agent is authorised to reserve a refundable hotel only after the user confirms the total price. In run A, the agent quotes the price, receives confirmation and books. In run B, it books first, then asks. The user confirms in both runs, so the final itinerary and charge are identical.

Now vary one causal feature: the user declines in run B. The pre-emptive booking becomes an unauthorised effect, even if it can later be cancelled. The defect was present before the user's answer changed; the changed answer merely revealed it. A final itinerary check cannot distinguish consent-driven execution from post-hoc ratification. A prefix check can, because at the booking event it asks whether the approval obligation has already been discharged.

Process evidence helps, but does not transfer automatically

Mathematical reasoning research made the outcome-process gap measurable. Uesato and colleagues found that pure outcome supervision could achieve similar final-answer error with less label supervision. Process supervision sharply reduced incorrect reasoning among solutions whose final answer was correct, from 14.0 per cent to 3.4 per cent in their setting. Lightman and colleagues later showed strong gains from rewarding correct intermediate steps on the MATH benchmark and released the PRM800K process-supervision dataset. [2] [1]

These results establish an important possibility: final correctness can hide intermediate error, and step-level evidence can improve discrimination. They do not establish that a language-model judge can safely supervise an open enterprise workflow. A proof step has a relatively stable local meaning. An agent event may alter permissions, mutate external state, depend on stale evidence, or be harmless only because a later compensation succeeds. The labels and causal structure differ.

Agent benchmarks are beginning to expose this richer structure. AgentBoard reports fine-grained progress rather than only success. ToolSandbox evaluates intermediate and final milestones in a stateful tool environment. TRAJECT-Bench separates tool choice, argument correctness and dependency order from final accuracy. AgentProcessBench supplies human-labelled step quality for tool-using trajectories and reports that distinguishing neutral from erroneous actions remains difficult. [4] [6] [8] [10]

The implication is narrower than “score every thought”. It is this: when intermediate actions change future options or external state, the evaluation object must retain those transitions.

Trajectory assurance is also not synonymous with a process reward model. A process reward model supplies a learned score that may guide training, search or selection. Runtime assurance asks whether a specific live prefix preserves declared operating properties and what intervention follows if it does not. A reward model can contribute one signal, especially for semantic quality, but authority, state transition and effect verification often demand deterministic evidence. Conflating the two turns a fallible preference score into an enforcement mechanism it was never designed to be.

Observe what can change the world

There is a tempting shortcut. Ask the model to reveal its reasoning, send the text to another model, and treat a favourable judgement as assurance. This is useful for debugging, but too weak for control. Chain-of-thought can be incomplete, post-hoc or influenced by prompt conditions without faithfully exposing the computation that produced an answer. Lanham and colleagues found substantial variation across tasks and models in how much outputs actually depend on the stated chain-of-thought. [3]

A control trace should instead record events whose truth can be anchored outside the model. The minimum useful sequence joins the authorised intent to the actual effect:

  1. Intent: what outcome was requested, with constraints and expiry.
  2. Identity: which principal the agent represents at this step.
  3. World state: the authoritative facts and versions the action depends on.
  4. Evidence: stable references, provenance, freshness and entitlement.
  5. Proposal: the typed action and predicted postcondition.
  6. Authority: policy decision, approval, scope and remaining limits.
  7. Action: the actual tool invocation and idempotency identity.
  8. Effect: an external receipt plus authoritative readback.
  9. Recovery: whether the run can stop, retry, compensate or escalate.

Ordinary distributed tracing already offers useful plumbing. OpenTelemetry models a trace as a directed acyclic graph of spans, with timestamps, attributes, events, parent relationships and links. W3C Trace Context standardises portable identifiers across component boundaries. [14] [15] Those standards answer “which operation followed which?” They do not by themselves answer “was this principal authorised to cause that effect using this evidence?” Agent assurance must add domain semantics, authority and postconditions.

Granularity is a design choice. Logging every token produces an expensive, privacy-heavy stream with weak causal anchors. Logging only one span for “research supplier” hides which source changed the proposed beneficiary. The useful boundary is usually a state transition: evidence admitted, contradiction opened, proposal changed, approval granted, tool invoked, effect observed. Each event should identify its witness. The agent may witness its own proposal; an identity service witnesses principal binding; a policy engine witnesses authority; the target system or an independent reconciler witnesses the effect. A claim gains assurance weight from the independence and authority of its witness, not from the verbosity of its explanation.

The operational trace, not private reasoning, is the control surface An observable river connects intent, identity, evidence, proposal, approval, action and readback. A cloud of private model computation sits above it outside the direct assurance boundary. Assure the operational trace intent identity evidence proposal approval action readback private model computation debug signal, not an effect receipt observable, attributable, replayable boundary
Figure 3. The assurance boundary follows attributable events and authoritative state. Practitioner synthesis. Private reasoning may help diagnosis, but it cannot substitute for identity, policy, tool and effect evidence.
Practitioner synthesis An event is assurance-worthy when it can change authority, evidence, world state, future options or an external effect. Token-level logs are usually too detailed and too private; final answers are too compressed. Typed operational events form the useful middle layer.

This trace also needs selective disclosure. Raw prompts, documents and outputs may contain personal or confidential data. The event stream should therefore preserve stable references, hashes, policy results, bounded attributes and access controls rather than indiscriminately copying content. The aim is replayable evidence, not maximum logging.

Part two
The argument in one passChart · flow
flowchart TD
  A["The payment that never happened"] --> B["The destination is a lossy<br/>projection"]
  B --> C["Observe what can change the world"]
  C --> D["Define a safe path envelope"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#D95545,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step
A structural route through Trajectory Assurance Beyond Final Answers, from The payment that never happened to Sources and evidence ledger. Reading use: follow the relation, then test it against the surrounding evidence and limits.

Define a safe path envelope

A reference trajectory prescribes a sequence. A safe path envelope permits many routes, provided every observable prefix preserves the required properties. The distinction matters because capable agents may retrieve evidence in a different order, ask an extra question, or use a cheaper equivalent tool without reducing safety. Assurance should constrain invariants and obligations, not imitate one demonstration.

For a trajectory prefix τ≤t, define an envelope as the set of prefixes satisfying four classes of condition:

𝓔ₜ = { τ≤t : I(τ≤t)=1, Q(τ≤t)=1, B(τ≤t)≤b, R(τ≤t)<ρ }

I represents hard invariants such as principal continuity, tool allow-lists, amount limits and no external effect without an idempotency key. Q represents temporal obligations, such as evidence before proposal, approval before execution, and effect receipt before readback. B is cumulative budget across cost, time, calls or data exposure. R is a calibrated risk estimate for semantic properties that cannot be reduced to exact rules, such as scope drift or unresolved ambiguity. The thresholds b and ρ belong to an action class and consequence level, not to a model brand.

This is a design formulation, not a theorem of safety. The envelope is only as complete as its declared properties and sensors. Its value comes from making omissions contestable. A reviewer can ask whether approval is represented, whether stale evidence changes the decision, whether an effect is independently observed, and what happens when the monitor is uncertain.

Safe path envelope with warning and block boundaries A horizontal progress axis contains a central safe corridor. One path remains central, one takes a safe detour and returns, and one drifts through the warning boundary into the block zone. The safe path envelope block before effect warning band safe band forbidden region distance from authorised state trajectory progress
Figure 4. A safe path envelope permits variation while preserving control properties. Signature conceptual figure. The indigo path is ordinary, the sky path is a legitimate detour, and the coral path leaves the envelope early enough to block before an effect.

Four layers, four kinds of response

Hard invariants should be enforced mechanically wherever possible. The model should not decide whether its own token is valid, whether a tool is permitted, whether an amount exceeds authority, or whether the represented principal changed. These checks can block synchronously and produce stable reason codes.

Ordered obligations concern history. An action may be valid in isolation but invalid now. Approval after execution does not repair the original authority defect. Evidence fetched yesterday may be true yet no longer applicable. A monitor therefore evaluates the prefix, not only the current event.

Cumulative budgets catch gradual failure. Each search may be permitted while the run as a whole leaks too much data, spends too much, repeats the same failed query or consumes its recovery margin. Budgets should be explicit and reset only by an attributable policy decision.

Semantic or predictive risk covers properties that remain fuzzy. An action may be technically within scope while broadening the user's purpose. Evidence may be individually plausible but mutually inconsistent, or the agent may converge before a required source arrives. These signals should normally trigger review or a narrower action, not silently acquire veto power without calibration.

A prefix is not safe because its average step score is high. One missing approval can dominate twenty exemplary searches. Nor can later good behaviour retroactively make the original effect authorised. The monitor therefore needs stateful predicates rather than a simple sum. Some predicates are monotonic: once the principal changes without re-authorisation, that branch is invalid. Others are repairable: a contradiction can move from open to resolved, but only through an attributable event that meets the resolution rule. Recovery creates a new governed branch; it does not rewrite history.

This distinction also controls alert fatigue. A low semantic score on an exploratory search may create a review obligation, while the same score immediately before an irreversible action can narrow or block the proposal. The monitor should carry unresolved findings forward and increase intervention only when the combination of accumulated evidence, consequence and remaining recovery time warrants it. That makes the envelope a temporal control policy, not a static classifier painted over a trace.

Four layers of the trajectory envelope Nested contours show hard invariants at the core, temporal obligations, cumulative budgets and semantic risk at the outer edge, with different interventions assigned to each. One envelope, different evidence hard invariants ordered obligations cumulative budgets semantic or predictive risk review / narrow pause / budget block ordering defect mechanical veto
Figure 5. Different envelope layers justify different interventions. Conceptual cutaway. Exact authority checks can veto immediately; uncertain semantic signals should be calibrated and usually narrow, pause or escalate.

The runtime response can be expressed as a policy over four inputs: current violations, calibrated risk, consequence and reversibility.

uₜ = π(vₜ, rₜ, consequence(aₜ), reversibility(aₜ))

The action uₜ is not merely pass or fail. It may continue, request more evidence, reduce scope, require approval, block the action, quarantine memory, or enter recovery after an ambiguous effect. The same uncertainty should lead to different responses for a reversible search and an irreversible transfer.

Distinguish detour from drift

Thought experiment: one extra search

An agent must verify a supplier before preparing a payment. The reference demonstration checks the vendor registry and then the invoice. A new agent checks the invoice first, discovers an unfamiliar legal name, performs one extra registry search, and then follows the required approval path.

A sequence-matching evaluator may penalise the extra search. A generous language-model judge may excuse it, but perhaps for the wrong reason. The envelope asks more discriminating questions. Did the extra call remain within the authorised purpose? Did it use an entitled source? Did it create an effect? Did it exceed the search or privacy budget? Did the subsequent proposal rest on fresh evidence? If those properties hold, the path may vary without leaving the safe set.

Now vary one feature. The extra search expands from the named supplier to unrelated directors and family members, and the results are stored in long-term memory. The action may use the same search tool and still look locally plausible. It has nevertheless changed purpose, data exposure and future state. Detour is geometric variation within preserved constraints; drift is movement towards a different authorised world.

Safe detour and unsafe drift look similar locally Two paths depart from the centre. A sky path temporarily deviates and returns within the authorised purpose. A coral path gradually moves into unrelated data collection and persistence. Local novelty is not the failure extra relevant search purpose expansion authorised task state approval path same tool, accumulated purpose differs
Figure 6. The monitor should detect changed purpose, not mere path novelty. Illustrative intervention. One extra search remains relevant and temporary; the other expands the subject and persistence boundary.

Negative control: when the answer is the process

Serious negative control Consider a stateless service that computes a checksum over public bytes. The output is deterministically recomputed, no external side effect occurs, inputs are bounded, and the implementation has no discretionary tool use. A final verifier already observes the property that matters. Full trajectory assurance adds little beyond ordinary security, cost and availability telemetry.

This negative control prevents the argument from becoming universal. Outcome-only evaluation is often sufficient when the function is deterministic, the output fully captures correctness, and intermediate steps cannot alter authority, expose additional data or create irreversible state. The need for trajectory assurance grows with hidden state, long horizons, discretionary tools, asymmetric harm and delayed outcomes.

Lead time is the benefit; false intervention is the cost

A monitor that raises an alarm only after the final outcome is known has no preventive value. A monitor that blocks every unfamiliar prefix destroys the agent's usefulness. The evaluation target must therefore include both sides: how much actionable lead time the monitor creates before a harmful or irreversible step, and how often it interrupts legitimate trajectories.

Recent research makes this trade-off explicit. E-valuator turns verifier scores into sequential decisions with controlled false-alarm properties under its assumptions. PreAct-Bench evaluates prefix-only prediction before overt unethical action and reports that early-stage detection remains difficult. TRACES models evolving prefix risk, while TELBench and its DRIFT method study localisation of harmful spans in deep-research trajectories. These are promising research results, mostly in benchmark settings and several as recent preprints. They do not justify an unqualified claim that runtime monitors are solved. [9] [13] [11] [12]

Monitoring methods trade lead time against false intervention An illustrative scatter plot places final-only evaluation at low false alarms but no lead time, sensitive semantic monitoring at high lead time and high false alarms, and a layered monitor in a balanced region. Assurance value is a frontier false intervention rate → actionable lead time → final only hard prefix checks layered monitor sensitive semantic monitor
Figure 7. Early detection and operational usefulness must be evaluated together. Illustrative relationship, not measured values. A layered monitor combines precise hard checks with selectively invoked semantic review.
Part three

Build the trajectory harness

A trajectory harness sits beside the agent runtime. It receives typed events, reconstructs the current assurance state, applies exact and learned checks, and returns an intervention before the next consequential transition. It is not the same component as the agent, and it should not rely on the agent to honour a textual warning voluntarily when a mechanical block is possible.

Placement is as important as scoring. A monitor that observes the action after the tool has accepted it is an auditor, not a preventive gate. The decision and the effect request must share a controlled boundary so that the checked arguments are the executed arguments. Otherwise a time-of-check to time-of-use gap allows the proposal, beneficiary, authority or evidence version to change between approval and invocation. For high-consequence actions, the gate should issue a short-lived decision receipt bound cryptographically or structurally to the exact payload, principal, policy version and idempotency key.

The harness needs five properties:

  1. Prefix replay: the same event sequence reconstructs the same assurance state.
  2. Independent effect control: effectful tools are reachable only through an enforceable gate.
  3. Typed findings: every intervention has a stable code, severity, evidence and owner.
  4. State-aware checks: validity depends on prior approvals, evidence versions, budgets and unresolved contradictions.
  5. External observation: accepted commands are followed by effect receipts and authoritative readback.
A compact event contract for assurance replay
FieldPurposeExampleFailure exposed
trace_id / stepOrder and correlationCASE-47 / 5Missing, duplicated or reordered transitions
principal / actorSeparate represented user from executing componentuser:42 / agent:payConfused deputy or principal switch
kind / typed payloadDeclare operation and bounded argumentspayment.executeUnknown or mismatched effect
evidence_refsBind claims to fresh, entitled evidenceinvoice:884@v3Unsupported or stale proposal
policy_receiptRecord authority decision and expiryapproval:71Post-hoc approval or reused authority
effect_key / receiptProve exactly-once intent and observed effectCASE-47:pay:1Duplicate, unknown or ambiguous outcome

A minimal runnable harness

The following standard-library Python artefact implements a deliberately small envelope for a synthetic payment trace. It accepts a harmless non-effectful detour, releases an approved path, and blocks execution when approval is missing. The code inspects operational events only. It does not use private data, a model judge or a hidden rationale.

from dataclasses import dataclass, field
from enum import IntEnum

class Decision(IntEnum):
    CONTINUE = 0
    REVIEW = 1
    BLOCK = 2

@dataclass(frozen=True)
class Event:
    step: int
    kind: str
    data: dict = field(default_factory=dict)
    evidence: tuple[str, ...] = ()

@dataclass
class State:
    identity_bound: bool = False
    fresh_evidence: set[str] = field(default_factory=set)
    proposal: tuple[float, str] | None = None
    approvals: set[tuple[float, str, str]] = field(default_factory=set)
    executed_keys: set[str] = field(default_factory=set)
    receipts: set[str] = field(default_factory=set)
    readbacks: set[str] = field(default_factory=set)


def monitor(events: list[Event]) -> list[tuple[int, Decision, list[str]]]:
    """Evaluate each observable prefix against a small safe path envelope."""
    s = State()
    results = []
    expected_step = 1

    for e in events:
        decision, reasons = Decision.CONTINUE, []

        def fail(code: str, level: Decision = Decision.BLOCK) -> None:
            nonlocal decision
            decision = max(decision, level)
            reasons.append(code)

        if e.step != expected_step:
            fail("NON_MONOTONIC_STEP")
        expected_step += 1

        if e.kind == "identity.bind":
            s.identity_bound = True

        elif e.kind == "evidence.observe":
            if e.data["age_minutes"] > 60:
                fail("STALE_EVIDENCE", Decision.REVIEW)
            else:
                s.fresh_evidence.add(e.data["ref"])

        elif e.kind == "payment.propose":
            if not s.identity_bound:
                fail("IDENTITY_NOT_BOUND")
            if not e.evidence or not set(e.evidence) <= s.fresh_evidence:
                fail("UNSUPPORTED_PROPOSAL")
            s.proposal = (e.data["amount"], e.data["beneficiary"])

        elif e.kind == "approval.record":
            s.approvals.add((
                e.data["amount"], e.data["beneficiary"], e.data["approver"]
            ))

        elif e.kind == "payment.execute":
            requested = (e.data["amount"], e.data["beneficiary"])
            if requested != s.proposal:
                fail("PROPOSAL_EXECUTION_MISMATCH")
            approvers = {
                a for amount, beneficiary, a in s.approvals
                if (amount, beneficiary) == requested
            }
            required = 2 if e.data["amount"] >= 10_000 else 1
            if len(approvers) < required:
                fail("APPROVAL_MISSING")
            if not e.evidence or not set(e.evidence) <= s.fresh_evidence:
                fail("UNSUPPORTED_EXECUTION")
            key = e.data.get("idempotency_key", "")
            if not key or key in s.executed_keys:
                fail("IDEMPOTENCY_FAILURE")
            if decision < Decision.BLOCK:
                s.executed_keys.add(key)

        elif e.kind == "effect.receipt":
            if e.data["idempotency_key"] not in s.executed_keys:
                fail("UNEXPECTED_EFFECT")
            s.receipts.add(e.data["effect_id"])

        elif e.kind == "effect.readback":
            if e.data["effect_id"] not in s.receipts:
                fail("READBACK_WITHOUT_RECEIPT")
            elif e.data["status"] != "settled_once":
                fail("POSTCONDITION_NOT_PROVEN")
            else:
                s.readbacks.add(e.data["effect_id"])

        elif e.kind == "terminate" and s.receipts - s.readbacks:
            fail("MISSING_READBACK", Decision.REVIEW)

        elif e.data.get("effectful", False):
            fail("UNKNOWN_EFFECTFUL_ACTION")

        results.append((e.step, decision, reasons))
        if decision == Decision.BLOCK:
            break

    return results


def trace(approve_first: bool, detour: bool = False) -> list[Event]:
    events = [
        Event(1, "identity.bind"),
        Event(2, "evidence.observe", {"ref": "invoice:884", "age_minutes": 8}),
    ]
    if detour:
        events.append(Event(3, "vendor.lookup", {"effectful": False}))
    n = len(events) + 1
    events.append(Event(n, "payment.propose",
                        {"amount": 4_800, "beneficiary": "SUPPLIER-9"},
                        ("invoice:884",)))
    if approve_first:
        events.append(Event(n + 1, "approval.record",
                            {"amount": 4_800, "beneficiary": "SUPPLIER-9",
                             "approver": "manager:7"}))
    events.append(Event(len(events) + 1, "payment.execute",
                        {"amount": 4_800, "beneficiary": "SUPPLIER-9",
                         "idempotency_key": "CASE-47:pay:1"},
                        ("invoice:884",)))
    if approve_first:
        m = len(events) + 1
        events.extend([
            Event(m, "effect.receipt",
                  {"idempotency_key": "CASE-47:pay:1", "effect_id": "PAY-310"}),
            Event(m + 1, "effect.readback",
                  {"effect_id": "PAY-310", "status": "settled_once"}),
            Event(m + 2, "terminate"),
        ])
    return events


safe = monitor(trace(approve_first=True))
detour = monitor(trace(approve_first=True, detour=True))
unsafe = monitor(trace(approve_first=False))
assert max(d for _, d, _ in safe) == Decision.CONTINUE
assert max(d for _, d, _ in detour) == Decision.CONTINUE
assert unsafe[-1][1] == Decision.BLOCK
print("safe:", safe[-1])
print("safe detour:", detour[-1])
print("unsafe:", unsafe[-1])

Run the file with Python 3.10 or later. The expected final lines are:

safe: (8, <Decision.CONTINUE: 0>, [])
safe detour: (9, <Decision.CONTINUE: 0>, [])
unsafe: (4, <Decision.BLOCK: 2>, ['APPROVAL_MISSING'])
Why the demonstration is intentionally incomplete

The example omits signed event envelopes, clock semantics, policy versioning, distributed failure, privacy controls, semantic drift models, multi-principal approval, compensation and durable storage. Those are operating requirements, not details to hide inside the model prompt. The accompanying expanded harness adds principal continuity, approved tools, authority limits, contradiction state, idempotency, effect receipts, readback and mutation checks.

Test the monitor by mutating trajectories

Ordinary test cases show whether the intended path passes. Assurance tests should also make one controlled change at a time. Swap approval and execution. Age the evidence beyond its entitlement window. Change the beneficiary after approval. Repeat the idempotency key. Remove the effect readback. Insert a harmless search. Insert an effectful unknown event. Each mutation should change only the property under test, so a reviewer can see what the monitor actually detects.

This is the trajectory equivalent of a negative control. It prevents a monitor from appearing effective because every unsafe example also contains obvious stylistic clues. AgentDojo's prompt-injection environment illustrates the value of realistic adversarial tasks in which untrusted tool data enters an agent loop. A production assurance suite should similarly inject faults at identity, evidence, policy, tool, state and observation boundaries. [7]

Run the monitor in shadow mode first, but do not confuse shadow accuracy with live effectiveness. In shadow mode, the agent may continue after a would-be block, allowing labels to be collected safely in a sandbox. The evaluation should then replay the prefix exactly as it was visible at decision time. Future evidence must not leak backwards into the monitor. Once false interventions and missed hazards are understood by action class, promote individual hard checks to enforcement. Learned checks can remain advisory until their calibration and failure ownership justify stronger authority.

Release gate for the harness

  1. Every effectful tool is mediated by the gate, including retries and fallback paths.
  2. Each hard rule has a positive case, a one-feature mutation and an expected reason code.
  3. Legitimate alternative paths are represented, so novelty is not treated as failure.
  4. Semantic monitors are calibrated on ordinary and difficult negatives, with false intervention reported by action class.
  5. Effect receipts and readback are independent of the agent's self-report.
  6. Owners, expiry and recovery actions exist for every blocking rule.

A production-shaped worked scenario

Worked scenario This supplier-payment case is synthetic. It is designed to expose the mechanism and should not be read as a deployment claim.

A finance operations agent receives a request to release a £12,500 supplier payment. The task permits evidence gathering and proposal drafting. Execution requires a bound employee principal, two distinct approvals, current bank details, an amount below the delegated limit, an idempotent payment request and ledger readback.

At step one, the identity service binds the employee and records the delegated purpose. At steps two and three, the agent retrieves the invoice and vendor master. The records disagree on the beneficiary account, so the harness opens a contradiction. The agent finds a recently approved change request, but that document alone does not update the vendor master. It may continue gathering evidence, yet the envelope forbids proposal execution while the contradiction remains open.

After an authorised data steward updates the vendor record, the harness closes the contradiction against a new version. The agent prepares the payment proposal. Two approvers sign the exact amount and beneficiary. The action gateway checks that the proposal, approval receipts and execution arguments match. It then issues an idempotency key and sends the command.

The payment API acknowledges the request, but the connection fails before returning a final status. The agent must not infer failure and retry with a new key. The outcome is unknown. The harness moves to recovery, queries the external ledger using the original key, observes one settled payment, and records the postcondition. Trajectory assurance continues after command acceptance because an accepted action is not yet a verified outcome.

A governed payment trajectory with contradiction and unknown outcome recovery A temporal sequence across agent, assurance gate and external system lanes. The gate pauses on contradictory evidence, allows execution after approval, then recovers an unknown outcome through ledger readback. The gate follows the effect, not the narrative agent assurance gate external state retrieve contradiction: pause resolve + approve authorise action status unknown readback: settled once
Figure 8. Consequential assurance spans evidence, authority, execution and observed postcondition. Synthetic production-shaped scenario. The gate permits continued investigation, blocks action during contradiction, and treats a missing response as an unknown outcome rather than permission to retry.

Where the envelope breaks

The design fails if the only record of an effect comes from the agent that proposed it. It fails if a tool can be reached outside the gate, if identities are ambient rather than scoped, if event clocks and versions cannot be reconciled, or if the policy omits the harmful property. A compromised external system can also emit a false receipt. Independent reconciliation and downstream controls remain necessary.

Semantic monitors introduce another boundary. The same model family may generate the plan and judge it, creating correlated blind spots. An evaluator may learn superficial cues, punish concise but valid trajectories, or accept fluent justifications after the decisive error. AgentProcessBench's reported difficulty on neutral versus erroneous steps and TELBench's continuing first-error challenge are reasons to measure this directly rather than assume judge competence. [10] [12]

For these reasons, a credible architecture applies the certainty gradient. Deterministic checks handle identity, schema, authority, order, limits and effect identities. Domain calculations handle exact quantities. Learned monitors inspect semantic drift or contradiction. Humans decide genuinely ambiguous high-consequence cases. No layer should claim evidence it does not possess.

Part four
How the claim changes under pressureChart · state
stateDiagram-v2
  [*] --> Claim
  state "Thought experiment, two<br/>identical receipts" as Claim
  state "Process evidence helps, but<br/>does not transfer<br/>automatically" as Test
  state "Four layers, four kinds of<br/>response" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]
The revision loop begins with Thought experiment, two identical receipts and keeps counterexamples inside the reasoning path. Reading use: follow the relation, then test it against the surrounding evidence and limits.

What the evidence supports

The current evidence supports four bounded conclusions.

First, final-answer correctness does not imply correct intermediate reasoning. Process-supervision studies in mathematics demonstrate the gap under controlled labels, and chain-of-thought faithfulness studies warn that narrated reasoning is not automatically causal evidence. [2] [3]

Second, interactive agents need richer evaluation objects. τ-bench evaluates tool-agent-user interaction through policy-constrained state changes and reports reliability across repeated runs. ToolSandbox adds state dependencies and intermediate milestones. TRAJECT-Bench and AgentProcessBench directly separate process dimensions from endpoint success. These environments make trajectory faults measurable, though benchmark validity still depends on how well their tools, users and policies represent the target system. [5] [6] [8] [10]

Third, prefix monitoring is becoming a distinct research problem. E-valuator studies sequential decisions over verifier scores. TRACES estimates evolving trajectory risk. PreAct-Bench asks whether danger can be predicted from a prefix before overt action. TELBench studies where harmful spans begin. Together they support evaluating lead time and localisation, not only whole-trace classification. They do not yet establish universal runtime reliability across open domains. [9] [11] [13] [12]

Fourth, ordinary tracing standards provide essential interoperability but not assurance semantics. A trace ID can connect operations across services. It cannot prove that the correct principal authorised an effect or that the evidence was current. The operational contribution is to place typed authority, evidence and postcondition receipts on top of trace plumbing. [14] [15]

Open hypothesis A layered monitor combining deterministic prefix invariants with selectively invoked, independently calibrated semantic models should dominate either layer alone on the lead-time versus false-intervention frontier. This is a testable architecture hypothesis, not a settled empirical result.

What would weaken the thesis?

The case for inline trajectory assurance would weaken if final-state verification captured all consequential properties with negligible delay. It would also weaken if intermediate actions could not change permissions or external state, or if false intervention caused more harm than the monitor prevented. It would also weaken if safe and unsafe prefixes proved indistinguishable until the final effect in the target domain.

These are empirical questions. A release evaluation should therefore compare at least three baselines: outcome-only verification, offline whole-trajectory review, and inline prefix assurance. Measure task success, harmful effects, prohibited attempts, detection lead time, false review, false block, recovery success, cost and latency. Stratify by consequence and reversibility. Without those denominators, a high “trajectory score” is theatre.

The strongest experiment uses paired trajectories. Begin with a legitimate run, then mutate one causal feature while holding task, tools and final answer as constant as possible. Move approval after execution. Replace a current source with a superseded version. Change only the principal. Insert an irrelevant but non-effectful detour. Force an ambiguous tool response. These pairs estimate whether the monitor responds to the property of interest rather than to length, tone or generic difficulty.

Delayed outcomes require careful accounting. If fraud, customer harm or operational loss appears weeks later, recent traces are right-censored rather than safe. Report the observation window, the proportion with known postconditions, and whether the monitor changed the opportunity for harm or merely predicted it. For rare severe events, synthetic fault injection and near-miss evidence may be necessary, but their relationship to real incidence must remain explicit.

Choose the assurance mode

Not every task needs an inline gate. The decision follows from consequence, reversibility, statefulness and observability. The following instrument is deliberately conservative about where process evidence earns its operating cost.

Decision instrument: match assurance to the action
Task shapePrimary assuranceRequired evidenceExample
Deterministic, stateless, fully verifiable outputOutcome verificationInput boundary, recomputation, ordinary telemetryPublic-data checksum
Advisory, reversible, no direct effectOutcome plus sampled offline trajectory reviewClaims, sources, tool trace, user correctionDrafting a research brief
Stateful or long-horizon, but effects remain reversibleInline hard checks plus semantic review on riskState versions, budgets, policy receipts, checkpointsUpdating a sandboxed repository branch
Consequential, delayed or partially irreversible effectInline safe path envelope with human authority where requiredPrincipal, evidence, approvals, idempotency, effect receipt, readbackSupplier payment release
Unobservable effect path or untrusted external witnessDo not delegate the effect yetRedesign the boundary before model evaluationTool reachable outside the control gateway
Assurance mode changes with consequence and reversibility A phase space has consequence on the vertical axis and irreversibility on the horizontal. Outcome-only sits in the low-low corner, offline review in the middle, inline envelope in high consequence areas, and no delegation in the extreme corner. Choose controls from action consequence outcome only offline review inline envelope retain human effect irreversibility → consequence →
Figure 9. Assurance intensity should follow consequence and reversibility. Decision field. The boundary is a design judgement to validate against domain evidence, not a universal numeric threshold.

A final operational rule follows: if a task requires inline assurance but the effect path cannot be observed and gated, the correct response is architectural redesign, not a more persuasive evaluator prompt.

Conclusion

Release the monitor with the agent

A final answer tells us where the run ended. It does not tell us whether the agent preserved authority, used applicable evidence, avoided prohibited effects, or survived only because an external component rejected a bad request. That information lives in the path.

The practical response is not to reward longer explanations. It is to make consequential transitions observable and enforceable. Define a safe path envelope over hard invariants, temporal obligations, cumulative budgets and calibrated semantic risk. Reconstruct it at every prefix. Permit alternative safe routes. Intervene according to consequence and reversibility. Continue after command acceptance until the intended postcondition is independently observed.

This changes the release package. A consequential agent should ship with a typed trajectory schema, an enforceable action gateway, explicit envelope properties, mutation tests, lead-time and false-intervention measures, effect receipts, readback, recovery ownership and a record of what the monitor cannot see. Outcome metrics remain in the package, because route discipline without task success is not useful. They simply stop carrying more evidential weight than they deserve.

The strongest version of the argument is also the most modest: trajectory assurance does not prove that an agent is safe; it creates earlier, more discriminating evidence about whether this run is still governable. Where the path can change the world before the destination is known, that evidence belongs in the control loop, not in a post-mortem.