Model-Risk Evidence for Agentic Systems: From Outcome Accuracy to Trajectory Assurance

The banking case is a worked example. Its figures make the operating model concrete; they are not results from a named deployment and need local calibration.

Outcome correctness does not establish process soundness

A model-risk function evaluating a conventional predictive model asks a bounded question: given this input, is the output accurate, and is that accuracy stable across the population served? An agent creates a second question. Even when the final output is correct, did the system arrive there through an authorized and supportable process?

A correct answer can conceal an unauthorized data access, a skipped verification or an intermediate claim that would fail under a small change in facts. Outcome-only evaluation cannot see this distinction because it inspects the destination rather than the route.

Consider the worked anti-money-laundering composite used throughout this paper. The agent's final escalate-or-clear disposition matches the human-adjudicated label on 94 percent of a held-out set. Step-level review then finds that, among the correctly escalated cases, roughly one in six traces cites a risk factor absent from the retrieved case data. The final label is right despite an ungrounded step, not because the process is sound.

Those percentages are illustrative inputs to the operating model, not reported results from a named pilot. The point is the diagnostic difference: outcome accuracy would miss the unsupported step, while a model-risk review needs to find it before approving a use case with regulatory consequence.

Process reward models motivate the broader discipline of trajectory-level assurance, but the enterprise claim must be narrower: not just was the destination correct, but did the observable evidence, tool, policy, approval and effect events support the route taken?

Research foundation

The research grounding for step-level evaluation comes most directly from mathematical reasoning, where proofs provide a clearer notion of a valid intermediate step. Lightman et al.'s “Let's Verify Step by Step” (2023) found process supervision more effective than outcome supervision for selecting correct solutions in the evaluated mathematics setting, with a larger advantage on harder problems. Uesato et al.'s earlier study (2022) similarly examined process and outcome supervision in mathematical reasoning. These results establish neither bank-grade assurance nor a universal evaluator; they provide the transferable distinction between grading the route and grading only the destination.

The useful insight is simple. Outcome supervision provides one judgement for a whole chain; process supervision can localize a judgement to the step where the problem occurred. For an enterprise agent, the relevant unit should be observable: a tool call, a retrieved item used as evidence, a policy result, an approval, an effect receipt or a concise structured rationale attached to the proposed disposition. Trajectory assurance adapts the localization principle without claiming access to hidden chain-of-thought. Its controls and claims still need independent validation in the target domain.

What trajectory-level assurance means for an agent

An agent trajectory is the ordered sequence of observable events during one task. It includes evidence and source identifiers, claims, tool proposals and results, policy decisions, approvals, effect receipts, the final action and, where useful, a concise structured rationale. Trajectory-level assurance evaluates that record against properties narrower than “was the reasoning good.” It neither requires nor treats hidden chain-of-thought as control evidence.

Each factual claim should be supported by retrieved or tool-returned evidence. Tool arguments should be validated against available context. The trajectory should obey every policy constraint, including escalation thresholds. The final action should also be consistent with the recorded evidence and preceding actions, rather than being correct by chance.

Each of these four checks can, in practice, be scored by a combination of automated methods and targeted human review, rather than requiring exhaustive human review of every trajectory, which is the property that makes this approach viable at production volume. Groundedness checks can be automated reasonably well using a dedicated verification model that compares a claim against the specific evidence cited for it, a narrower and more tractable task than open-ended reasoning quality assessment.

Tool-argument validation can often be checked deterministically, since tool schemas are typed and the available context is known. Policy checkpoint adherence is close to fully mechanical, a rule evaluation against a logged trace. The final action-to-evidence consistency check is the property still best served by human review on a sampled basis, because judging whether an action is supported by the evidence, checked results, prior observable events and concise rationale remains less reliable for current automated verifiers than the other three checks.

A lightweight internal process scorer

Very few enterprise deployments need, or can justify, training a full process reward model from scratch in the style of mathematical-reasoning research. It requires substantial step-level labelled data and a dedicated training pipeline. A lighter adaptation is usually more practical: a purpose-built verification model, deterministic assertion set, or tightly specified judge that scores each step against the four properties above. Calibration data should come from the actual domain, not from a general mathematical-reasoning corpus.

The labelling exercise is often the most valuable artefact, independent of the scorer it enables, because it forces business and compliance stakeholders to write down what a sound step looks like in their domain. That specification often exists only as tacit knowledge held by experienced case handlers. In the worked AML composite, a six-week exercise with two senior investigators covers 1,100 trajectory steps drawn from 180 historical cases and surfaces eleven categories of process failure. Several become explicit tool constraints and review rules before the scorer is complete. The numbers are planning assumptions; the transferable point is that adjudicated labelling exposes missing operating rules as well as training data.

Scoring dimensions

Groundedness scoring checks each factual claim against the evidence cited for it. The narrow question is whether the retrieved evidence supports the claim, not whether the answer sounds good. For the worked design, assume a held-out set produces a false-negative rate of roughly 4 to 6 percent and a false-positive rate of roughly 8 to 11 percent. That is useful for prioritising trajectories for human review, but not strong enough to make the scorer the sole gate on a consequential action. A real programme should report its own confidence intervals, class balance and error costs instead of adopting these illustrative ranges.

Tool-argument validity scoring is close to fully deterministic where tool schemas are properly typed: does the value passed to a date parameter fall within a plausible range given the context, does an account identifier argument actually appear somewhere in the retrieved context rather than being fabricated, does a monetary amount argument match a figure that was actually retrieved rather than approximated. This dimension has the highest signal-to-noise ratio of the four and the lowest cost to automate, because it requires no natural-language judgement at all, only a cross-reference check against logged context.

Policy adherence scoring checks the trajectory against explicit, enumerated rules, did the trajectory reach a notional threshold without routing to human review, did it access a data source outside its declared scope for the task, did it skip a mandatory verification step defined in the agent's specification.

This is scored by directly querying the structured trace log (the same BigQuery-backed trace store described in the companion piece on Agent Runtime observability) against a rule set maintained as code, and it requires no model-based judgement whatsoever, only a well-maintained rule catalogue that is kept in sync with the actual compliance requirements it is meant to encode, which is itself an ongoing governance task rather than a one-off build.

Action-evidence consistency, the fourth dimension, uses an automated judge to flag trajectories where the final action appears inconsistent with cited evidence, checked results, prior observable events or the concise structured rationale. Human review then examines the flagged and risk-sampled subset. The worked operating model assumes a 5 to 12 percent review range by risk tier; this is an illustrative capacity input, not an observed industry rate. The sample is weighted toward higher-notional or higher-risk cases because the cost of a missed inconsistency is not uniform across the population.

Calibrating against genuine human disagreement

A subtlety that surfaces during adversarial testing is that human labellers disagree more often than the first programme design usually assumes. In the worked composite, the illustrative calibration assumes that two senior investigators independently label the same 200-event subset and agree on action-evidence consistency 88 percent of the time. It also models scorer agreement of 84 percent with one investigator to show why a nominal 90 percent can be misleading if it merely reproduces one person's preferences. These are scenario values, not institutional results. Local inter-rater agreement, disagreement categories and adjudication rules matter more than an isolated headline percentage.

The composite operating response is to have every step in the initial labelling set reviewed independently by two investigators, with disagreements adjudicated by a third senior reviewer. This roughly doubles the assumed initial labelling cost, but the result is more defensible than single-labeller ground truth. It also identifies the trajectory categories that remain ambiguous even to experts. Those disagreements should improve investigator guidance as well as the scorer.

A calibration protocol for the evaluator

The evaluator is part of the control system, so a high agreement figure is not enough. Calibration begins with a defect taxonomy written before scorer selection. Each defect should be observable and tied to consequence: unsupported material claim, wrong entity, invalid tool parameter, missing prerequisite, authority breach, stale evidence, unhandled contradiction, unsafe recovery or mismatch between recorded evidence and action. A category such as “poor reasoning” is too vague to test or remediate.

Build the calibration set from three sources. First, sample ordinary historical cases in proportion to the population the agent will serve. Second, oversample rare but severe cases, because a purely representative sample may contain too few to measure. Third, inject controlled defects into otherwise valid traces. The injected set establishes whether the evaluator can detect the exact failures it is meant to control; ordinary cases establish its false-alarm burden. Keep the clean original and the altered trace as a pair so the test isolates the injected change.

Split the set by case or customer, never by individual step. If steps from one trajectory appear in both calibration and test data, shared facts and phrasing can make agreement look better than it is. Preserve a final locked set that the team tuning prompts, rules or thresholds does not see. Its role is similar to independent validation: it tests the released evaluator rather than helping to build it.

Measure performance by defect and severity. A scorer can have excellent aggregate accuracy while missing most authority breaches if those are rare. For each category, report sensitivity, specificity, positive predictive value, false-negative count and the review volume created at the chosen threshold. For graded scores, add a reliability plot: among items assigned an 80 percent probability of defect, roughly 80 percent should be confirmed under the adjudicated reference. Confidence that has not been checked against outcomes is only a ranking signal.

Thresholds follow consequence, not a generic F-score. Missing an unauthorized payment action is costlier than sending a harmless but ambiguous trace for review. A team may therefore choose a high-sensitivity threshold for authority and effect checks while accepting a lower-sensitivity, higher-precision threshold for stylistic or explanatory quality. One score should not collapse these different loss functions.

There are also cases where a language-model judge should not be used. Entity equality, schema validity, policy version, approval order, amount limits, idempotency and observed effect are better checked deterministically. A judge model adds variance to questions the system can answer exactly. Use a model-based evaluator only where the property is genuinely semantic, such as whether evidence supports a material claim or whether the action is supported by the recorded evidence and concise rationale. Even there, require citations to the trace segments used for the score so a reviewer can inspect the basis.

Prompt diversity is not evaluator independence. Running the same model with two prompts may produce two highly correlated opinions. Independence is stronger when checks use different mechanisms: a typed assertion for parameters, a policy engine for authority, a smaller verifier for claim support and a human for genuinely ambiguous consistency. Where two model judges are used, measure their residual error correlation on the adjudicated set. Agreement between two systems that share the same blind spot is not corroboration.

The release threshold should include operational capacity. Suppose a scorer catches more defects at a threshold that routes 35 percent of traces to people, but the team can review only 10 percent. The nominally safer threshold creates an unmanaged queue and delayed decisions. Either fund the review load, narrow the automated scope, improve the scorer or apply risk-weighted sampling. Do not publish the laboratory recall figure while ignoring whether flagged cases can be handled.

After release, draw a fixed human sample from both sides of the threshold. Reviewing only flagged cases measures precision but cannot show what the scorer missed. Include unflagged traces, severe cases, newly introduced products and cases after any model, prompt, tool, source or policy change. Plot disagreement by category over time. A change in the mix of disagreements often appears before aggregate agreement falls enough to trigger an alert.

Finally, version the evaluator with the system it certifies. Record the scorer model, prompt, rule set, thresholds, defect taxonomy, calibration data window and adjudication guidance. A new agent version evaluated by an old scorer may no longer be covered by the original evidence. The question for release is not “Do we have an evaluator?” It is “Which system version, population and failure classes has this evaluator demonstrated that it can assess, at a review load the operating team can sustain?”

Cost of step-level labelling

Step-level labelling is materially more expensive per unit of data than outcome labelling, because a nine-step trajectory can require nine judgements rather than one, and each judgement needs the evidence available at that point. The composite cost model assumes roughly six minutes for one step judgement, against about ninety seconds for one outcome judgement. A step judgement therefore costs four times as much as an individual outcome judgement. Labelling all nine steps takes fifty-four minutes, or thirty-six times the ninety-second outcome label for the whole trajectory. These are illustrative planning ratios, not observed benchmarks. A delivery plan should time a representative local sample before fixing its budget.

This cost is why a substantial initial labelling exercise is normally followed by a smaller continuing sample. In the worked plan, the initial 1,000-plus labelled steps produce a usable scorer. A monthly refresh then adds 60 to 100 newly adjudicated steps to test whether agreement with people has degraded after a task-distribution or model-version change.

Those sample sizes are planning assumptions, not universal guidance. The local rate should follow defect prevalence, desired detection power and consequence. The continuing sample belongs in the operating budget in the same way as conventional model monitoring; otherwise the programme is likely to lose support after the unusually well-resourced pilot period.

Fit with existing model validation

A trajectory-assurance programme does not automatically need a separate governance structure. It can fit within an existing model-risk or independent-validation function, while the institution decides whether the scorer itself belongs in a model inventory. The practical advantage is reuse: the function already has methods for fitness for purpose, data lineage, performance monitoring and escalation. The important caveat is the Federal Reserve's 2026 scope statement discussed later: generative and agentic AI are outside that guidance's formal scope. Reusing a familiar discipline must not be misrepresented as a regulatory classification.

Validation frequency is the material adaptation. A stable conventional model may sit on an annual or biennial cycle under an institution's policy. An agent and its scorer can change when the underlying model, prompt, tool, source or policy changes. Their revalidation triggers should therefore be component changes and observed drift, not only a fixed calendar date. The model-risk function should agree those triggers explicitly rather than assume an existing annual review covers them.

Failure modes

One predictable failure is stopping the human calibration sample after the scorer appears stable. New fraud typologies, products and upstream model versions can change the traces being scored. The scorer may then retain its dashboard while losing agreement with expert review. A continuing, adjudicated sample is the direct mitigation; without it, an audit may find that the latest calibration evidence belongs to the original build rather than the current system.

A second failure mode is allowing cheap-to-automate dimensions to crowd out the one that genuinely needs human judgement. Because policy adherence and tool-argument validity are close to free to compute at scale, it is tempting to build extensive dashboards around them while under-resourcing the harder action-evidence consistency review. This produces an assurance programme that looks comprehensive while concentrating its coverage on the dimensions least likely to catch a subtle, high-consequence failure: a technically compliant trajectory whose conclusion is not supported by the recorded evidence.

A third failure mode, specific to regulated settings, is treating the process reward model's or scorer's output as itself exempt from model risk governance, on the theory that a control mechanism does not need the same scrutiny as the thing it is controlling. A scorer that has itself never been validated against a representative, adversarially constructed test set (deliberately including trajectories designed to look sound while containing a specific injected process flaw) can develop its own systematic blind spots, and treating the scorer as a lightweight internal tool rather than a model subject to its own validation cycle is a governance gap that a thorough external review will eventually find.

Worked composite: trajectory scoring for AML alert adjudication

The worked composite assumes a retail bank's transaction-monitoring system produces roughly 14,000 alerts a month requiring investigator review. A triage agent gathers initial evidence and proposes an escalate-or-clear recommendation, while a human retains the final decision during the pilot. The trajectory-assurance programme supplies evidence for a risk committee to decide whether well-grounded recommendations may influence investigator prioritisation. It does not remove adjudication authority.

The illustrative plan labels 1,100 steps across 180 historical cases and develops a scorer over eleven weeks: six weeks of labelling, three of development and calibration, and two of adversarial testing. At the modelled operating volume, all alerts receive automated checks and roughly 9 percent receive additional human trajectory review, on top of final adjudication. Those figures are deliberately specific enough to test capacity and cost; they are not benchmarks to copy.

At the modelled nine-month review point, the illustrative dashboard shows 93 percent recommendation agreement, 96.4 percent material-claim groundedness and 99.1 percent policy adherence. For the modelled fully reviewed subset, scorer and investigator flags agree 84 percent of the time. None is a reported institutional result or a proposed universal threshold. The point is that a committee sees outcome, process, disagreement and residual human authority together, with denominators and limitations visible.

Direction of travel

Trajectory assurance remains an institution-specific governance practice rather than a standardised industry methodology. The likely direction is from validation-time analysis towards continuously running controls. Tool-schema, authority and policy assertions are good candidates for inline verification because they can be evaluated mechanically. Groundedness at the margins and action-evidence consistency remain harder; they should not be represented as solved simply because an LLM judge can assign a score.

The harder open question is how to make the two dimensions that still require human judgement, groundedness at the margins and action-evidence consistency, cheap and reliable enough to run inline without replacing one unverified model with another and calling the problem solved. My expectation is that the answer will look less like one breakthrough model and more like narrower, mechanically checkable sub-claims that are independently validated and then composed, rather than one large judgement asked to do all the work.

The assurance case, not the benchmark score

A production agent should be released against an assurance case: a structured argument that its intended use is clear, its important risks are identified, its controls address those risks, and the evidence is strong enough for the accountable owner to accept the remaining exposure. A benchmark score is one item in that evidence. It is not the argument.

The assurance-case framing is useful because an agent is not a single model. It is a changing system containing a model or several models, prompts, context assembly, retrieval, memory, tools, policies, workflow state, human approvals, routing and operational dependencies. A validation focused only on the foundation model’s general capability cannot determine whether the configured system chooses the correct account, calls a permitted tool, uses the latest policy, handles a timeout safely or writes enough evidence to reconstruct the decision.

The case has five claims.

1. Purpose claim: the intended users, decisions, populations, exclusions and action rights are explicit.
2. Performance claim: the system performs adequately on representative and severe scenarios, including uncertainty and refusal.
3. Control claim: identity, context, policy, tool and human controls remain effective under expected failure conditions.
4. Operations claim: monitoring, incident response, change control and recovery keep the system within its approved envelope.
5. Accountability claim: a named owner understands the evidence, limitations and residual risk and has authority to accept or constrain use.

Each claim should link to evidence, counter-evidence and an owner. Counter-evidence matters. If the agent performs poorly on a narrow language group or a specific document type, the assurance case should not hide the result inside an average. It should either constrain the approved population, add a control, or obtain explicit acceptance of the limitation. This is closer to engineering safety practice than to a marketing evaluation, and it gives independent validation something concrete to challenge.

trace_id, event_id, parent_event_id
timestamp, actor_principal, actor_role
system_release_id, model_release_id
context_snapshot_id, evidence_ids[]
event_type, tool_or_policy_id
input_hash, structured_input
structured_output, confidence_or_score
decision, reason_codes[], obligations[]
approval_id, effect_receipt_id
latency, token_and_cost_metrics

Scope and limitations

This is a practitioner architecture, not a new regulatory standard or a claim of empirical superiority. The cited trajectory benchmarks are preprints unless independently verified as peer reviewed. Their results show why path-level evaluation matters; they do not establish performance for a bank’s system. Model-based evaluation remains an active research area, and recent work indicates that long-horizon trajectory judgment is difficult even for strong evaluators.

Legal and supervisory applicability varies by jurisdiction, institution, use and date. The regulatory sources are used to identify convergent control themes, not to provide legal advice. The 2026 Federal Reserve guidance’s explicit exclusion of generative and agentic AI is important and should not be blurred. Banks still need an appropriate governance treatment, but the precise classification and accountable forum may differ.

The central position is narrower: when an AI system can select tools and affect enterprise state, outcome accuracy is not enough evidence. The institution needs a record and evaluation of how the outcome was produced, which controls operated, what effect occurred and whether the system stayed within its approved action rights. That is trajectory assurance.

Change impact and revalidation

Agent systems change more frequently than traditional validation calendars assume. Models are revised, aliases move, prompts are tuned, tools gain fields, sources change schema, policies take effect, retrieval indexes refresh and evaluators improve. Treating every change as a full revalidation stops delivery; treating only model replacement as material misses the changes most likely to alter real behaviour. The answer is a change-impact graph tied to the trajectory and control model.

For every component, record which assertions and scenarios depend on it. A tool-schema change should rerun static contract, parameter, ordering, authority, effect and recovery tests for every journey using that tool. A new source should rerun authorization, data-quality, provenance, injection and evidence-coverage tests. A prompt change should rerun affected task, claim, escalation and action-selection scenarios. A model change should rerun the full behavioural suite for the populations it serves, plus calibration and severe-tail analysis. A policy change should rerun every scenario whose expected permit, review, deny or obligation outcome depends on the changed rule.

The graph allows selective depth without selective blindness. A low-impact component can be changed quickly because evidence shows its blast radius. A seemingly small change receives deeper review when it reaches an action boundary. This is more defensible than categorizing changes by team ownership: a prompt owned by an application team and a threshold owned by risk can both change the same customer outcome.

Revalidation also needs an expiry concept. Approval should expire when the evidence no longer represents production, not simply after twelve months. Triggers include material drift, policy change, unresolved incident, evaluator degradation, source-quality breach, expansion of autonomy, repeated human overrides, severe vendor change and movement beyond the approved population. A calendar review remains useful as a forcing function, but event triggers protect the periods between calendars.

The production assurance scorecard

Executives, validators and operators need different views of the same evidence. The executive view should show exposure and decision quality, not token counts. The validation view needs failure strata, control results and limitations. The operations view needs service and incident signals. All should reconcile to the same trace population.

An executive scorecard can remain compact:

The validation view should preserve denominators. “Ninety-nine percent policy compliance” is meaningless if policy results were captured for only half the traces. “Two incidents” is misleading without the population and exposure. Report completeness before quality: trace capture rate, evaluator coverage, human sample coverage and source availability. Missing evidence is not a neutral gap; for a consequential system it is a failed control result.

Use control limits rather than vanity targets. An unexplained drop in escalation can be as concerning as a spike, because the system may have become overconfident. A sudden fall in human overrides can reflect improvement, automation bias or a user-interface change that made correction harder. Pair behavioural metrics with process and customer outcomes, and investigate causal hypotheses rather than celebrating movement in the preferred direction automatically.

The scorecard should expose uncertainty. Confidence intervals, sample sizes, “not measured” states and known blind spots belong on the page. Quietly replacing missing values with green status is how assurance becomes theatre. A credible system can state: the common population is performing within tolerance; one severe scenario remains open; the new language segment is advisory-only, and expansion will wait for evidence. That precision increases trust because it shows the boundary of the claim.

The independent challenge agenda

Independent review adds value when it challenges the architecture’s assumptions rather than reruns the delivery team’s preferred benchmark. The challenge agenda should ask:

• Is the intended use narrower in evidence than in business language?
• Are action rights and cumulative exposure fully represented?
• Does the scenario portfolio cover foreseeable severe and excluded conditions?
• Are important labels grounded in real expert agreement?
• Can the evaluator detect the failure modes it is used to certify?
• Are source authority, authorization and freshness tested separately from answer quality?
• Do tool tests verify semantic parameters, order and observed effect?
• Can a model, prompt or retrieved document bypass policy or identity controls?
• Does human review have enough information and time to exercise judgment?
• Can operators stop, roll back and investigate without the original developers?
• Are third-party changes observable, testable and contractually supported?
• Which residual risks are accepted, by whom, and for how long?

The validation function should have direct access to the release artifact, evaluation manifest and selected raw evidence under appropriate controls. A slide deck summarizing results is not effective challenge. Nor should independence mean separation from technical detail. Agent validation needs people who can read traces, reason about workflow and identity, understand evaluation statistics, and recognize domain-policy ambiguity. The institution may need a multidisciplinary validation cell rather than forcing the work into one existing specialization.

This is where forward-deployed architecture and risk advisory meet. The technical team must make control evidence a product of the runtime. The risk team must translate supervisory principles into testable questions. The domain team must adjudicate meaning and consequence. None can produce a decision-grade system alone.

Questions for an investment or release committee

Before approving a consequential agent, a committee should be able to answer several questions from evidence rather than presentation. What exact population, decision and action classes are in scope? Which actions remain recommendations, which may execute, and which are prohibited? What is the maximum single and cumulative exposure? Which severe scenarios have been tested, and how many passed without relying on an unobserved human rescue? Can the system demonstrate the identity, authority, state, policy result, tool parameters and observed effect for a sampled decision without reconstructing them manually?

The committee should also ask what could make today’s evidence invalid tomorrow. A model alias change is obvious, but a tool schema, source policy, retrieval index, prompt, threshold, customer population or human-review interface can alter behaviour just as materially. The release record should therefore state the components and versions covered, the change triggers requiring revalidation, the metrics that can suspend autonomy, the person who can activate the stop mechanism and the route by which affected customers or cases are identified.

Finally, the benefit claim should use the same population as the risk claim. If productivity is measured across every assisted case while error is reported only on high-confidence automated cases, the comparison is misleading. Report volume, coverage, selective risk, severe-case performance, review effort, customer outcome and operational cost together. A credible release may still contain limitations: one segment may remain advisory, a rare action may require dual approval, or a new language may wait for more evidence. Those boundaries are not signs of an immature system. They are proof that autonomy has been allocated deliberately.

Notes for practitioners

Do not accept outcome accuracy alone as sufficient evidence for a regulated agent. Build a trajectory-level review before production use when the journey has genuine compliance consequence. Outcome metrics cannot expose every gap between a correct answer and an unsound process.

Decompose trajectory scoring into narrow dimensions: groundedness, tool-argument validity, policy adherence and action-evidence consistency. Match automation to each dimension's verifiability. Uniform human review and uniform automated scoring are both poor defaults.

Budget step-level labelling honestly. The worked programme assumes six to eight times the cost of outcome-only labelling per case. Replace that assumption with a measured pilot and retain a smaller monthly refresh sample.

Subject the trajectory scorer to the same model-risk governance as the agent. Test it against deliberately flawed traces before launch. A scorer that has not met its target failure modes is not sufficient assurance.

Fit the programme into the existing validation function and tie review triggers to material component changes. Expect expert reviewers to disagree on ambiguous steps. Surface and adjudicate that disagreement instead of treating one reviewer as unquestioned ground truth.