A case is an argument, not an evidence folder

The assurance-case tradition begins with a deceptively simple distinction. Evidence is an observation, test, analysis, review or operating record. A claim is a statement about the system. An argument explains why the evidence and subordinate claims support the claim under stated assumptions. The active ISO/IEC/IEEE 15026-2:2022 standard specifies minimum requirements for assurance-case structure and meaning, while explicitly declining to guarantee the quality of the contents. That boundary matters. A perfectly formatted case can still contain a weak claim, a non-discriminating test or an invalid inference.

The common graphical languages are aids to reasoning. The Goal Structuring Notation makes goals, strategies, contexts, assumptions and solutions visible. The Object Management Group's Structured Assurance Case Metamodel provides a machine-readable way to represent and exchange argument and evidence structures. Neither notation proves that a claim is true. Their value is that a reviewer can see what must be challenged.

For an agentic system, the top claim must be narrower than “the agent is safe” or “the model is accurate”. It should name the configured system, intended users, population, environment, action class, consequence and period of validity. A useful form is:

Each phrase closes an escape route. “Configured release” includes the model, instructions, retrieval, memory, tools, policy, orchestration and human gates. “Action A” prevents a drafting score from silently justifying execution. “Population P” blocks a test on familiar domestic payments from authorising a new cross-border corridor. “Environment O” exposes dependencies such as a sanctions service, identity provider or source-of-record. “Remain supported” makes approval perishable. The final clause prevents positive evidence from burying known reasons for doubt.

The minimum living structure

A practical case needs seven typed objects. A claim is a proposition that can be challenged. Evidence records what was observed, how, on which version and population. An inference states why that evidence supports the claim. An assumption names a condition accepted without direct proof in this case. A defeater records a condition that would weaken or invalidate a node. An owner has responsibility and authority to accept, restrict or reopen the claim. An expiry rule states the events or time limits after which the support is no longer current.

The missing object in many assurance programmes is the permission itself. The case ends in a coloured status, while production authority lives in a separate access-control system. That separation allows a withdrawn claim to coexist with active credentials. The stronger design attaches one or more action classes to the top claim and exposes its evaluated state to the deterministic authority kernel.

One score cannot carry the argument

Suppose a programme creates an “assurance score” by weighting task accuracy, groundedness, security tests and operational resilience. The compression is convenient, but it destroys the logic needed for a consequential release. A severe authority defect cannot be compensated by better prose. Missing effect receipts cannot be offset by a higher benchmark result. An untested population is an absence of evidence, not a low number that can be averaged away.

The case should therefore use hard claim states rather than one master confidence score. Let the release contain action classes r. For each class, define four predicates at time t: scope match S, current evidential support E, no sustained material defeater D, and effective operating controls O. Then:

Here, A is the permission state for action class r. The conjunction is deliberate. A release may preserve one permission while losing another. A retrieval change might reopen the claim for policy interpretation yet leave deterministic calculations intact. A failed effect observer might withdraw execution while preserving analysis and proposal. The system degrades by claim, not by mood.

Thought experiment: identical intelligence, different authority

Place the same model and prompt behind two interfaces. The first drafts a payment-repair recommendation that a trained operator must accept. The second directly resubmits the payment. Hold model behaviour, test results and user population constant. Change only the effect authority.

The two systems now require different top claims. The advisory system must show that the recommendation is useful, evidence-linked and presented so that review remains meaningful. The executing system additionally needs identity binding, immutable critical fields, policy enforcement, idempotency, effect verification, recovery and tested authority withdrawal. Authority changes the assurance burden even when model capability is identical. This is why a model card, benchmark or generic “agent safety” result cannot travel unchanged across deployments.

The graph must stay alive as the system changes

A conventional release dossier often treats evidence as permanent until the next scheduled review. Agentic systems make that assumption fragile. Behaviour can change when the foundation model alias moves, the system instruction changes, a retrieval index is rebuilt, a policy version takes effect, a tool gains a field, a memory record is promoted, a routing threshold changes, a human queue becomes overloaded or a dependency returns a new state.

The case must therefore model two things separately: the truth of a claim and the current fitness of its support. A test can remain historically true while becoming irrelevant to the deployed release. “The agent passed 1,000 payment scenarios” may be accurate, yet it no longer supports the current control claim after the sanctions schema changes. The correct state is neither pass nor fail. It is unsupported pending targeted revalidation.

Thought experiment: the frozen model and the changed world

Freeze the model weights, prompt and decoding settings. Keep the evaluation set untouched. Now change one external fact: a policy raises the threshold at which a second approver is required. The system continues to behave exactly as before. Its behavioural reproducibility is perfect, but the approval claim is false for the new policy.

Reverse the experiment. Keep policy constant, but change the retrieval filter so one region's rules are no longer returned. The model might still produce correct answers on familiar cases, and the average could remain stable. The relevant defeater is now population-specific evidence omission. In both variants, the error belongs to the configured system and its context, not necessarily to the model. An assurance case that versions only the model will certify the wrong object.

Defeaters belong in the case before the incident

Positive evidence answers “why believe the claim?” A defeater asks “what would make this support fail?” The Assurance 2.0 work treats logical assessment, probabilistic assessment, dialectical challenge and residual risk as distinct perspectives. Its treatment of defeaters is especially useful for AI systems because many claims are defeasible rather than deductive. A test campaign can support a claim about a population while remaining vulnerable to distribution shift, correlated labels or a missing hazard.

A defeater should be typed. A rebutting defeater contests the claim itself: the system used the wrong customer. An undercutting defeater attacks the support relation: the evaluator and agent share the same model family and may share the same blind spot. A scope defeater says the case was stretched beyond its tested population. An operational defeater says a necessary control or observation is unavailable. An authority defeater says the principal or purpose no longer permits the action.

Recording defeaters changes the culture of review. A red node is no longer evidence that the team has failed to present a clean story. It is a tracked reason for doubt with an owner, status and resolution path. A proposed defeater can be rejected with evidence, controlled by narrowing scope, accepted as residual risk by an authorised owner, or sustained so that the claim and permission are withdrawn. Silence is not resolution.

Dynamic does not mean automatically trustworthy

Research on dynamic safety cases for frontier AI proposes systematic, partly automated revision as system state and operational indicators change. The older dynamic-safety-case tradition makes the same through-life point for autonomous systems. Automation is useful for version propagation, evidence freshness, monitor thresholds and dependency impact. It cannot decide every contested claim.

A monitor can tell the case that the override rate doubled. It cannot, by itself, establish whether the cause is degraded model behaviour, a new population, a user-interface change or improved reviewer vigilance. The case should reopen the relevant oversight claim and create an investigation obligation. Automation may change the status of support; it should not silently invent the missing argument.

The same caution applies to AI-generated defeaters. A language model can widen the search for counterexamples, but its suggestions are leads rather than findings. Human and mechanised challenge should include independent mechanisms: deterministic assertions, domain calculations, separate test data, security analysis, operator evidence and, where consequence warrants it, independent review. Two prompts to the same judge model do not create two independent witnesses.

flowchart TD
  A["The green dashboard that<br/>cannot answer the release<br/>question"] --> B["A case is an argument, not an<br/>evidence folder"]
  B --> C["The graph must stay alive as<br/>the system changes"]
  C --> D["Two worked cases reveal the<br/>right amount of argument"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#B96D00,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Two worked cases reveal the right amount of argument

Minimal case: a travel-expense assistant

The minimal top claim is not “the assistant processes expenses accurately”. It is: “Release E4 may submit a domestic meal claim below £25 for an authenticated employee when the receipt amount, date, currency and policy category are verified, no exception is detected, and the submission effect is observed exactly once.” This statement exposes what must be supported.

The identity claim needs evidence that the employee and expense account are bound. The extraction claim needs field-level tests on the supported receipt types, including a negative case with low contrast. The policy claim needs versioned deterministic rules. The calculation claim should use a calculator rather than generated arithmetic. The authority claim needs a hard limit and a forbidden path for exceptions. The effect claim needs idempotency and a readback from the expense system.

Now inject one defect: the receipt date is ambiguous. The extraction claim becomes unsupported. The correct response is not a lower global confidence score. The submission permission becomes false, while the assistant may still draft a claim and ask the employee to confirm the date. A single unresolved field has produced a precise reduction in authority.

Negative control: the case that should stay small

Consider a deterministic service that receives a typed amount and policy category, applies a published table and returns the permitted reimbursement. It has no retrieval, language interpretation, memory, planning or effect authority. A large agent-style assurance graph would add ceremony without improving the decision. A versioned contract, unit and property tests, change control, access control and ordinary operational monitoring may provide the required evidence.

This negative control tests the preferred mechanism. The need for an assurance case does not arise merely because software contains AI. It grows with consequence, uncertainty, environmental dependence, compositional complexity and delegated authority. Even then, the case should reuse ordinary engineering evidence rather than rebuild it as prose. Proportional assurance is thinner where the claim is narrow and the mechanism is already explicit.

Production-shaped case: commercial payment repair

The case now follows ten distinct control objects rather than calling all of them “reasoning”. Intent binds the task to payment repair. Identity binds the customer, operator and agent principal. World state records the authoritative payment status. Context supplies permitted evidence and current policy. Reasoning proposes the rejection cause and repair. Authority decides whether the proposal may cross the effect boundary. Action invokes the typed resubmission tool. Evidence records support for material claims. Outcome confirms what changed. Release and recovery determine whether the system remains inside its approved envelope.

The top claim can be decomposed into four branches. The purpose and scope branch constrains payment type, corridor, amount, customer segment and prohibited edits. The behavioural branch covers rejection classification, evidence support, calibrated abstention and severe cases. The control branch covers principal propagation, policy decisions, immutable fields, tool contracts and ordering. The operating branch covers trace completeness, effect readback, duplicate prevention, rollback, incident handling and reviewer capacity.

The table makes a release committee's decision reversible. A new tool field does not create a vague “medium risk”. It sustains a known defeater against the immutability claim, so resubmission authority is withdrawn. Evidence assembly and proposal can continue because their claims remain supported. When targeted contract, policy and recovery tests pass on the new version, the owner may close the defeater and restore the action class.

The whole-system emphasis is consistent with AMLAS, which integrates machine-learning evidence into a wider autonomous-system safety case, and with the BIG argument, which stresses balanced, integrated and grounded treatment across technical and sociotechnical context. The payment case extends that principle into enterprise authority: the relevant release object includes policy, identity, tools, human work and recovery, not only the learned component.

Turn the case into an authority-bearing state machine

An assurance case changes decisions only when its state has operational consequence. A PDF reviewed once a year is documentation. An authority-bearing case is versioned with the release, evaluated when dependencies change and consulted before the system receives or exercises an action right.

The state machine can remain small. Draft means the argument is incomplete and grants no action. Supported means every hard claim is current and no material defeater is sustained. Conditional means the owner has narrowed population, action or time while an issue is controlled. Withdrawn means at least one necessary claim is unsupported or defeated. Retired means the release can no longer act and the historical case is preserved for replay and audit.

A four-gate decision instrument

Release review should resist the attraction of one score. The following instrument uses four hard gates. A “no” or “unknown” on any gate blocks the requested action class, although a narrower action may still pass. The gates are intentionally few enough for senior review and precise enough to trace into engineering evidence.

The claim owner prepares and maintains the branch. Independent challenge tests the inferential link, searches for defeaters and examines evidence fitness. The accountable approver decides whether the residual exposure is acceptable for the bounded action. Operations owns runtime evidence and authority withdrawal. One person may hold more than one role in a small, low-consequence system, but the case should still record which judgement was made in which capacity.

NIST's AI Risk Management Framework Core organises risk work through Govern, Map, Measure and Manage and describes risk management as continuous across the lifecycle. Its Playbook includes documenting go and no-go decisions, exceptions, escalations and downstream actions. The assurance case can organise the evidence created by such practices, but it should not be advertised as automatic conformity with NIST, the EU AI Act or any sector rule. Legal applicability and adequacy remain context-specific judgements.

function evaluateCase(data) {
  const evidenceIsCurrent = id => {
    const e = data.evidence[id];
    return e.status === "current" && Date.parse(e.validUntil) >= data.now;
  };

  const evaluateClaim = id => {
    const c = data.claims[id];
    const openDefeater = c.defeaters.some(
      d => data.defeaters[d].status === "open" ||
           data.defeaters[d].status === "sustained"
    );
    const evidenceOK = c.evidence.every(evidenceIsCurrent);
    const childrenOK = c.subclaims.every(evaluateClaim);
    return !openDefeater && evidenceOK && childrenOK;
  };

  const executionSupported = evaluateClaim(data.topClaim);
  return executionSupported
    ? { state: "SUPPORTED", authority: data.authority.approved }
    : { state: "WITHDRAWN", authority: data.authority.fallback };
}

// Positive: SUPPORTED -> AUTO_RESUBMIT_UNCHANGED_PAYMENT
// Negative: open defeater or expired effect evidence -> PROPOSAL_ONLY

The demonstration is intentionally strict. It treats every listed subclaim as necessary and uses no compensating score. A production implementation would add claim criticality, controlled conditional states, approval signatures, change-impact links and separate fallbacks by capability. The invariant should survive: the runtime cannot exercise an action class whose assurance claim is withdrawn or unknown.

Where the mechanism fails, and how to know

The strongest objection is that assurance cases can rationalise a desired release. A delivery team writes a vague top claim, selects favourable evidence, decomposes around what it already tested and labels every difficult issue “residual risk”. The result looks more rigorous than a checklist while carrying the same confirmation bias.

This objection is valid. Argument structure creates inspectability, not truth. The response is to evaluate the case as an engineered object. Are claims precise enough to be false? Does each inference name the theory or mechanism connecting evidence to claim? Could the evidence have produced an unfavourable result? Are severe cases and subpopulations visible? Are assumptions monitored? Are material defeaters generated by people and mechanisms with enough independence? Does a sustained defeater actually remove authority?

A second failure is evidence monoculture. The agent, judge and synthetic-data generator may share a foundation model or training lineage. Their agreement looks like corroboration while preserving one blind spot. A credible case records evidence provenance and correlation. It combines different mechanisms where possible: exact schema assertions, policy engines, deterministic calculations, independent data, human adjudication, operational effects and adversarial tests.

A third failure is the completeness illusion. A beautiful tree invites the reader to assume that every relevant hazard appears somewhere. No method guarantees that. Open-world agents can meet novel tools, data and users. The case should state the hazard-analysis method, known exclusions and residual uncertainty. It should include an “unknown” state that blocks consequential action when required evidence is missing. Green-by-default is an assurance defect.

A fourth failure is frozen ownership. Claims span teams and vendors. A retrieval claim may depend on a data owner, a tool claim on a platform team, a policy claim on compliance and an effect claim on an external service. If no named person can reopen a claim or withdraw permission, the graph becomes a map without a government. Ownership must include decision rights, response times and a route for unresolved cross-team disagreement.

A fifth failure is confusing process evidence with outcome evidence. Perfect logs can prove that a weak process ran exactly as designed. Conversely, good outcomes can occur through an unauthorised or brittle path. The case needs both. It must link intended business and user outcomes to the process properties that make those outcomes supportable, contestable and recoverable.

A sixth failure is case-runtime separation. A governance repository may mark a claim withdrawn while cached credentials, queues or long-running tasks retain the old permission. The assurance state then becomes advisory metadata rather than a control. The release design must specify propagation latency, token revocation, in-flight handling, retries and proof that the affected capability actually stopped. Test it by sustaining a defeater during an active mission and observing whether new calls are rejected, queued effects are cancelled or reconciled, and unaffected capabilities remain available. Until that property is demonstrated, automatic authority withdrawal is an open design proposal, not an achieved control.

What would weaken this paper's thesis?

The thesis would weaken if a simpler release method consistently produced the same or better decisions. A discriminating study could compare three matched review regimes across several agent changes: a benchmark dashboard, a checklist-based gate and a living assurance case. Inject policy, tool, population and operating changes after initial approval. Measure time to identify affected permissions, false withdrawals, missed unsafe continuations, reviewer effort and restoration time.

If the dashboard or checklist catches the same permission-impacting changes with lower cost and equal traceability, the extra argument structure is unnecessary for that class of system. If the assurance case improves detection only because its team receives more information or stronger reviewers, the mechanism has not earned credit. The comparison must equalise evidence access and reviewer expertise.

The expected advantage is narrower. A claim graph should outperform when a change affects some permissions but not others, when evidence has different validity conditions, and when reviewers need to expose why a result justifies an action. This is an open hypothesis, not a reported enterprise result.

Assurance is a disciplined permission to proceed

The UK AI Security Institute describes safety cases as structured arguments for safety in a particular training or deployment context, complementing empirical evaluation rather than replacing it. That relationship generalises. The assurance case is not the control plane, evaluator, risk register, audit report or legal opinion. It is the argument that connects their evidence to a bounded decision.

For agentic systems, the decision should be expressed in action rights. Can the system retrieve? Draft? Recommend? Place a hold? Resubmit? Communicate externally? Move money? Change a record? Each right deserves a claim proportional to consequence and reversibility. Some claims can share evidence; none should inherit authority merely because a neighbouring claim passed.

The practical standard is therefore not “do we have an assurance case?” It is “which exact permission is justified by which current argument, and what will remove it?” That question turns assurance from document production into release engineering.

Release the claim, not the model

An agentic release is a permission-bearing configuration, not a model score. Its real subject includes instructions, context, tools, policy, identity, state, human work, effects and recovery. The release decision must therefore say what this configured system may do, for whom, under which conditions and on the strength of which current evidence.

A living assurance case supplies the missing mechanism. It turns a broad approval into explicit claims, exposes the inference from tests to permission, records assumptions and serious defeaters, and attaches owners and expiry rules. Its most important output is not a diagram. It is a machine-enforceable state that grants, narrows or withdraws an action class.

The architecture decision changes accordingly. Do not place assurance beside the runtime as a reporting function. Put the evaluated claim state on the path to authority. A tool call should cross a consequential boundary only when the relevant top claim is supported for the current release, population and environment. A stale evidence branch, unresolved defeater or missing effect observer should move the system to a safer capability before a committee convenes.

The research decision changes too. Compare assurance mechanisms by whether they identify permission-impacting change, preserve valid partial capability and recover efficiently after targeted revalidation. That experiment can fail. It should. The first defensible contribution is an approval that knows why it exists, when it expires and how to stop acting when its reasons disappear.