What makes an answer defensible

A correct answer is an outcome. A defensible answer is an outcome plus a reconstructable basis for accepting it. The basis does not need to expose private hidden reasoning or produce a theatrical transcript. It needs to establish a smaller set of operational facts: what question was authorised, what evidence was available, which transformations occurred, which claims were checked, which conditions remained unresolved and who or what was allowed to accept the result.

This is stricter than citation. A cited passage may be irrelevant, obsolete, unauthorised or quoted in a way that reverses its meaning. Work on citation-bearing language models and citation evaluation demonstrates the usefulness of evidence-linked answers while also showing incomplete support and source-quality limits.¹²¹³ It is also stricter than a verifier score. A verifier can share the proposer’s blind spot, reward surface or contaminated context. Defensibility depends on the whole acceptance path.

A useful formalisation begins with the material claims in an answer. Let M(A) be the set of claims that could change a decision, obligation or action. For each claim c, let S(c) mean that applicable evidence supports it, and let V(c) mean that a suitable check passed. Let B mean that the task and authority boundary held. Then the stack may accept only when:

The symbols are intentionally modest. They do not prove truth in an unrestricted philosophical sense. They state an engineering condition: every material claim must have support and a check appropriate to its type, and the system must remain inside its boundary. As consequence rises, the definition of “suitable” should become stricter. A calculator can verify an addition. It cannot determine whether a policy exception is fair. A source entailment model can test textual support. It cannot establish that the source was legally applicable on the decision date.

A proposal becomes defensible through acceptance criteria that are external to the prose that proposed it. That independence can be mechanical, evidential, organisational or human. The relevant question is whether the checker has a path to catch the proposer’s failure, not whether it has a different name.

The seam is the unit of assurance

Software architecture usually treats a seam as a place where one component can be replaced or tested independently. In a reasoning system, a seam has a second role: it marks a change in epistemic responsibility. Retrieval claims that a source is relevant. A parser claims that a field was extracted. A calculator claims that an operation produced a value. A model claims that several facts support an interpretation. A policy engine claims that an action or answer is permitted. When these claims are fused into one generation, their errors become difficult to distinguish.

Practitioner synthesisThe most useful seam is one that changes the available failure test. Separating two model calls merely to draw another box does not help. Separating retrieval from synthesis does help because retrieval can be tested for recall, entitlement, freshness and source identity, while synthesis can be tested for claim support and omission. Separating calculation from narrative helps because arithmetic can be recomputed exactly. Separating proposal from acceptance helps because the proposer does not control the conditions under which its own answer is released.

Decomposed prompting demonstrated that complex tasks can benefit when subtasks are delegated to specialised handlers and symbolic functions.⁵ Retrieval-augmented generation made external evidence a first-class input rather than relying only on parametric memory.⁶ ReAct and Toolformer showed how language models can interleave reasoning with external actions or learn when to invoke APIs.⁷⁸ These are capability mechanisms. A defensible stack adds explicit acceptance contracts around them.

The seam also prevents a stronger model from hiding a weaker system. A model may compensate for a poor retriever on familiar questions, then fail abruptly when a rare document or superseded rule matters. It may repair malformed tool output in prose, masking a broken interface until the repair is wrong. It may generate a citation that looks plausible while the source does not entail the claim. Exposed intermediate artefacts make those debts observable before an incident supplies the label.

What must cross a seam

A seam is useful only when the next stage can tell what the previous stage actually established. Passing a natural-language paragraph is usually too weak because observations, assumptions and recommendations become indistinguishable. Passing the entire hidden trace is usually too strong because it adds volume without guaranteeing causal faithfulness. The practical middle is a typed receipt that records the proposition asserted and the basis on which another component may rely on it.

Practitioner synthesisA minimal seam record should carry five things:

• Assertion. The smallest decision-relevant proposition, with a type such as observation, calculation, source-supported claim, interpretation or requested action.
• Inputs. Stable identifiers for source passages, data fields, tool arguments and versions, including valid time and entitlement where they affect applicability.
• Transformation. The code, rule, prompt or human procedure that produced the assertion, together with its version and material parameters.
• Quality state. Confidence only when calibrated, plus conflict, missingness, freshness and scope flags that downstream policy can test directly.
• Receipt. The result, rejection reason and linkage to the parent task, so replay can distinguish a failed check from a check that never ran.

These fields create a disciplined distinction among evidence, inference and authority. A parser may assert that a document contains “14 July”. It cannot silently promote that string into the transaction date. A policy resolver may assert that version P-17 covered that date. It cannot authorise reimbursement. A model may infer that the meal exceeded the cap. It cannot convert an unsupported inference into an observed fact. Each stage may add information, but it must not inherit decision rights merely because its output is fluent.

Local receipts compose only when dependencies remain visible. Eight individually valid stage outputs do not guarantee a valid answer if decomposition removed a cross-constraint or if two claims rely on mutually inconsistent source versions. The final manifest therefore needs both lineage and reconciliation: which accepted claims depend on which receipts, which conflicts were resolved, and which assumptions survived into the answer. This is why a reasoning manifest is closer to an executable proof obligation than to a transcript. It records what must be checked again when evidence, policy, code or authority changes.

flowchart LR
  A["Two identical answers, one<br/>defensible"] --> B["On this page"]
  B --> C["What makes an answer defensible"]
  C --> D["The seam is the unit of assurance"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#7B4CD8,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

The eight-stage reasoning stack

The stack below is deliberately expressed as transformations rather than products. A single runtime may implement several stages, and a stage may use rules, code, a model or a person. What matters is the contract between them. Every output should be serialisable, versioned and small enough to inspect. W3C PROV supplies a general vocabulary for entities, activities, derivations and responsible agents, while JSON Schema supplies a practical mechanism for structural validation.¹⁷¹⁸ The article’s executable artefact uses a lighter application-specific form.

1. Task contract. Translate the request into a bounded decision: what must be answered, for whom, at what time, using which evidence classes, with which allowed effects. Record explicit non-goals and abstention conditions. “Assess this case” is not a contract. “Identify missing evidence and draft a refer-back without approving credit” is.
2. Dependency-aware decomposition. Split the task only where the pieces have different evidence, tools or checks. Preserve interactions that matter. A debt-service calculation can be separated from a qualitative management assessment; two mutually dependent policy exceptions may need to stay together. The output is a dependency graph, not a bag of prompts.
3. Evidence acquisition. Retrieve sources against the task contract, then qualify them for identity, entitlement, version, valid time, transaction time, provenance and conflict. Retrieval relevance is necessary but insufficient. Self-RAG’s adaptive retrieval results are a useful reminder that indiscriminate retrieval can reduce quality when evidence is unnecessary or irrelevant.¹⁴
4. Typed tool execution. Move exact computation and external state access out of free-form generation. A tool call should declare preconditions, arguments, authority, expected result and failure modes. Its receipt should record the observed result, error class, idempotency key where relevant and whether the result is authoritative or advisory.
5. Proposal formation. Let the model synthesize a candidate answer from the evidence envelope and tool receipts. The proposal should expose material claims, source links, assumptions and unresolved points. It may use self-consistency, search or extra inference. Self-consistency can improve benchmark performance by aggregating multiple reasoning paths, but agreement among samples is still evidence about model distribution, not external truth.⁹
6. Claim-level verification. Extract the claims that could alter the decision and route each to a matching checker. Recompute numbers, test temporal applicability, check source entailment, validate required process steps and reserve genuinely interpretive propositions for calibrated review. Process-supervision research shows the potential value of step-level feedback on mathematically verifiable tasks; the transferable lesson is to place feedback where errors first become observable.¹⁰
7. Disposition. Choose among accept, revise, gather evidence, abstain and escalate. The gate uses check results, evidence sufficiency, consequence, time and review capacity. It should not ask the proposer whether it feels confident. The disposition is a policy decision over evidence, not another paragraph generated from the same context.
8. Answer and manifest. Render a concise answer for the user and preserve the machine-readable record: request, task contract, evidence versions, tool receipts, proposal claims, checks, disposition, unresolved issues and a replay identifier. The answer may omit internal detail that would confuse or expose sensitive reasoning. The manifest must retain enough structure to reconstruct why release was allowed.

The stack is complete only when the final disposition can be derived from recorded checks rather than inferred from the tone of the response. A human reviewer may still make the final judgement. In that case, the manifest should show the proposition presented, the evidence supplied, the action or answer authorised and the reviewer’s reason code. “Human in the loop” is not a stage until the person’s decision right and required evidence are explicit.

Verification must match the claim type

A universal “LLM judge” is attractive because it can score any text. It also collapses unlike questions. The statement “£128.40 + £162.00 + £35.00 = £325.40” calls for exact recomputation. “Policy P-17 applied on 14 July” calls for a temporal applicability check. “The source supports the claim” calls for citation entailment and completeness. “The evidence is sufficient to approve an exception” may require a named human authority. Passing all four through the same generative judge creates a common-mode failure.

Chain-of-Verification reported lower hallucination rates when a draft was followed by independently answered verification questions before revision.¹¹ The important structural feature is not the phrase “verify step by step”. It is the attempt to reduce anchoring by separating draft generation from fact-checking. The same principle applies beyond language models: the checker needs information or mechanics that can expose a failure the proposer cannot simply restate.

Thought experiment: the mirror verifier

Hold the task, model, prompt length and total token budget constant. System A asks a model to draft an answer, then shows the same model its complete draft and asks, “Is this correct?” System B withholds the draft, gives a second pass the original question plus authoritative evidence, and asks it to compute the required claims independently. Both systems spend equal compute. Only the correlation structure changes.

System A may catch surface inconsistency, but it is anchored to the original framing. System B can disagree on values or evidence because it does not inherit the proposal’s conclusion. If matched tests show no difference, independent recomputation has not earned its complexity for that task. If System B catches material errors that System A repeats, the seam has causal value. Verification independence is an empirical property of error overlap, not a label attached to another model call.

Abstention is a disposition, not a mood

“I am not confident” is an observation about a model signal. “The system must not answer” is a policy decision. The two can be related, but they are not interchangeable. Semantic-entropy research has shown that disagreement among meanings can help detect a class of confabulations and improve selective answering.¹⁵ A more recent negative result found model-dependent, confidently wrong regimes in which entropy alone did not support reliable operation at strict target error rates.¹⁶ A stack should therefore combine uncertainty with evidence sufficiency, verifier results and consequence.

The disposition gate needs a small, explicit vocabulary. Accept means every release condition passed. Revise means the evidence is sufficient but the proposal failed a repairable check. Gather evidence means a named missing item could resolve the case. Abstain means the system cannot produce an answer within its contract. Escalate means the residual judgement or consequence requires another authority. These states should be mutually distinguishable in metrics and user experience.

Abstention should name the blocking proposition and the next admissible move. “I cannot help” is often useless. “No policy version authorised for the travel date was found; retrieve the effective version or send the case to the policy owner” is operational. It preserves progress without manufacturing certainty.

Consequence changes thresholds. A low-consequence internal summary may tolerate one weakly supported background sentence if the limitation is visible. A payment release, eligibility decision or safety instruction may require every material claim to pass and every missing source to stop the path. The system should encode that asymmetry before generation. Asking the model to become cautious after it has already composed a confident answer is a weak control.

Worked example: one expense claim

Worked scenarioConsider the synthetic claim introduced at the start. EC-217 contains an economy train ticket for £128.40, one hotel night for £162.00 and a meal for £38.00. All receipts are present. Travel policy P-17 reimburses economy rail at actual cost, caps the hotel at £180 and caps the meal at £35. The question is whether the claim can be reimbursed in full.

A monolithic answer can solve this easily. That is precisely why the case is useful: the arithmetic is not the hard part. The experiment isolates the acceptance path.

Stage 1: contract and decomposition

The task contract permits an answer only. It does not authorise a payment or alter the expense system. It requires the claim, receipts and policy effective on the travel date. It decomposes into four checks: evidence validity, rail eligibility, cap application and total reconciliation. The decision rule says to abstain if no effective policy version is available.

Stage 2: evidence and tools

The evidence envelope records claim version 3 and policy version 4, including their validity intervals and entitlement status. A deterministic function calculates the requested total of £328.40, then applies the caps to obtain £325.40. It emits the inputs and output. The model does not perform hidden arithmetic in prose.

Stage 3: proposal and verification

The proposal contains four material claims: the requested total, allowed total, excluded amount and full-or-partial conclusion. Independent checks recompute the three values and confirm that the decision follows from the £3.00 excess. The support checker confirms that each claim names an existing evidence object. Only after those checks pass does the disposition become ACCEPT.

Now alter one causal feature. Policy version 3 expired before the travel date, and no current version is retrieved. The arithmetic can still run. The prose can still sound certain. The effective-date check fails, so the disposition becomes ABSTAIN. The answer states the missing evidence rather than guessing whether the old cap persisted. This is the difference between a model that can solve the example and a system that can defend its release decision.

stateDiagram-v2
  [*] --> Claim
  state "What must cross a seam" as Claim
  state "Thought experiment, the mirror<br/>verifier" as Test
  state "Stage 1, contract and<br/>decomposition" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Production-shaped example: a credit refer-back

Worked scenarioA commercial borrower requests a facility increase. The system is allowed to assemble evidence and draft a refer-back asking for missing or inconsistent information. It is not allowed to approve credit, waive policy or send a customer communication without review. The case contains financial statements, an application form, customer and facility identifiers, existing exposure, product policy and a covenant schedule.

The task contract matters immediately. “Assess creditworthiness” would invite the model to blend calculation, policy, evidence quality and judgement into one narrative. The bounded contract asks a narrower question: “Can the evidence package support underwriting review, and if not, which specific items must be corrected or supplied?” That decision can be decomposed into checkable burdens.

First, identity and world state are reconstructed. The system confirms that the statements, application and exposure record refer to the same legal entity and accounting period. A mismatch is not a reasoning problem; it is a case-state conflict. Second, deterministic tools calculate leverage, debt-service coverage and covenant headroom from named fields. Third, policy retrieval is filtered by product, jurisdiction, effective date and borrower segment. Fourth, the model proposes a compact set of refer-back items, each linked to a failed requirement or conflict.

Claim-level verification then changes the review experience. “Leverage is 3.8×” is recomputed. “Policy requires two years of audited statements” is checked against the applicable policy span. “The latest accounts are unaudited” is checked against document metadata. “The management forecast appears optimistic” is interpretive and should be presented as a bounded observation with supporting variances, not promoted to a deterministic failure. The final answer separates established deficiencies from judgemental concerns.

The disposition gate may accept a refer-back even though it could not accept a lending decision. That asymmetry is deliberate. Requesting a missing audited statement is reversible and evidence-bound. Approving a facility is consequential and requires a different decision right. The same proposal model can assist both workflows, but the authority boundary and verification burden differ.

The production gain is diagnostic ownership. If the wrong policy was applied, the context layer owns the defect. If leverage is wrong, the calculation or source mapping owns it. If a supported fact becomes an exaggerated narrative claim, synthesis owns it. If the system sends an unauthorised request, authority enforcement owns it. A single “reasoning accuracy” score cannot direct those repairs.

The manifest also supports replay. When policy changes, the institution can distinguish what was knowable at the original decision time from what is true now. When a source is corrected, affected manifests can be located through provenance links. When a model changes, the proposal stage can be rerun against the same evidence envelope, revealing behavioural change without silently mixing retrieval drift into the comparison.

Executable artefact: an end-to-end reasoning manifest

The following standard-library Python programme implements the expense example. It is intentionally small. It does not call a language model or retriever because the artefact’s purpose is to reveal the acceptance logic that prose tends to hide. A real system would replace the synthetic evidence builder and proposal function while retaining the manifest and checks.

Assumptions are explicit: evidence objects carry validity intervals and entitlement; money uses decimal arithmetic; the tool has answer-only authority; every material claim declares its supporting evidence; and a missing effective policy forces abstention. The short hash is a reproducibility fingerprint, not a cryptographic signature or proof of integrity.

from __future__ import annotations

from dataclasses import asdict, dataclass
from datetime import date, datetime, timezone
from decimal import Decimal
from enum import Enum
from hashlib import sha256
import json
from typing import Any


class Disposition(str, Enum):
    ACCEPT = "ACCEPT"
    REJECT = "REJECT"
    ABSTAIN = "ABSTAIN"


@dataclass(frozen=True)
class Evidence:
    evidence_id: str
    kind: str
    version: str
    valid_from: date
    valid_to: date | None
    entitled: bool
    payload: dict[str, Any]

    def is_valid_on(self, when: date) -> bool:
        return (
            self.entitled
            and self.valid_from <= when
            and (self.valid_to is None or when <= self.valid_to)
        )


@dataclass(frozen=True)
class Claim:
    claim_id: str
    text: str
    claim_type: str
    value: str
    support_ids: tuple[str, ...]


@dataclass(frozen=True)
class Check:
    check_id: str
    target_id: str
    passed: bool
    detail: str
    independent_of_proposal: bool = True


def money(value: str | Decimal) -> Decimal:
    return Decimal(value).quantize(Decimal("0.01"))


def canonical_hash(payload: dict[str, Any]) -> str:
    encoded = json.dumps(
        payload,
        sort_keys=True,
        default=str,
        separators=(",", ":"),
    )
    return sha256(encoded.encode("utf-8")).hexdigest()[:16]


def build_evidence(case: str) -> dict[str, Evidence]:
    policy_valid_to = (
        date(2026, 6, 30) if case == "stale_policy" else None
    )
    return {
        "claim": Evidence(
            evidence_id="claim:EC-217",
            kind="expense_claim",
            version="3",
            valid_from=date(2026, 7, 14),
            valid_to=None,
            entitled=True,
            payload={
                "travel_date": "2026-07-14",
                "train_gbp": "128.40",
                "hotel_gbp": "162.00",
                "meal_gbp": "38.00",
                "train_class": "economy",
                "receipts_present": True,
            },
        ),
        "policy": Evidence(
            evidence_id="policy:P-17",
            kind="travel_policy",
            version="4" if case != "stale_policy" else "3",
            valid_from=date(2026, 4, 1),
            valid_to=policy_valid_to,
            entitled=True,
            payload={
                "hotel_cap_gbp": "180.00",
                "meal_cap_gbp": "35.00",
                "train_requires_economy": True,
            },
        ),
    }


def form_proposal(
    evidence: dict[str, Evidence],
    case: str,
) -> list[Claim]:
    claim = evidence["claim"].payload
    policy = evidence["policy"].payload

    requested = sum(
        (
            money(claim["train_gbp"]),
            money(claim["hotel_gbp"]),
            money(claim["meal_gbp"]),
        ),
        start=Decimal("0.00"),
    )
    allowed = (
        money(claim["train_gbp"])
        + min(
            money(claim["hotel_gbp"]),
            money(policy["hotel_cap_gbp"]),
        )
        + min(
            money(claim["meal_gbp"]),
            money(policy["meal_cap_gbp"]),
        )
    )
    excess = requested - allowed

    # Mutation used by the negative test.
    if case == "bad_proposal":
        allowed = requested
        excess = Decimal("0.00")

    supports = ("claim:EC-217", "policy:P-17")
    return [
        Claim(
            "c1",
            "The claim requests £328.40.",
            "calculation",
            f"{requested:.2f}",
            ("claim:EC-217",),
        ),
        Claim(
            "c2",
            f"The reimbursable amount is £{allowed:.2f}.",
            "calculation",
            f"{allowed:.2f}",
            supports,
        ),
        Claim(
            "c3",
            f"The excluded amount is £{excess:.2f}.",
            "calculation",
            f"{excess:.2f}",
            supports,
        ),
        Claim(
            "c4",
            (
                "The claim is not reimbursable in full."
                if excess > 0
                else "The claim is reimbursable in full."
            ),
            "decision",
            "PARTIAL" if excess > 0 else "FULL",
            supports,
        ),
    ]


def verify(
    evidence: dict[str, Evidence],
    claims: list[Claim],
) -> list[Check]:
    claim_e = evidence["claim"]
    policy_e = evidence["policy"]
    travel_date = date.fromisoformat(
        claim_e.payload["travel_date"]
    )

    requested = (
        money(claim_e.payload["train_gbp"])
        + money(claim_e.payload["hotel_gbp"])
        + money(claim_e.payload["meal_gbp"])
    )
    allowed = (
        money(claim_e.payload["train_gbp"])
        + min(
            money(claim_e.payload["hotel_gbp"]),
            money(policy_e.payload["hotel_cap_gbp"]),
        )
        + min(
            money(claim_e.payload["meal_gbp"]),
            money(policy_e.payload["meal_cap_gbp"]),
        )
    )
    excess = requested - allowed
    by_id = {claim.claim_id: claim for claim in claims}

    return [
        Check(
            "evidence_entitled",
            "evidence",
            all(item.entitled for item in evidence.values()),
            "All sources are permitted for this request.",
        ),
        Check(
            "policy_effective",
            "policy:P-17",
            policy_e.is_valid_on(travel_date),
            (
                f"Policy version {policy_e.version} must cover "
                f"{travel_date.isoformat()}."
            ),
        ),
        Check(
            "receipts_present",
            "claim:EC-217",
            bool(claim_e.payload["receipts_present"]),
            "Every claimed line has a receipt.",
        ),
        Check(
            "train_class",
            "claim:EC-217",
            claim_e.payload["train_class"] == "economy",
            "Policy requires economy rail.",
        ),
        Check(
            "requested_total",
            "c1",
            money(by_id["c1"].value) == requested,
            f"Recomputed requested total is £{requested:.2f}.",
        ),
        Check(
            "allowed_total",
            "c2",
            money(by_id["c2"].value) == allowed,
            f"Recomputed allowed total is £{allowed:.2f}.",
        ),
        Check(
            "excluded_total",
            "c3",
            money(by_id["c3"].value) == excess,
            f"Recomputed excluded amount is £{excess:.2f}.",
        ),
        Check(
            "decision_consistency",
            "c4",
            by_id["c4"].value
            == ("PARTIAL" if excess > 0 else "FULL"),
            "Decision must follow the checked excess.",
        ),
        Check(
            "claim_support",
            "proposal",
            all(
                set(claim.support_ids)
                <= {
                    item.evidence_id
                    for item in evidence.values()
                }
                for claim in claims
            ),
            "Every material claim names existing evidence.",
        ),
    ]


def run_case(case: str) -> dict[str, Any]:
    evidence = build_evidence(case)
    claims = form_proposal(evidence, case)
    checks = verify(evidence, claims)

    abstention_messages = {
        "evidence_entitled": (
            "I cannot determine reimbursement because the "
            "required evidence is not authorised for this request."
        ),
        "policy_effective": (
            "I cannot determine reimbursement because no "
            "entitled policy version covers the travel date."
        ),
        "receipts_present": (
            "I cannot determine reimbursement because one or "
            "more required receipts are missing."
        ),
    }
    failed_boundary_checks = [
        check
        for check in checks
        if check.check_id in abstention_messages
        and not check.passed
    ]

    if failed_boundary_checks:
        disposition = Disposition.ABSTAIN
        answer = abstention_messages[
            failed_boundary_checks[0].check_id
        ]
    elif all(check.passed for check in checks):
        disposition = Disposition.ACCEPT
        answer = (
            "No. Reimburse £325.40; exclude £3.00 above "
            "the meal cap."
        )
    else:
        disposition = Disposition.REJECT
        failed = ", ".join(
            check.check_id
            for check in checks
            if not check.passed
        )
        answer = (
            "Do not issue the proposal. Independent checks "
            f"failed: {failed}."
        )

    manifest = {
        "manifest_version": "1.0",
        "request": {
            "request_id": f"req:{case}",
            "question": (
                "Can expense claim EC-217 be reimbursed "
                "in full?"
            ),
            "received_at": datetime(
                2026,
                8,
                30,
                9,
                0,
                tzinfo=timezone.utc,
            ).isoformat(),
        },
        "task_contract": {
            "decision": "reimbursement_eligibility",
            "allowed_effect": "answer_only",
            "required_evidence": [
                "expense_claim",
                "effective_travel_policy",
            ],
            "abstain_when": [
                "missing_receipt",
                "no_effective_policy",
                "unentitled_source",
            ],
        },
        "decomposition": [
            "identify applicable policy",
            "validate evidence",
            "calculate requested and allowed totals",
            "verify material claims",
            "choose disposition",
        ],
        "evidence": [
            asdict(item) for item in evidence.values()
        ],
        "proposal": [asdict(item) for item in claims],
        "checks": [asdict(item) for item in checks],
        "disposition": disposition.value,
        "answer": answer,
    }

    # This is a reproducibility fingerprint, not a signature.
    manifest["trace_hash"] = canonical_hash(manifest)
    return manifest


if __name__ == "__main__":
    for case_name in (
        "positive",
        "stale_policy",
        "bad_proposal",
    ):
        result = run_case(case_name)
        passed = sum(
            item["passed"] for item in result["checks"]
        )
        total = len(result["checks"])
        print(
            f"{case_name:12} "
            f"{result['disposition']:8} "
            f"{passed}/{total} checks | "
            f"{result['answer']}"
        )

Save the listing as reasoning_manifest.py and run python reasoning_manifest.py. The positive case passes all nine checks. The stale-policy case abstains even though the numbers can still be calculated. The mutated proposal is rejected because independent recomputation catches three inconsistencies.

positive     ACCEPT   9/9 checks | No. Reimburse £325.40; exclude £3.00 above the meal cap.
stale_policy ABSTAIN  8/9 checks | I cannot determine reimbursement because no entitled policy version covers the travel date.
bad_proposal REJECT   6/9 checks | Do not issue the proposal. Independent checks failed: allowed_total, excluded_total, decision_consistency.

The negative cases matter more than the happy path. The stale source demonstrates that evidence applicability can block an otherwise coherent answer. The bad proposal demonstrates that citations alone do not validate derived claims. The programme also keeps the proposal visible after rejection, which supports diagnosis without allowing failed content to become the user-facing answer.

An architecture decision instrument

The stack should not be imposed uniformly. Use the following card during architecture review. A “no” is not automatically a release blocker, but it identifies where the design is asking model behaviour to substitute for evidence or control.

The final comparator prevents architecture theatre. More seams mean more latency, code, operating ownership and possible failure. A stage earns its place when it removes a meaningful class of error, strengthens diagnosis or creates a necessary authority boundary. Otherwise, collapse it.

Negative control and failure boundary

Negative controlAsk for 17 × 19 in a low-consequence setting. A deterministic multiplication function returns 323 and can be tested exhaustively over a useful range. Retrieval, model decomposition, self-consistency, citation generation and semantic verification add no epistemic value. The appropriate “reasoning stack” collapses to task contract, calculator receipt and answer. If the full architecture cannot recognise this case, it has become a ritual rather than a decision system.

A second control is a stable, well-separated routing task where a small classifier already meets the required risk and coverage. A larger reasoning model with a manifest may improve explanation while worsening latency and adding new failure modes. The stack must earn its cost against that baseline, not against an imagined unstructured workflow.

There is also a social failure boundary. A detailed manifest can create automation bias if reviewers interpret volume as proof. Evidence displays should foreground unresolved propositions, source conflict and failed checks rather than burying them under a long trace. The reviewer’s job is to decide the residual issue, not to admire the amount of computation already spent.

Open hypothesisA well-designed manifest should improve reliability most on tasks where error sources are heterogeneous: retrieval, calculation, policy and judgement can each fail differently. It may offer smaller gains on homogeneous tasks with a strong end-to-end verifier. The discriminating experiment is a matched-compute comparison across failure injections, measuring task success, false acceptance, diagnostic localisation, latency and review effort. If a monolith matches those outcomes, the proposed seams should be removed.

sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: Two identical answers, one defensible
  M->>E: What must cross a seam
  E-->>M: compare rival explanations
  E->>D: Design the evidence path, not the performance theatre
  D-->>Q: revise scope or act

Design the evidence path, not the performance theatre

Reasoning systems are often presented as a ladder of increasingly capable techniques: chain-of-thought, tool use, search, reflection, verifiers and more test-time compute. Those techniques can improve proposal quality. They do not, by themselves, explain why a particular answer was safe to release.

The architecture decision changes when the proposal is treated as an untrusted intermediate object. Decomposition is chosen for stronger tests, not for prompt fashion. Retrieval produces a qualified evidence envelope, not a pile of chunks. Tools emit receipts. The proposal exposes material claims. Verification follows claim type and preserves independence where consequence demands it. Abstention names the missing proposition. The answer ships with a manifest that can be replayed, challenged and withdrawn.

This design is less dramatic than asking whether a model “really reasons”. It is more useful. It lets a team state what was established, what was inferred, what remained uncertain and why the system answered anyway. It also makes improvement cumulative: a better retriever, calculator, verifier or policy engine can replace one weak stage without forcing the whole system to become a new black box.

The durable question is therefore not “How long was the reasoning trace?” It is: Which path turned this proposal into a defensible answer, and where would the system have stopped if that path had failed?