Executive briefA technically healthy request can belong to an indefensible decision

A model gateway reports a successful response, normal token use and latency within its objective. Every span is green. Weeks later, an appeal reveals that the recommendation relied on an expired policy, crossed a delegation boundary and led a downstream service to make an external commitment. The trace explains which components ran. It does not explain which business alternative was chosen, what the decision-maker could see, who was authorised, whether a human changed the recommendation or what the customer experienced.

Decision observability is the ability to reconstruct and test one consequential choice across organisational, data and system boundaries. Its unit is the stable business case and decision point, not an individual model request. It joins the evidence snapshot, context assembly, model contribution, tools and agents, human judgement, policy, commitment, later outcome, correction and learning response.

The Decision Evidence Graph developed here is a twelve-node, time-aware index. Each node records event time, recording time, provenance, access class, retention class and integrity evidence. Typed edges state whether information was used, derived, authorised, overridden, caused an action, arrived later, was corrected or informed learning. A link records a relationship; it does not manufacture a causal claim.

This is not a mandate to keep every prompt. Indiscriminate storage can create surveillance, cost, privacy and investigation failure while burying the decisive event. The Decision Observability Contract starts from bounded questions, selects minimum sufficient evidence and assigns access, correction and retention. The resulting decision receipt points to protected evidence and explicit gaps rather than generating a retrospective story.

The strongest rival: request telemetry is sometimes enough

A low-risk summarisation tool may create no external effect, influence no consequential decision and allow immediate correction. Standard request traces and logs can support reliability and debugging without a decision graph. Extra joins and retention would be disproportionate. Escalate to decision observability when consequence, cross-system authority, external commitment, delayed outcome or redress creates questions that request evidence cannot answer.

Observable unitRequest, workflow, model and decision views answer different questions

Request observability diagnoses a call. Workflow observability follows steps, retries and state. Model observability examines inputs, outputs, versions and performance. Decision observability connects those views to a business question, alternative set, accountable principal, external commitment and later outcome. None replaces the others, and a trace identifier is not a durable case identifier.

OpenTelemetry provides stable tracing concepts such as spans, context and links, and its logs data model supports event correlation. W3C Trace Context enables interoperable propagation. CloudEvents supplies portable event context. These standards can carry and connect technical evidence, but none defines the whole business decision record. The case identifier and typed decision relationships remain domain contracts.

Decision Evidence GraphTwelve nodes preserve the choice without inventing a story

Every node separates event time from recording time, carries provenance and states who may inspect it and for how long. Integrity evidence can come from independent systems, signed receipts or append-only controls appropriate to consequence. The graph stores pointers when copying the source would violate minimisation or create inconsistent duplicates.

Temporal truthKeep the evidence state that existed when the decision occurred

Decision reconstruction needs at least four clocks. Event time says when a business fact applied. Recording time says when a system captured it. Decision time fixes the evidence snapshot and applicable policy. Outcome or correction time records what arrived later. A late record can improve future knowledge without changing what was available to the original decision-maker.

Never overwrite a stale document or mistaken value in place and then claim the corrected store represents historical context. Preserve the original version, the access and date filters, the exclusion reason and the later correction. This supports a fair question: was relevant evidence available, permitted and reasonably discoverable at that time? It also prevents hindsight from turning a weak decision into an apparently well-informed one.

Joining solely by timestamp is unsafe. Independent cases can occur together; clocks drift; batch records arrive late. Use stable domain identifiers, scoped correlation and typed relationships. Trace identifiers can link technical work, but a new trace may continue the same case after a human pause and one trace may serve several business records. The case identity outlives any execution session.

Accountability pathTrace authority to the external commitment, not only to the model

A principal may delegate analysis to a human, model, agent or service, but delegation has purpose, scope, tenant and expiry. Each boundary narrows authority. Record the applicable policy check and the actor that accepted the final judgement. A service credential proves the caller workload; it does not prove the sponsoring person or business owner authorised a consequential act.

Human intervention needs evidence proportionate to its claim. A click proves that a control was activated. Meaningful review may also require the recommendation shown, evidence added, time available, alternative chosen, reason and final judgement. Do not capture hidden reasoning or demand private chain of thought. A concise reason code and cited added evidence can establish an accountable intervention without surveillance.

The commitment node is the external effect: a payment, communication, denial, recommendation, record write or obligation. Tool and protocol receipts should identify whether the effect committed, failed, was reversed or remains uncertain. An observed system must not be the sole witness to its own high-consequence action; independent transaction records or protected downstream acknowledgements strengthen integrity.

Privacy and utilityMinimise by decision question, not by arbitrary sampling

Start with the questions that an operator, affected person, auditor or owner must answer. Select the node fields and integrity evidence needed for those questions. Store references to protected artefacts when direct copying is unnecessary. Irreversibly minimise data that has no justified use, and assign access and retention by evidence class rather than keeping the whole trace indefinitely.

More logging can reduce accountability. In a high-volume workflow, full prompt and response archives can swamp investigators, expand sensitive access and conceal the one policy failure that caused an external effect. Decisive receipts, explicit gaps and typed links make the signal visible. Sampling for performance telemetry must not drop rare commitments, overrides, appeals or control failures that the contract declares mandatory.

Learning after commitmentJoin delayed outcomes to the eligible decision population

Outcomes arrive on different schedules. Operational completion may be immediate, a customer appeal may take days, a financial reversal may take weeks and distributional effects may require a longer window. Define the population eligible for evaluation at decision time, then join outcomes using stable case identity and documented coverage. Otherwise, only easy or escalated cases may receive labels.

An outcome observed after a decision is associated evidence. It is not automatically caused by the model, route, human or policy. Causal attribution requires an identification design, such as a defensible experiment or quasi-experimental comparison, with assumptions exposed. The graph preserves exposures, interventions and timing so analysts can test claims later without writing causality into every edge.

Learning links name which outcome label or incident changed which threshold, evaluation, policy or process, and which population the change is expected to affect. This closes an accountability loop without allowing a correction to erase the original choice. Model cards, datasheets and scenario evaluation provide lifecycle context, but the decision receipt records what was actually deployed and observed in the case.

Operational useA decision receipt turns an incident into answerable questions

Without a case anchor, investigators search request logs by time, collect several plausible prompts and cannot tell which output influenced the commitment. With a decision receipt, they start from the external effect, traverse caused-action and authorised-by edges, recover the evidence snapshot, identify an override or missing policy check and see which outcome and redress records followed.

The receipt is a compact index, not an after-the-fact explanation. It names missing nodes and broken edges. A gap can be more accountable than a generated narrative that fills uncertainty. Alerts should fire only when an owner can act: a commitment without authority evidence, a policy-ineligible route, an absent independent receipt, an overdue outcome join or a correction that has not reached affected cases.

Reconstruct one disputed commitment from the outside in

Begin with the durable external effect, not the model call. A payment ledger, sent communication, case decision or system-of-record write identifies what the enterprise committed. The independent effect receipt supplies its business identifier, commit state, event time and acting workload. From there, traverse the caused-action edge to the decision point and recover the alternative selected. This direction prevents an investigation from stopping at a plausible generated answer while missing what the organisation actually did.

Next, resolve the accountable principal and each delegation hop. Retrieve the policy version and authorisation decision applicable at decision time. Check whether a human or agent remained within the approved purpose, tenant, action and expiry. If a service token replaced the sponsor or a later policy version overwrote the original, record the authority edge as broken. Do not repair the historical graph by assuming that a current permission applied earlier.

Then rebuild the evidence view. The snapshot lists authoritative sources, versions, effective dates, expiry, missing facts and excluded candidates. The context-assembly node explains which retrieval, memory and transformation produced the model input without requiring a permanent copy of all content. A protected digest or source reference can prove which version was used. Reviewers can now distinguish unavailable evidence, an access-filter exclusion, failed retrieval and a model contribution that ignored material context.

Human evidence closes a different question. Record what recommendation and supporting material the reviewer saw, whether an alert was visible, what additional evidence entered, which alternative was chosen and a concise reason. Time on screen may help diagnose a rushed process but is not proof of attention. Private deliberation and hidden chain of thought remain outside the contract. Accountability rests on observable evidence, authority, judgement and commitment rather than speculative mental reconstruction.

Finally, attach later outcomes, appeals, containment, correction and redress without modifying the original nodes. Verify that correction reached any dependent commitment and that the affected person or operator received the promised response. The graph can show that a decision preceded an outcome and that a particular commitment followed the decision. It should label stronger causal conclusions as unsupported unless a valid study design supplies them. The resulting review identifies a control change, its owner, affected population and validation case.

Boundary testsTen cases choose evidence by consequence and question

The cases demonstrate proportionality. Low-consequence tools retain ordinary diagnostic evidence. Consequential workflows add only the nodes needed to answer authority, evidence, commitment, outcome or redress questions. High volume strengthens the need for semantic selection rather than weakening it. Rare appeals and commitments can require complete receipts even when routine request telemetry is sampled.

Decision Observability ContractEight questions establish what must remain answerable

The contract creates a decision receipt containing case and decision identifiers, applicable policy, evidence snapshot, principal and delegation, model and tool contributions, human intervention, commitment, control events, outcome links, corrections and gaps. It is assembled during execution from signed or access-controlled records. A generated summary may help a reviewer navigate it, but the summary is never the evidence.

Interactive instrumentDecision Observability Coverage Test

Choose a scenario or alter the record. The test returns one of exactly five actions. It does not calculate a maturity score and never recommends capturing hidden reasoning. Results identify the unanswerable question, graph gaps, minimum next evidence, privacy change, witness need, alert, validation case and accountable owner.

Evidence and governance boundaryObserve enough to answer, not enough to surveil

The Decision Evidence Graph, Decision Observability Contract, decision receipt and Coverage Test are this article's synthesis. Decision Provenance motivates visibility into information flows across systems of systems. PROV-O provides entity, activity and agent relationships. OpenLineage provides job, run, dataset and lifecycle concepts. They inform the graph but do not jointly certify that a decision was correct, lawful or causally responsible for an outcome.

OpenTelemetry tracing is stable except where individual sections say otherwise, and current documentation identifies version 1.60.0. Its generative AI conventions remain an active surface with status varying by area, so deployments should pin the convention and stability designation they use. Trace Context and CloudEvents support propagation. A business case identifier, evidence effective time, decision semantics and retention policy still require domain design.

NIST log-management guidance supports ownership and operational process. NIST AI guidance, ISO/IEC 42001 and applicable EU AI Act provisions support lifecycle measurement, records, human oversight and monitoring. Scope depends on the system, actor, jurisdiction and use. This article offers an evidence architecture, not a legal conclusion or a guarantee of compliance.

Model cards, datasheets and scenario evaluation document intended use, data and measured limitations. The decision graph links the versions actually relevant to a case. It must not store private chain of thought, treat model confidence as calibrated decision confidence, or assume a human control was meaningful because it was clicked. It preserves observable reasons, evidence and actions proportionate to the decision.

At operating review, begin with questions that could not be answered during incidents, appeals or outcome analysis. Examine missing mandatory nodes, broken authority and action edges, overdue outcome joins, correction propagation, access violations, retention exceptions and noisy alerts. Reduce evidence that no longer serves a declared question. Expand only when a consequential question remains unanswerable and the privacy basis is explicit.

Assign stewardship at node and edge level. Domain owners define the decision question, alternatives, commitment and outcome. Data stewards protect source meaning and effective time. Platform teams maintain propagation and receipt reliability. Security and privacy teams govern access, integrity and minimisation. Risk, compliance or internal audit can challenge the record without becoming its daily operator. A shared evidence service may host the index, but it does not inherit accountability for the decision. During incidents, one named coordinator follows broken links to their owners and records which questions remain unresolved. This allocation keeps the graph from becoming an unowned data lake or a central team’s claim that instrumentation alone supplies accountability. These responsibilities should appear in the decision receipt and operating runbook.

The executive standard is answerability with restraint. Teams should be able to reconstruct what was knowable, who could act, how automation contributed, what was committed, which human intervention mattered, what happened later and how challenge or correction proceeded. They should also be able to say what the record cannot establish. That combination supports accountability more reliably than a perfect-looking request dashboard or an unlimited archive.

Source recordWorks cited

1. Singh, J., Cobbe, J. and Norval, C. Decision Provenance: Harnessing data flow for accountable systems.
2. W3C. PROV-O: The PROV Ontology.
3. OpenTelemetry. Tracing API.
4. OpenTelemetry. Logs data model.
5. OpenTelemetry. Generative AI semantic conventions.
6. W3C. Trace Context.
7. Cloud Native Computing Foundation. CloudEvents Specification.
8. OpenLineage. Object Model.
9. Google. Monitoring Distributed Systems.
10. NIST. SP 800-92 Guide to Computer Security Log Management.
11. NIST. Artificial Intelligence Risk Management Framework.
12. NIST. Generative Artificial Intelligence Profile.
13. ISO. ISO/IEC 42001 Artificial intelligence management system.
14. European Union. Regulation (EU) 2024/1689.
15. Mitchell, M. et al. Model Cards for Model Reporting.
16. Gebru, T. et al. Datasheets for Datasets.
17. Liang, P. et al. Holistic Evaluation of Language Models.