The future can corrupt the past

Most organisations can recover an old result. Far fewer can recover the exact evidence boundary that made the result legitimate.

The usual audit recipe sounds sensible: retrieve the historical record, load the old model or rule, and compare the result. The weakness hides inside the word “historical”. A record may describe an earlier period while carrying a correction learned much later. A current table may preserve the latest truth about the past, not the knowledge that was available in the past.

This is not an exotic database puzzle. It appears whenever information arrives late, is backdated, is retracted, or changes authority. Employer data, medical coding, sanctions lists, identity attributes, consent, delegated permissions, fraud labels, market prices and operational status all acquire new versions. A system that silently reads the newest version during replay can look more accurate than the system that actually acted. The past can be made wiser by evidence it never possessed.

Temporal database research gives us the first necessary distinction. Valid time is when a fact is true in the modelled reality. Transaction time is when that fact is current in a database and can be retrieved. The dimensions are orthogonal: a correction can be recorded today yet claim validity last month.[1][2] System-versioned tables make database history queryable, which is valuable for forensics and point-in-time reconstruction.[3] But a consequential decision needs one more distinction. A fact can be present and still be forbidden, expired, insufficiently authoritative or outside the declared purpose.

Three judgements, not one

Return to Maya. The correction supports a judgement about the world: £62,000 was the better account of her income. It does not by itself establish that the 09:40 service ignored available evidence. Nor does it show that the employer feed met its latency obligation. One event can therefore produce three different findings:

flowchart TD
  A["The payslip that arrived after<br/>the decision"] --> B["The future can corrupt the past"]
  B --> C["A decision needs three clocks"]
  C --> D["Build the decision slice, not<br/>just the history table"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#D95545,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

A decision needs three clocks

A useful temporal model must tell us when a claim held, when the system could retrieve it, and when the system was permitted to use it.

Imagine four marks on a wall. The first marks when Maya’s income actually changed. The second marks when the employer asserted a value. The third marks when the decision service received and committed that assertion. The fourth marks when the service had authority to use that source for credit assessment. Collapsing these marks into a single timestamp creates a clean table and a dirty explanation.

The established terms valid time and transaction time cover the first and database-facing third marks. Streaming systems add the practical distinction between event time and processing time because late and out-of-order arrivals force a trade-off among completeness, latency and cost.[4] For compound decision systems, transaction time must be interpreted carefully. A fact committed in a central source may still be absent from the snapshot, cache or message set used by one decision node. I will call the time at which a version crosses the decision system’s evidence boundary its record time. This is a practitioner extension, not a replacement for the database term.

The third clock is entitlement time: the interval during which a particular version may be used for a declared principal, subject, purpose, action and resource. Attribute-based access control already evaluates subject, object and environmental attributes against policy, while NIST guidance stresses authoritative establishment, update, readiness and revocation of attributes.[6][7] Cedar makes the request shape explicit as principal, action, resource and context, with time available as request context.[11] Temporal entitlement applies that logic to evidence selection itself.

The smallest useful formalism

Formal notation earns its place here because it prevents a familiar substitution. Let a fact version f contain a value, a valid interval V, a record interval R, a source and provenance. Let a decision d have a target time τ, a cutoff κ, a purpose, a principal, an action and a policy version. Let Allow evaluate the grant and policy state applicable at κ.

The equation assumes crisp intervals and a policy engine able to reconstruct historical grants. Remove either assumption and the result can become indeterminate. If a distributed clock says the cutoff lies somewhere between 09:39:59.970 and 09:40:00.030, while a permission is revoked at 09:40:00.000, no honest Boolean answer follows. Systems such as Spanner expose clock uncertainty to support externally consistent ordering, while Zanzibar’s authorisation design preserves causal ordering amid permission and content changes.[9][8] Many ordinary systems provide neither guarantee.

The decision cutoff is part of the decision, not an audit parameter chosen later. It can be a point, an uncertainty interval, or a consistency token. What matters is that the system declares what it treated as knowable before it acts.

A decision slice is a temporal join with a refusal state

The mechanism can be understood as a join across several histories. Start with versions whose valid interval covers the time the decision is about. Remove every version whose record interval begins after the cutoff. Join the survivors to the policy and grant versions visible at that same cut. Apply source-quality and freshness requirements. Only then evaluate the rule or model. A normal database join tries to return rows. A defensible decision join must also be able to return unresolved.

That refusal state matters when intervals overlap, a revocation sits inside the clock-uncertainty band, or the receipt identifies a source version that has since been deleted. Choosing the newest remaining row would hide the missing guarantee. Choosing the oldest would be equally arbitrary. The system should instead name the failed predicate: fact version unknown, policy revision unavailable, grant order ambiguous, source below threshold, or snapshot not causally comparable.

Consider a service with ten relevant fact versions, three policy revisions and two grant revisions. The Cartesian product contains sixty combinations, most of which were never simultaneously eligible. The decision slice collapses that space by applying the same target and cutoff before evaluation. This is more than an optimisation. It prevents an auditor from assembling a plausible combination that no historical actor could have seen. The slice is therefore both an input set and a proof of exclusion.

Why authorisation needs its own history

A common design versions the business data but evaluates replay with today’s policy engine, today’s permission graph and today’s source classifications. That design answers a useful counterfactual: what would the old input produce under current governance? It does not reconstruct the old decision.

Policies and grants are temporal facts too. A policy can be published, activated, superseded, withdrawn or applied retroactively. A purpose grant can be issued, narrowed or revoked. A source can move from approved to quarantined. Each change needs valid and record semantics. OPA decision logs illustrate the kind of evidence a policy layer can retain: decision identifiers, query input, the policy queried and bundle metadata for audit and offline debugging.[10] The log becomes much more useful when its bundle revision and data snapshot can be joined to the decision receipt.

Provenance has a similar role. W3C PROV provides relations for generation, use, derivation, invalidation and responsible agents, including times that bound an entity’s lifetime.[5] Provenance does not decide whether an input was lawful or reliable. It tells the replay which entity version came from which activity and who was responsible for it. A timestamp is not a licence, and a hash is not an explanation.

Build the decision slice, not just the history table

A temporal database preserves versions. A decision slice proves which versions crossed the evidence and authority boundary of one action.

Adding system-versioned tables is a strong first step. It is not the finish. A compound decision may read from a customer store, a feature service, a policy engine, an authorisation service, a model gateway and a human review queue. Each component can expose a different “as of” state. Replaying each store independently at 09:40 can still combine versions that never coexisted.

The implementation target is a coordinated slice. The slice is the smallest bundle that lets another party answer four questions: What was the decision about? What crossed the evidence boundary? What authority applied? Which executable configuration produced the proposal or action? A replay without the original evidence boundary is a counterfactual, even when every component is individually versioned.

What belongs in the slice

The exact schema depends on consequence and regulation, but the causal minimum is stable. The decision receipt should identify the subject and decision target, the target time, the evidence cutoff, the principal, purpose and typed action, the selected fact versions, the policy bundle revision, the authorisation decision or snapshot, the executable model or rule version, and the output. It should also capture provenance and any uncertainty that affected ordering.

Record what was excluded as well as what was used. An excluded-after-cutoff manifest turns “the system ignored the corrected income” into a testable statement. It shows that the correction was seen during audit but intentionally barred from historical replay. An excluded-for-purpose entry explains why a retrievable attribute did not enter. An unresolved entry preserves missingness rather than converting it into an implicit default.

The negative space is part of the evidence

Two replays can use the same three inputs and still differ in integrity. One may have evaluated five candidate versions and excluded two under explicit rules. The other may have retrieved only the three convenient rows because an index, cache or query filter hid the alternatives. The visible input set is identical, but only the first replay can show that a later correction or prohibited attribute was deliberately kept outside the boundary.

An exclusion manifest need not copy every sensitive value. It can retain the candidate version identifier, source, discovery time, exclusion reason, policy clause and a commitment hash. For a purpose-restricted attribute, the receipt may say that a qualifying version existed but was not disclosed to the decision component. This preserves the control claim without broadening access during audit. Where even the existence of an attribute is sensitive, the manifest can be released selectively to an authorised reviewer.

The practical test is counterfactual. Remove the exclusion record and ask whether a reviewer could distinguish a principled omission from a retrieval failure. If not, the receipt is incomplete. Evidence of non-use is what separates bounded context from accidental blindness.

Several systems already expose pieces of this pattern. Temporal tables retain row versions and support point-in-time queries.[3] Provenance vocabularies can represent generation, derivation and invalidation.[5] Policy engines can log decision identifiers, inputs and bundle metadata.[10] Consistent authorisation systems can bind checks to causally ordered state.[8] Temporal entitlement is the integration rule: no load-bearing item enters unless its time and authority are evaluated against the same decision cut.

A historical-replay benchmark

The most useful test is not whether the system can return an old row. It is whether several replay strategies disagree for reasons the evidence can localise. The benchmark below generates synthetic decisions with late corrections, policy changes, permission changes and missing receipts. It compares three methods:

The lab encodes a simple threshold decision, so it does not prove that temporal entitlement will improve every system. Its purpose is narrower: make three sources of disagreement observable rather than mixing them into “the replay changed”. A positive result permits confidence that the captured slice reproduces the configured decision. A negative result localises what changed. It does not establish that the original source was true, the policy was wise, or the outcome was fair.

from __future__ import annotations

from dataclasses import dataclass
from random import Random
from typing import Literal

Decision = Literal["approve", "deny", "unresolved"]


@dataclass(frozen=True)
class Scenario:
    observed_at_cutoff: float
    corrected_value: float
    threshold_at_cutoff: float
    threshold_now: float
    grant_at_cutoff: bool
    grant_now: bool
    receipt_present: bool


def decide(value: float, threshold: float, grant: bool) -> Decision:
    return "approve" if grant and value >= threshold else "deny"


def captured(s: Scenario) -> Decision:
    return decide(s.observed_at_cutoff, s.threshold_at_cutoff, s.grant_at_cutoff)


def replay_latest(s: Scenario) -> Decision:
    return decide(s.corrected_value, s.threshold_now, s.grant_now)


def replay_fact_time(s: Scenario) -> Decision:
    return decide(s.observed_at_cutoff, s.threshold_now, s.grant_now)


def replay_temporal_entitlement(s: Scenario) -> Decision:
    if not s.receipt_present:
        return "unresolved"
    return decide(s.observed_at_cutoff, s.threshold_at_cutoff, s.grant_at_cutoff)


def scenario(rng: Random, late: float, policy: float,
             auth: float, missing: float) -> Scenario:
    true_score = max(0.0, min(100.0, rng.gauss(62.0, 12.0)))
    if rng.random() < late:
        direction = 1.0 if rng.random() < 0.5 else -1.0
        initial = true_score + direction * rng.uniform(12.0, 28.0)
    else:
        initial = true_score + rng.uniform(-2.0, 2.0)
    initial = max(0.0, min(100.0, initial))

    historical_threshold = 60.0 if rng.random() < 0.5 else 65.0
    current_threshold = historical_threshold
    if rng.random() < policy:
        current_threshold = 65.0 if historical_threshold == 60.0 else 60.0

    historical_grant = rng.random() > 0.05
    current_grant = historical_grant
    if rng.random() < auth:
        current_grant = not historical_grant

    return Scenario(
        observed_at_cutoff=initial,
        corrected_value=true_score,
        threshold_at_cutoff=historical_threshold,
        threshold_now=current_threshold,
        grant_at_cutoff=historical_grant,
        grant_now=current_grant,
        receipt_present=rng.random() >= missing,
    )


def benchmark(n: int = 10_000, *, late: float = 0.15,
              policy: float = 0.08, auth: float = 0.08,
              missing: float = 0.02, seed: int = 183) -> dict[str, float]:
    if n <= 0:
        raise ValueError("n must be greater than zero")
    rates = {"late": late, "policy": policy, "auth": auth, "missing": missing}
    invalid = {name: value for name, value in rates.items()
               if not 0.0 <= value <= 1.0}
    if invalid:
        raise ValueError(f"rates must lie between 0 and 1: {invalid}")

    rng = Random(seed)
    latest_diff = fact_diff = entitled_diff = unresolved = resolved = 0

    for _ in range(n):
        s = scenario(rng, late, policy, auth, missing)
        actual = captured(s)
        latest_diff += replay_latest(s) != actual
        fact_diff += replay_fact_time(s) != actual
        entitled = replay_temporal_entitlement(s)
        if entitled == "unresolved":
            unresolved += 1
        else:
            resolved += 1
            entitled_diff += entitled != actual

    return {
        "latest_divergence_pct": 100 * latest_diff / n,
        "fact_time_divergence_pct": 100 * fact_diff / n,
        "temporal_entitlement_divergence_pct": 100 * entitled_diff / max(1, resolved),
        "unresolved_pct": 100 * unresolved / n,
    }


if __name__ == "__main__":
    for name, value in benchmark().items():
        print(f"{name}: {value:.2f}")

stateDiagram-v2
  [*] --> Claim
  state "Replace the database, preserve<br/>the visible history" as Claim
  state "The bitemporal cube and the<br/>decision cut" as Test
  state "Three judgements, not one" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Where the decision slice breaks

Temporal entitlement improves reconstruction. It does not turn records into reality, settle the law, or make distributed time exact.

A faithful replay can reproduce a bad source

The strongest boundary is simple. A complete slice proves what the configured system consumed and which authority it invoked. It cannot prove that the employer feed, identity register or model feature was substantively correct. Maya’s v1 can be perfectly preserved and still be false. Temporal entitlement cannot make a bad source true. Source quality, appeal, corroboration and correction obligations remain separate controls.

This boundary matters because audit language often slides from “the system used the authorised version” to “the system was right”. The first is an operating-evidence claim. The second is an outcome claim. A mature review states both and records disagreement.

Retroactivity is a rule, not a database trick

Some domains intentionally apply corrections or permissions retroactively. A court order, accounting restatement or policy decision may alter how earlier periods are treated. Temporal entitlement does not forbid that. It requires the retroactivity rule to be explicit, versioned and attributable. The system should be able to distinguish “this value was learned later but governs the earlier period” from “this value was learned later and is being smuggled into a historical reconstruction”.

That distinction is partly legal and organisational. This paper offers a systems grammar, not legal advice. The authority responsible for a domain must define whether a later correction changes past liability, past process validity, present remediation, or all three. Where the rule is disputed, the receipt should preserve the competing interpretations rather than force one Boolean.

Distributed time creates uncertainty bands

A timestamp produced by one node is not automatically comparable with a timestamp produced by another. Replication lag, queue delay, cache activation and clock skew can place a permission change on either side of the apparent cutoff. Spanner’s TrueTime is notable because it exposes clock uncertainty rather than pretending to perfect time, and Zanzibar binds authorisation checks to causal order across changing access-control state.[9][8]

Most systems should not imitate their scale. They should imitate the honesty of the contract. Carry a monotonic revision, snapshot token, log sequence, signed event time, or explicit uncertainty interval. Require a freshness bound for permissions and policy. When the bound straddles a consequential change, abstain, retry against a stronger read, or require review.

Deletion and minimisation can conflict with replay

Keeping every historical input forever is neither necessary nor always permissible. A decision receipt can retain identifiers, hashes, reason codes, provenance pointers and selective evidence manifests while the underlying content follows its retention rule. Yet a hash only verifies a later copy against a commitment. It does not reveal the deleted content or prove that use was authorised. The architecture must therefore declare which future questions it chooses to remain capable of answering.

A practical design separates three layers: an operational record needed for immediate recovery, an audit manifest needed for accountability, and the source record governed by its own retention and access policy. Erasure may intentionally make some exact replays impossible. In that case, the correct historical verdict is “not reconstructable under the retained evidence”, not a guess derived from current state.

Models and people may not replay bit for bit

A deterministic rule can usually be rerun if code and inputs are captured. A sampled model may also depend on weights, runtime, prompt construction, tool results, random seed and provider behaviour. A human reviewer depends on the evidence displayed, instructions, delegated authority and reasons recorded, but not on a recoverable copy of private cognition.

For these systems, exact output equality may be the wrong target. Use tiers. A bitwise replay reproduces the same bytes. A decision-equivalent replay reaches the same typed outcome within declared tolerance. An explanatory replay reproduces the evidence, policy and authority path even when the proposal is stochastic. Exact replay is a systems property, not a model feature.

Define the replay envelope before seeing the rerun

A team should declare what counts as reproduction before it observes the new output. For a threshold rule, the envelope may require the same eligibility Boolean and reason code. For a ranked recommendation, it may require the same top choice, no movement across a prohibited boundary and a bounded change in score. For a human review, it may require the same evidence display, instruction set and authority, while treating the reviewer’s judgement as a fresh decision rather than a mechanical replay.

Suppose a sampled model originally produced risk 0.64 and the captured policy mapped 0.60 or above to manual review. A rerun returns 0.67. Byte equality fails, but decision equivalence holds if the model version, decoding contract, input slice and review threshold are all preserved. If the rerun returns 0.58, the audit should not immediately call the original result irreproducible. It should test whether stochastic variance can cross the threshold at an unacceptable rate. The relevant artefact is then a distribution and acceptance bound, not one lucky rerun.

Predeclaring the envelope blocks another form of hindsight. Without it, reviewers can tighten or relax similarity after seeing whether the replay supports their preferred conclusion. A negative result should therefore identify which layer failed: bytes, typed decision, evidence path, authority path, or effect. This keeps nondeterminism from becoming a blanket excuse and prevents exact sameness from becoming an impossible standard.

The smallest implementation worth building

An organisation does not need a three-dimensional database product to begin. Add immutable recorded_at and source-version identifiers to load-bearing facts. Capture the target time and cutoff in the decision request. Version the policy bundle and authorisation decision. Store an input manifest with included and excluded versions. Record clock or snapshot evidence where ordering matters. Return unresolved when a required join cannot be made.

Then run one incident through three replays. If latest-state, fact-time and temporal-entitlement results are identical, the extra structure may still prove why. If they differ, the design has already paid for itself by locating the leak. The minimum is not a warehouse. The minimum is a decision receipt that refuses to confuse current truth with historical entitlement.