AI Research Library

Document Intelligence as Evidence Reconstruction

Rajesh Ranjan Mahapatra

Reliable document systems do not answer by compressing a file pile. They rebuild a typed, versioned evidence state in which every claim, identity, conflict, rule and decision remains traceable to its source.

© 2026 Rajesh Ranjan Mahapatra. 28 min read

Document Intelligence as Evidence Reconstruction

Reliable document systems do not answer by compressing a file pile. They rebuild a typed, versioned evidence state in which every claim, identity, conflict, rule and decision remains traceable to its source.

TLDR

  • Document intelligence is reliable only when extraction and generation remain reversible views over a typed evidence state.
  • The mechanism preserves source regions, propositions, identities and capacities, temporal relations, conflicts, rule applications and reviewer decisions as separate objects.
  • Use reconstruction when an answer depends on several documents, changing versions, disputed identity, conditions or consequential downstream action.
  • A clean one-page form is the negative control: direct extraction may be better, while an explicit graph can add cost or encode the wrong ontology.
  • Release on conflict recall, false-merge rate, temporal mutation and claim support, rather than page throughput or summary fluency.

Opening case

The amendment that a good summary swallowed

A supplier file contains a forty-two-page master agreement, a signed one-page amendment and a later purchase order. The agreement says invoices are payable within thirty days. The amendment replaces that clause with forty-five days. The purchase order, generated from an old template six weeks later, still says “Net 30”. A fluent summary reports that the parties agreed payment terms and that the latest order lists thirty days.

The summary is readable, faithful to visible words and operationally useless. The decision-maker needs to know which proposition applies to the invoice now, whether the amendment was validly executed, whether it covered this order, and why the later purchase order repeated the earlier term. The disagreement is not noise to be averaged away. The disagreement is the material fact.

The same signature page contains “A. Shah” twice. One appearance is a director; the other is a witness. A conventional entity matcher may decide that the names refer to one person. Even if that identity guess happens to be correct, the capacities remain different. A director’s capacity cannot be inferred from a witness signature, and a shared name does not carry authority across roles.

This small file exposes the central burden of document intelligence. Perception must recover words, layout and regions. Extraction must propose fields and clauses. Identity resolution must connect mentions without collapsing capacities. Temporal reasoning must distinguish execution, effect and knowledge. Reconciliation must preserve support, supersession and contradiction. Rules must run against admitted evidence, while judgement remains with the authorised reviewer. The answer should be generated only after that state exists.

Figure 1. An evidence mosaic becomes an argument graph

Evidence fragments transformed into a claim and argument graph Document pages, tables, signature regions and dates on the left flow through typed proposition extraction in the centre into a graph on the right. Solid teal edges indicate support, dashed coral edges indicate conflict, and a double indigo edge indicates supersession. Source mosaic Net 30 A. Shah Director / Witness? Effective 01 Jul Clause 8 replaced Typed claims ORG-ACME payment_days = 30 Amendment payment_days = 45 A. Shah + capacity Argument graph Current payment term? Net 45 amendment Net 30 order Clause 8 source Reviewer decision supports conflicts supersedes requires
Practitioner synthesis, illustrative. The graph does not “understand the whole file” by magic. It preserves a chain from native evidence to typed propositions, explicit relations and an authorised decision. Colour is paired with edge style and labels.

Long-context vision-language models have improved the ability to locate text, tables and figures across PDFs. Yet a benchmark such as MMLongBench-Doc deliberately includes cross-page and unanswerable questions because evidence integration remains difficult even after perception succeeds.[3] M-LongDoc extends the challenge to documents with hundreds of pages and open-ended explanations.[4] These tasks show why longer context is useful. They do not establish that a generated answer preserves document identity, temporal effect or accountable judgement.

Evidence reconstruction is the missing systems layer. It converts a collection of mutable, multimodal files into a minimal case state that can be inspected, challenged, corrected and replayed. Summaries remain valuable, but only as disposable views over that state.

Part I

Evidence before prose

A document is more than its extracted text. Meaning may depend on page order, a table boundary, a struck-out phrase, a stamp, a footnote marker, a signature region or a chart legend. DocVQA was created because answering questions about document images requires visual and textual evidence together.[1] LayoutLMv3 similarly learns from text, image patches and their alignment rather than treating layout as decoration.[2]

The first architectural mistake is therefore to call OCR output “the document”. OCR is a transformation. It can be useful, cheap and searchable, but its errors become upstream causes for every later stage. OHRBench was designed to measure this cascade through retrieval and generation, demonstrating that OCR quality cannot be judged only by character accuracy.[5] A missing minus sign may change a figure. A broken row may move a value to the wrong entity. A lost negation may reverse a rule.

The one-word amendment thought experiment

Take two otherwise identical amendments. The first says, “The borrower may dispose of the asset with written consent.” The second inserts one word: “The borrower may not dispose of the asset with written consent.” Hold every other feature constant. If OCR loses the word “not”, both documents produce the same text stream. If a summariser compresses either into “asset disposal is subject to consent”, both produce the same narrative. The system has erased the causal feature that should change the decision.

A reliable pipeline retains the page image, the exact region, the OCR hypothesis, the parser version and the proposition that carries negation explicitly. The extracted proposition can be wrong, but it remains reversible. A reviewer can open the region that produced it and correct the earliest faulty object. Reversibility is more valuable than a polished explanation built on an invisible transformation.

Figure 2. Summarisation flattens the relation that reconstruction must preserve

Comparison of a summary path and a reconstruction path Three documents appear on a timeline. The summary path compresses them into one ambiguous sentence, while the reconstruction path preserves an explicit supersession edge and an unresolved conflict. One file, three propositions AgreementNet 30 AmendmentNet 45 Later orderNet 30 Summary path “Payment terms were updated; latest order says Net 30.” Readable, but precedence and contradiction are hidden. Reconstruction path C1: Net 30 C2: Net 45 C3: Net 30 C2 explicitly supersedes C1 conflict
Synthetic worked case. The summary contains the words but loses the relation. Reconstruction preserves source identity, explicit supersession and the later contradictory order.

The source manifest is the first evidence object

For every admitted file, record the native bytes, content hash, repository identity, version, access decision, page count, render output, text layer, OCR engine and parser build. Each page region receives stable coordinates. Derived text points back to the page and the exact transformation that produced it. W3C PROV provides a domain-neutral vocabulary for entities, activities and agents involved in producing a derived object.[11] A document system can specialise that model without pretending provenance alone determines evidential weight.

Evidence role must also stay explicit. “A mark is visible in the signature block” is an observation. “OCR reads A. Shah” is an extraction. “The identifier matches an approved record” is a verification. “The sender states that approval occurred” is an assertion. “The amendment probably narrows clause 8” is an inference. “The reviewer accepts this treatment” is an authorised decision. Combining these into one confidence score hides different kinds of uncertainty and different owners.

Figure 3. The transformation ledger keeps every interpretive step openable

Layered transformation ledger from source bytes to decision view Six offset layers show native file, rendered page, OCR and layout, source span, typed proposition and decision view. A provenance spine on the right links every layer. A reversible evidence stack 1. Native file, version, hash and entitlement 2. Rendered page and visual regions 3. OCR, layout and parser hypotheses 4. Source span with coordinates and quality 5. Typed proposition, identity, time and status 6. Reviewer view, rule output and narrative Provenance spine: source, transform, agent, version, time
Architecture cutaway, illustrative. A correction at layer three should invalidate dependent objects at layers four to six. Editing only the final paragraph leaves the faulty state available for reuse.

Part II

The reconstruction pipeline

Reconstruction begins with an investigation question, not a generic request to “analyse these documents”. The case envelope names the subjects, document classes, time frame, jurisdictions or policy domains, reviewer role and downstream exclusions. These fields limit what the system is allowed to admit and which questions it may answer. A discovered document outside that envelope becomes a proposed scope change rather than silent context expansion.

The pipeline then performs seven separable operations: admit and fingerprint sources; render and segment pages; extract candidate spans; normalise spans into typed propositions; resolve entities and capacities; reconcile propositions across time and scope; and apply deterministic validation or decision rules before generating a review view. Models can assist several operations. No model output becomes authoritative merely because it appears later in the pipeline.

A proposition is smaller than a summary and richer than a field

A useful proposition contains at least a subject, capacity, predicate, value, qualifiers, negation, modality, source span, valid time, evidence role, confidence and review state. For example, “ORG-ACME, acting as supplier, has payment_days 45 from 1 July, according to page 1 of signed amendment 01, verified, superseding claim C1.” That object is verbose compared with a field-value pair. Its extra structure is exactly what permits comparison and correction.

claim = subject × capacity × predicate × value × scope × time × source × status

The multiplication sign indicates that each dimension constrains the comparison frame. Two claims about the same subject and predicate may still be incomparable if they concern different capacities, assets, jurisdictions or periods. Removing a dimension increases apparent agreement by merging claims that should remain distinct.

Minimal worked example: three payment-term propositions
ClaimTyped propositionSource stateRelation
C1ORG-ACME · supplier · payment_days = 30 · from 1 JanMaster agreement, page 4, extractedSuperseded by C2
C2ORG-ACME · supplier · payment_days = 45 · from 1 JulSigned amendment, page 1, verifiedExplicitly supersedes C1
C3ORG-ACME · supplier · payment_days = 30 · from 12 AugPurchase order, page 2, extractedConflicts with active C2

A summary-first system may choose C3 because it is most recent. A date-first rule may choose C3 for the same reason. Reconstruction refuses that shortcut. The amendment explicitly states what it replaces; the purchase order does not. Recency is evidence about sequence, not authority. The output is therefore a review question: does the order validly vary the agreement, or is it an operational error?

Document-level natural language inference provides useful machinery for identifying entailment, contradiction and evidence spans. ContractNLI asks whether hypotheses are entailed, contradicted or not mentioned in a contract and links decisions to supporting spans.[6] DocInfer builds a hierarchical document graph and selects evidence for document-level inference.[7] These are important components. A production investigation must add source admission, identity, temporal effect, rule ownership, reviewer authority and correction semantics.

Why the evidence graph is not just a knowledge graph

Enterprise knowledge graphs often aim to state durable facts about entities. An evidence graph carries competing claims, their provenance and their review status. It is comfortable with “unknown”, “conflicting” and “not searched”. It may contain two incompatible values for the same frame because suppressing one would destroy evidence. Its edges describe how propositions relate, while a separate decision object records which treatment an authorised reviewer accepted.

This distinction prevents a common failure: promoting a model inference into enterprise knowledge before the case is settled. A candidate identity match can exist as a proposal with a score and compared attributes. It should not silently merge records, transfer permissions or rewrite all dependent propositions. Promotion is a governed transition with an owner, rationale and reversible change set.

Recent work is beginning to measure this gap directly. DocScope frames long-document understanding around verifiable reasoning trajectories and precise evidence grounding, rather than answer text alone.[9] The broader hypothesis is that evidence reconstruction will become a reusable intermediate representation for such systems. That hypothesis remains open: a future end-to-end model may preserve equivalent structure internally and expose it reliably. Until that is demonstrated under mutation and review, explicit state is easier to test.

Part III

The argument in one passChart · flow
flowchart LR
  A["The amendment that a good<br/>summary swallowed"] --> B["Evidence before prose"]
  B --> C["The reconstruction pipeline"]
  C --> D["Identity, time and conflict<br/>are separate problems"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#087A98,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step
A structural route through Document Intelligence as Evidence Reconstruction, from The amendment that a good summary… to Sources. Reading use: follow the relation, then test it against the surrounding evidence and limits.

Identity, time and conflict are separate problems

Entity resolution asks whether two records refer to the same real-world entity. Systems such as Ditto cast matching as a learned comparison between record pairs and show how language models can improve matching quality.[8] That capability is useful, but a consequential document investigation needs two extra disciplines. Match uncertainty must survive downstream, and identity must remain distinct from capacity.

The same-name role-switch thought experiment

Imagine two bundles that differ in one feature. In bundle A, “A. Shah” signs an amendment as director and a second “A. Shah” signs as witness. In bundle B, a verified identifier proves both signatures belong to the same person. The identity answer changes. The authority answer may not. Even in bundle B, the witness signature does not become a director action. A system that stores only a person node and a generic “signed” edge cannot represent the distinction.

The minimum structure is mention → candidate entity → resolved entity, followed by a separate entity → capacity → instrument relation. Capacity can include director, trustee, attorney, beneficiary, customer, supplier or reviewer. It has its own evidence and valid interval. A correct person match can still support an incorrect authority claim when role evidence is collapsed.

Figure 4. Identity confidence does not carry capacity authority

Identity and capacity evidence shown on separate axes A two-dimensional field has identity evidence on the horizontal axis and capacity evidence on the vertical axis. Points show same-name mentions that can have strong identity evidence but weak capacity evidence. Two questions, two evidence axes Identity and capacity supported Capacity evidence, identity unresolved Same person, wrong role risk Neither supported “A. Shah” name match only Verified ID + director clause Role stated, person ambiguous Identity evidence → Capacity evidence → weakstrong weakstrong
Illustrative decision field. High identity similarity with weak role evidence is the dangerous quadrant. The system may have found the person while still lacking the capacity required for the proposition.

Time has several clocks

Document collections carry execution date, stated effective date, filing date, receipt date, knowledge date and source-system version time. These clocks answer different questions. A later-discovered amendment may have changed the legal or contractual position at an earlier transaction date. The institution may nevertheless have acted on the documents it possessed then. Reconstruction should show both the historic effect and the historic knowledge state without treating either as the other.

Every proposition therefore needs a valid-time interval and a recorded-time stamp. Supersession should be an explicit relation with scope: amendment B replaces clause 8 of instrument A from 1 July for invoices under schedule X. A later document does not automatically supersede an earlier one. It may be a copy, an assertion, an ineffective draft or a statement about another scope.

Contradiction is a relation, not a bad score

Claims can be equivalent, corroborating, superseding, narrowing, conflicting, missing or incomparable. The classification depends on identity, capacity, predicate, scope and time. Two different values are not contradictory when they apply to different periods. Two identical sentences may conflict when their subjects only appear similar. “No evidence found” is different from “repository not searched”, “access denied” and “referenced schedule missing”.

Figure 5. A contradiction surface separates semantic incompatibility from temporal overlap

Conflict classification surface A two-dimensional surface plots temporal overlap horizontally and semantic incompatibility vertically. Regions indicate corroboration, supersession, narrowing and active contradiction. Relation classification requires both axes Corroborating or equivalent Superseding or narrowing Active contradiction Net 45 vs active Net 30 Explicit replacement Independent support Temporal overlap → Semantic incompatibility → separate periodssame active period lowhigh
Illustrative relationship map. Contradiction requires incompatible propositions inside the same active frame. Explicit scope and time can move a pair from conflict to supersession or simple incomparability.

Part IV

Rules operate on evidence; they do not manufacture it

Once propositions are typed, deterministic checks become possible. A graph shape can require every accepted claim to have a source span, evidence role and reviewer state. W3C SHACL defines a language for validating RDF graphs against conditions.[13] Decision Model and Notation offers a common way to specify business decisions and rule dependencies.[14] LegalRuleML adds constructs for legal norms, permissions, prohibitions and defeasibility.[15]

These standards are representation tools, not sources of truth. A rule may test that a required schedule is present, that a date falls inside an interval, or that two jointly required signatories are distinct. It should not invent an absent clause or decide a disputed interpretation. The rule output carries its input claim identifiers, rule version and execution result. A reviewer decision then cites both the evidence and the rule output.

Authority must remain a separate object from confidence. An extraction with 0.99 model confidence is still extracted evidence. A lawyer, underwriter or investigator may accept or reject it under a defined role. Conversely, a human decision can have authority while remaining contestable and subject to later correction. The system should record who decided, under which role, for what purpose and effective period.

The case pack should lead with the investigation question, admitted sources, parties and capacities, material timeline, agreed propositions, conflicts, missing evidence and decisions. Each material sentence links to the proposition graph, and every proposition opens to its page region. Protected values such as names, amounts, dates and clause references are rendered from structured state rather than rewritten by the language model.

Corrections occur at the earliest wrong layer. A bad OCR token updates the extraction. A false identity merge is reversed and dependent capacity claims are recalculated. A new amendment invalidates affected propositions and marks issued packs stale. A revised interpretation creates a superseding decision object; it does not overwrite the historic decision. This event-sourced pattern permits replay and impact analysis.

Figure 6. Choose the mechanism from ambiguity and consequence

Decision frontier for choosing extraction, reconstruction or human-led investigation A field plots document ambiguity horizontally and decision consequence vertically. Low-low cases use direct extraction, middle cases use evidence reconstruction and high-consequence high-ambiguity cases use a human-led investigation with machine assistance. Mechanism placement, not universal graphing Direct extraction Typed evidence reconstruction Human-led investigation clean one-page invoice amended contract bundle disputed authority or privilege Document ambiguity → Decision consequence → lowhigh lowhigh
Decision instrument, illustrative. Reconstruction earns its cost when ambiguity and consequence rise together. High-consequence interpretation remains human-led even when machines assemble and test the evidence.

Documents are untrusted input

A retrieved page can contain hidden text, comments, tracked changes, QR codes or instructions directed at the model. Research on indirect prompt injection shows how external content can redirect an LLM-integrated application.[17] The parser must therefore treat document content as evidence, never as authority to change tools, scope, access or destination. Tool calls come from the case workflow. Network access and disclosure remain behind deterministic policy.

The boundary matters most when the generated pack triggers another system. Permission to read a document does not imply permission to send an email, update a record or disclose the synthesis. A decision receipt should bind purpose, identity, evidence version, rule version and reviewer authority. An effect receipt should record what the downstream system actually changed. Unknown outcomes remain explicit until readback or reconciliation confirms them.

Figure 7. Corrections propagate through dependency, not prose editing

Correction and replay lifecycle A circular lifecycle moves from source version to extraction, proposition, relation, decision and issued pack. A correction arrow invalidates dependent objects and returns the case to review. Replayable case state Sourceversion Extractionobjects Propositionsand links Reviewerdecision Issuedcase pack Observedoutcome New scan, split identity or revised interpretation invalidateand replay
Temporal sequence, illustrative. A change creates a new version and a dependency-aware replay. Historic packs and decisions remain intact, while affected outputs become stale and receive owners.

Part V

Evaluate the system by material failure

Field accuracy is necessary and insufficient. A pipeline can read every amount correctly and still merge two people, apply an amendment to the wrong period, hide a contradiction or cite a superseded page. Evaluation must follow the evidence chain from source admission to observed outcome. NIST AI RMF provides a general risk-management frame for mapping, measuring and managing system risks, while the concrete tests remain use-case specific.[16]

A release corpus should contain clean forms and difficult combinations: missing pages, duplicate files with subtle changes, handwriting, low-resolution scans, multi-column tables, inconsistent identifiers, same names in different capacities, amendments without originals, ineffective drafts, contradictory dates, conditional clauses, missing schedules and malicious embedded instructions. Gold state should include evidence spans, accepted propositions, legitimate alternatives and required review questions. Do not force one answer where experts reasonably disagree.

Release evidence by layer
LayerMeasureMaterial failure
Source integrityFile, version, page and hash accuracyAmended or rescanned page omitted
PerceptionRegion, text, table and reading-order accuracy by quality cohortNegation or row association lost
PropositionPredicate, qualifier, negation, condition and source supportClause normalised into a broader claim
IdentityFalse merge, false split and uncertainty calibrationSame name transfers capacity
Temporal relationEffective interval and supersession mutation testsLater document treated as automatically controlling
ReconciliationConflict recall, false conflict and missing-evidence classificationNarrative chooses one unresolved claim
RulesInput lineage, version and boundary testsRule runs on unadmitted or stale evidence
Review and outcomeCorrection trace, replay, destination readback and reopen ratePack corrected but dependent action remains wrong

Failure injection is the primary test

Mutation reveals whether the architecture uses the structure it claims to use. Delete a negation and expect the proposition or decision to change. Swap two same-name identities and expect a protected merge to fail. Move an effective date and expect the active claim set to change. Remove a referenced schedule and expect insufficient evidence, not a guessed default. Insert a hidden instruction and expect no tool or scope change. Simulate a destination timeout and expect an unknown outcome followed by readback, not a blind retry.

Figure 8. A failure-injection matrix tests causal use of the evidence state

Failure injection matrix for document intelligence Rows list mutations to source, extraction, identity, time, rules and security. Columns show expected changes in claims, relations, decisions and outcomes. Mutation → expected observable ClaimRelationDecisionOutcome Drop “not”Swap same nameMove effective dateRemove scheduleEmbed instructionTimeout export changeblockchangemissingno actionunknown must movemust stopmust moveabstaincontainedreadback Green circles mark the first state expected to change; hatched cells mark the protected downstream invariant.
Test design, illustrative. A passing test proves more than final-answer accuracy because the expected intermediate change is named. The mutation asks whether the claimed mechanism is causally active.

The negative control: one clean invoice

A single verified invoice contains one gross total, one currency and no amendment chain. Direct extraction with a schema validator is the stronger baseline. Building a proposition graph, identity model and reviewer workflow adds latency and objects without changing the decision. This case matters because it prevents evidence reconstruction from becoming architecture theatre.

The mechanism earns its cost when the case crosses a threshold: multiple authoritative sources, material versioning, identity ambiguity, conditional rules, contradiction, privileged access or downstream consequence. The threshold should be measured against the simplest baseline. If an end-to-end model can preserve source support, conflict recall, identity separation and temporal behaviour under controlled mutations at lower total cost, the explicit graph should shrink or disappear.

Benchmark labels also need provenance

Evaluation data is another document collection. In August 2026, MMLongBench-Doc-V2 reported brittle answer normalisation and a concentration of wrong, ambiguous or incomplete annotations in the original benchmark.[10] The lesson is broader than one dataset. Preserve benchmark version, annotation history, adjudication and metric code beside every score. A model can appear to regress when the reference is wrong, or appear to improve by exploiting a metric that ignores the evidential question.

Production monitoring should stratify by document type, scan quality, language, amendment depth, entity ambiguity and case consequence. Track reviewer corrections at the layer where they occur, conflicts found after sign-off, reopened cases, stale-pack exposure, access denials, source latency and unknown outcomes. Pages processed and summaries accepted are activity measures. Better value measures include first-time-complete evidence packs, earlier conflict discovery, reduced duplicate review and fewer downstream defects.

Executable artefact

How the claim changes under pressureChart · state
stateDiagram-v2
  [*] --> Claim
  state "The one-word amendment thought<br/>experiment" as Claim
  state "The source manifest is the<br/>first evidence object" as Test
  state "A proposition is smaller than<br/>a summary and richer than a<br/>field" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]
The revision loop begins with The one-word amendment thought experiment and keeps counterexamples inside the reasoning path. Reading use: follow the relation, then test it against the surrounding evidence and limits.

A minimal investigation pipeline

The accompanying Python programme uses only the standard library. It represents source spans, typed claims and relations; requires explicit supersession; preserves identity and capacity in the comparison frame; and returns one of three states: SUPPORTED, REVIEW_REQUIRED or INSUFFICIENT_EVIDENCE. It is a teaching artefact, not a parser or legal decision engine.

The positive case contains a master agreement at thirty days, an explicit amendment to forty-five days and a later purchase order that repeats thirty. It also contains two “A. Shah” mentions with different stable identities and capacities. The pipeline marks the master claim superseded, preserves the later active conflict and returns review required. The negative control contains one verified invoice total and returns supported.

Run

python3 document_evidence_reconstruction.py

Assumptions: source admission already occurred; stable entity identifiers are supplied; supersession must be explicit; comparison is limited to one subject-capacity-predicate frame.

positive_case.decision.state = REVIEW_REQUIRED
positive_case.active_values = ["30", "45"]
negative_control.decision.state = SUPPORTED
negative_control.value = "1250.00"

The negative case is as important as the positive one. A mechanism that flags every file as complex has no placement discipline. Here the one-page invoice produces no relations and one unrebutted active proposition. The code therefore shows both when reconstruction changes the outcome and when it should remain thin.

Open the complete runnable Python programme

The programme is intentionally small enough to inspect. Replace the synthetic claims with outputs from a parser only after adding source admission, access controls and domain review.

#!/usr/bin/env python3
"""A small evidence-reconstruction pipeline using only the Python standard library.

The example keeps source spans, typed claims, explicit supersession, conflicts and
rule outcomes separate. It is deliberately not a legal interpretation engine.
"""
from __future__ import annotations

from dataclasses import asdict, dataclass, field
from datetime import date
from enum import Enum
import json
from typing import Iterable


class RelationType(str, Enum):
    CORROBORATES = "corroborates"
    CONTRADICTS = "contradicts"
    SUPERSEDES = "supersedes"


class DecisionState(str, Enum):
    SUPPORTED = "SUPPORTED"
    REVIEW_REQUIRED = "REVIEW_REQUIRED"
    INSUFFICIENT_EVIDENCE = "INSUFFICIENT_EVIDENCE"


@dataclass(frozen=True)
class SourceSpan:
    document_id: str
    version: str
    page: int
    quote: str
    sha256: str
    evidence_role: str = "extracted"


@dataclass(frozen=True)
class Claim:
    claim_id: str
    subject_id: str
    capacity: str
    predicate: str
    value: str
    valid_from: date
    source: SourceSpan
    confidence: float
    supersedes: str | None = None

    @property
    def frame(self) -> tuple[str, str, str]:
        """Claims can be compared only inside the same subject-capacity-predicate frame."""
        return (self.subject_id, self.capacity, self.predicate)


@dataclass(frozen=True)
class Relation:
    source_claim: str
    relation: RelationType
    target_claim: str
    reason: str


@dataclass
class EvidenceGraph:
    claims: dict[str, Claim] = field(default_factory=dict)
    relations: list[Relation] = field(default_factory=list)

    def add_claim(self, claim: Claim) -> None:
        if claim.claim_id in self.claims:
            raise ValueError(f"Duplicate claim id: {claim.claim_id}")
        self.claims[claim.claim_id] = claim

    def reconcile(self) -> None:
        self.relations.clear()
        ordered = sorted(self.claims.values(), key=lambda c: (c.frame, c.valid_from, c.claim_id))

        for claim in ordered:
            if claim.supersedes:
                prior = self.claims.get(claim.supersedes)
                if prior is None:
                    raise ValueError(f"{claim.claim_id} supersedes unknown claim {claim.supersedes}")
                if prior.frame != claim.frame:
                    raise ValueError("Supersession must stay inside one identity-capacity-predicate frame")
                self.relations.append(
                    Relation(claim.claim_id, RelationType.SUPERSEDES, prior.claim_id,
                             "Explicit amendment relation; date alone is not precedence")
                )

        for index, left in enumerate(ordered):
            for right in ordered[index + 1:]:
                if left.frame != right.frame:
                    continue
                if left.value == right.value:
                    self.relations.append(
                        Relation(left.claim_id, RelationType.CORROBORATES, right.claim_id,
                                 "Same typed proposition in the same comparison frame")
                    )
                    continue
                explicit_pair = (
                    left.supersedes == right.claim_id or right.supersedes == left.claim_id
                )
                if not explicit_pair:
                    self.relations.append(
                        Relation(left.claim_id, RelationType.CONTRADICTS, right.claim_id,
                                 "Incompatible values with no admitted supersession relation")
                    )

    def active_claims(self, as_of: date) -> list[Claim]:
        superseded_ids = {
            relation.target_claim
            for relation in self.relations
            if relation.relation is RelationType.SUPERSEDES
            and self.claims[relation.source_claim].valid_from <= as_of
        }
        return [
            claim for claim in self.claims.values()
            if claim.valid_from <= as_of and claim.claim_id not in superseded_ids
        ]

    def decide(self, *, required_frame: tuple[str, str, str], as_of: date) -> dict[str, object]:
        active = [claim for claim in self.active_claims(as_of) if claim.frame == required_frame]
        if not active:
            return {
                "state": DecisionState.INSUFFICIENT_EVIDENCE.value,
                "reason": "No active claim supports the required frame",
                "claims": [],
            }

        active_ids = {claim.claim_id for claim in active}
        conflicts = [
            relation for relation in self.relations
            if relation.relation is RelationType.CONTRADICTS
            and relation.source_claim in active_ids
            and relation.target_claim in active_ids
        ]
        values = sorted({claim.value for claim in active})
        if conflicts or len(values) > 1:
            return {
                "state": DecisionState.REVIEW_REQUIRED.value,
                "reason": "Active evidence contains incompatible propositions",
                "claims": sorted(active_ids),
                "values": values,
                "conflicts": [asdict(item) for item in conflicts],
            }

        return {
            "state": DecisionState.SUPPORTED.value,
            "reason": "One unrebutted active proposition supports the required frame",
            "claims": sorted(active_ids),
            "value": values[0],
        }

    def report(self, *, required_frame: tuple[str, str, str], as_of: date) -> dict[str, object]:
        self.reconcile()
        return {
            "as_of": as_of.isoformat(),
            "claims": [serialise_claim(c) for c in sorted(self.claims.values(), key=lambda c: c.claim_id)],
            "relations": [asdict(r) for r in self.relations],
            "decision": self.decide(required_frame=required_frame, as_of=as_of),
        }


def serialise_claim(claim: Claim) -> dict[str, object]:
    data = asdict(claim)
    data["valid_from"] = claim.valid_from.isoformat()
    return data


def build_positive_case() -> EvidenceGraph:
    """An amendment is explicit, but a later purchase order still asserts the old term."""
    graph = EvidenceGraph()
    graph.add_claim(Claim(
        "C1", "ORG-ACME", "supplier", "payment_days", "30", date(2025, 1, 1),
        SourceSpan("master-agreement", "v1", 4, "Payment is due within 30 days.", "sha256:a1"),
        0.99,
    ))
    graph.add_claim(Claim(
        "C2", "ORG-ACME", "supplier", "payment_days", "45", date(2025, 7, 1),
        SourceSpan("amendment-01", "signed", 1, "Clause 8 is replaced: payment is due within 45 days.", "sha256:b2", "verified"),
        0.98, supersedes="C1",
    ))
    graph.add_claim(Claim(
        "C3", "ORG-ACME", "supplier", "payment_days", "30", date(2025, 8, 12),
        SourceSpan("purchase-order-884", "v1", 2, "Terms: Net 30.", "sha256:c3"),
        0.97,
    ))
    # Same surface name, different stable identities and capacities. No automatic merge.
    graph.add_claim(Claim(
        "C4", "PERSON-ASHA-SHAH-001", "director", "signed_document", "amendment-01", date(2025, 6, 28),
        SourceSpan("amendment-01", "signed", 2, "A. Shah, Director", "sha256:b2", "observed"),
        0.86,
    ))
    graph.add_claim(Claim(
        "C5", "PERSON-ARUN-SHAH-017", "witness", "signed_document", "amendment-01", date(2025, 6, 28),
        SourceSpan("amendment-01", "signed", 2, "A. Shah, Witness", "sha256:b2", "observed"),
        0.82,
    ))
    return graph


def build_negative_control() -> EvidenceGraph:
    """A clean one-page invoice where direct extraction is adequate."""
    graph = EvidenceGraph()
    graph.add_claim(Claim(
        "N1", "INVOICE-1042", "document", "gross_total_gbp", "1250.00", date(2025, 8, 12),
        SourceSpan("invoice-1042", "original", 1, "Gross total GBP 1,250.00", "sha256:n1", "verified"),
        0.995,
    ))
    return graph


def main() -> None:
    cases: Iterable[tuple[str, EvidenceGraph, tuple[str, str, str], date]] = (
        ("positive_case", build_positive_case(), ("ORG-ACME", "supplier", "payment_days"), date(2025, 8, 20)),
        ("negative_control", build_negative_control(), ("INVOICE-1042", "document", "gross_total_gbp"), date(2025, 8, 20)),
    )
    output = {
        name: graph.report(required_frame=frame, as_of=as_of)
        for name, graph, frame, as_of in cases
    }
    print(json.dumps(output, indent=2, default=str))


if __name__ == "__main__":
    main()
Open the complete synthetic JSON result
{
  "positive_case": {
    "as_of": "2025-08-20",
    "claims": [
      {
        "claim_id": "C1",
        "subject_id": "ORG-ACME",
        "capacity": "supplier",
        "predicate": "payment_days",
        "value": "30",
        "valid_from": "2025-01-01",
        "source": {
          "document_id": "master-agreement",
          "version": "v1",
          "page": 4,
          "quote": "Payment is due within 30 days.",
          "sha256": "sha256:a1",
          "evidence_role": "extracted"
        },
        "confidence": 0.99,
        "supersedes": null
      },
      {
        "claim_id": "C2",
        "subject_id": "ORG-ACME",
        "capacity": "supplier",
        "predicate": "payment_days",
        "value": "45",
        "valid_from": "2025-07-01",
        "source": {
          "document_id": "amendment-01",
          "version": "signed",
          "page": 1,
          "quote": "Clause 8 is replaced: payment is due within 45 days.",
          "sha256": "sha256:b2",
          "evidence_role": "verified"
        },
        "confidence": 0.98,
        "supersedes": "C1"
      },
      {
        "claim_id": "C3",
        "subject_id": "ORG-ACME",
        "capacity": "supplier",
        "predicate": "payment_days",
        "value": "30",
        "valid_from": "2025-08-12",
        "source": {
          "document_id": "purchase-order-884",
          "version": "v1",
          "page": 2,
          "quote": "Terms: Net 30.",
          "sha256": "sha256:c3",
          "evidence_role": "extracted"
        },
        "confidence": 0.97,
        "supersedes": null
      },
      {
        "claim_id": "C4",
        "subject_id": "PERSON-ASHA-SHAH-001",
        "capacity": "director",
        "predicate": "signed_document",
        "value": "amendment-01",
        "valid_from": "2025-06-28",
        "source": {
          "document_id": "amendment-01",
          "version": "signed",
          "page": 2,
          "quote": "A. Shah, Director",
          "sha256": "sha256:b2",
          "evidence_role": "observed"
        },
        "confidence": 0.86,
        "supersedes": null
      },
      {
        "claim_id": "C5",
        "subject_id": "PERSON-ARUN-SHAH-017",
        "capacity": "witness",
        "predicate": "signed_document",
        "value": "amendment-01",
        "valid_from": "2025-06-28",
        "source": {
          "document_id": "amendment-01",
          "version": "signed",
          "page": 2,
          "quote": "A. Shah, Witness",
          "sha256": "sha256:b2",
          "evidence_role": "observed"
        },
        "confidence": 0.82,
        "supersedes": null
      }
    ],
    "relations": [
      {
        "source_claim": "C2",
        "relation": "supersedes",
        "target_claim": "C1",
        "reason": "Explicit amendment relation; date alone is not precedence"
      },
      {
        "source_claim": "C1",
        "relation": "corroborates",
        "target_claim": "C3",
        "reason": "Same typed proposition in the same comparison frame"
      },
      {
        "source_claim": "C2",
        "relation": "contradicts",
        "target_claim": "C3",
        "reason": "Incompatible values with no admitted supersession relation"
      }
    ],
    "decision": {
      "state": "REVIEW_REQUIRED",
      "reason": "Active evidence contains incompatible propositions",
      "claims": [
        "C2",
        "C3"
      ],
      "values": [
        "30",
        "45"
      ],
      "conflicts": [
        {
          "source_claim": "C2",
          "relation": "contradicts",
          "target_claim": "C3",
          "reason": "Incompatible values with no admitted supersession relation"
        }
      ]
    }
  },
  "negative_control": {
    "as_of": "2025-08-20",
    "claims": [
      {
        "claim_id": "N1",
        "subject_id": "INVOICE-1042",
        "capacity": "document",
        "predicate": "gross_total_gbp",
        "value": "1250.00",
        "valid_from": "2025-08-12",
        "source": {
          "document_id": "invoice-1042",
          "version": "original",
          "page": 1,
          "quote": "Gross total GBP 1,250.00",
          "sha256": "sha256:n1",
          "evidence_role": "verified"
        },
        "confidence": 0.995,
        "supersedes": null
      }
    ],
    "relations": [],
    "decision": {
      "state": "SUPPORTED",
      "reason": "One unrebutted active proposition supports the required frame",
      "claims": [
        "N1"
      ],
      "value": "1250.00"
    }
  }
}

Compact glossary

Evidence object
A source or derived object with identity, version, provenance, quality and access state.
Proposition
A typed claim that preserves subject, capacity, predicate, value, scope, time and source span.
Evidence role
The status of a statement as observed, extracted, verified, asserted, inferred or accepted.
Valid time
The period during which a proposition applies in the represented world.
Recorded time
When the system learned or stored the proposition.
Supersession
An explicit relation stating that one proposition replaces another for a defined scope and period.
Decision receipt
A record binding purpose, evidence, rules, identity, authority and the accepted decision.
Effect receipt
Evidence that an authorised downstream action produced the intended state, or remains unknown.

Build the case state first

Document intelligence should make a difficult decision smaller without making its uncertainty disappear. The durable architecture begins with admitted source objects, preserves every transformation, turns spans into typed propositions, keeps identity separate from capacity, models time and contradiction explicitly, and records rules and reviewer decisions as different objects. A narrative comes last because it is one view of the case, not the case itself.

This changes the engineering decision. Do not ask first which model can summarise the bundle. Ask which evidence state must survive model replacement, correction, audit and dispute. Use direct extraction for clean, low-consequence forms. Introduce reconstruction when versioning, identity, conditions or conflict determine the answer. Keep human authority where interpretation remains consequential or legitimately contested.

The release gate is simple to state and hard to satisfy: can the system preserve the exact disagreement that should change the decision? If a one-word mutation, role swap or missing schedule does not move the structured state before it moves the prose, the system is reading documents without reconstructing evidence. Its summaries may be fluent. Its decisions are not yet defensible.

The evidence conversationChart · sequence
sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: The amendment that a good summary swallowed
  M->>E: The one-word amendment thought experiment
  E-->>M: compare rival explanations
  E->>D: Sources
  D-->>Q: revise scope or act
Question, mechanism, evidence and decision exchange the burdens developed in Document Intelligence as Evidence Reconstruction. Reading use: follow the relation, then test it against the surrounding evidence and limits.

Sources

  1. Mathew, M., Karatzas, D. and Jawahar, C. V. “DocVQA: A Dataset for VQA on Document Images.” WACV 2021. Primary paper.
  2. Huang, Y. et al. “LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking.” 2022. Primary paper and model description.
  3. Ma, Y. et al. “MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizations.” NeurIPS Datasets and Benchmarks 2024. Primary benchmark paper.
  4. Chia, Y. K. et al. “M-LongDoc: A Benchmark for Multimodal Super-Long Document Understanding and a Retrieval-Aware Tuning Framework.” EMNLP 2025. Primary benchmark paper.
  5. Zhang, J. et al. “OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation.” ICCV 2025. Primary benchmark paper.
  6. Koreeda, Y. and Manning, C. D. “ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts.” Findings of EMNLP 2021. Primary paper and dataset.
  7. Mathur, P. et al. “DocInfer: Document-level Natural Language Inference using Optimal Evidence Selection.” EMNLP 2022. Primary paper.
  8. Li, Y. et al. “Deep Entity Matching with Pre-Trained Language Models.” PVLDB 2020. Primary paper.
  9. “DocScope: Benchmarking Verifiable Reasoning for Trustworthy Long-Document Understanding.” 2026. Recent primary preprint.
  10. Zhang, M. “MMLongBench-Doc-V2: A Corrected-Annotation, Semantics-Aware Revision of MMLongBench-Doc.” 2026. Negative and correction result.
  11. W3C. “PROV-O: The PROV Ontology.” Recommendation. Official specification.
  12. W3C. “PROV-DM: The PROV Data Model.” Recommendation. Official specification.
  13. W3C. “Shapes Constraint Language (SHACL).” Recommendation. Official specification.
  14. Object Management Group. “Decision Model and Notation.” Official specification family.
  15. OASIS. “LegalRuleML Core Specification Version 1.0.” Official standard.
  16. NIST. “Artificial Intelligence Risk Management Framework 1.0.” Official framework.
  17. Greshake, K. et al. “Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection.” 2023. Primary security paper.