Document Intelligence as Evidence Reconstruction
Reliable document systems do not answer by compressing a file pile. They rebuild a typed, versioned evidence state in which every claim, identity, conflict, rule and decision remains traceable to its source.
TLDR
- Document intelligence is reliable only when extraction and generation remain reversible views over a typed evidence state.
- The mechanism preserves source regions, propositions, identities and capacities, temporal relations, conflicts, rule applications and reviewer decisions as separate objects.
- Use reconstruction when an answer depends on several documents, changing versions, disputed identity, conditions or consequential downstream action.
- A clean one-page form is the negative control: direct extraction may be better, while an explicit graph can add cost or encode the wrong ontology.
- Release on conflict recall, false-merge rate, temporal mutation and claim support, rather than page throughput or summary fluency.
Opening case
The amendment that a good summary swallowed
A supplier file contains a forty-two-page master agreement, a signed one-page amendment and a later purchase order. The agreement says invoices are payable within thirty days. The amendment replaces that clause with forty-five days. The purchase order, generated from an old template six weeks later, still says “Net 30”. A fluent summary reports that the parties agreed payment terms and that the latest order lists thirty days.
The summary is readable, faithful to visible words and operationally useless. The decision-maker needs to know which proposition applies to the invoice now, whether the amendment was validly executed, whether it covered this order, and why the later purchase order repeated the earlier term. The disagreement is not noise to be averaged away. The disagreement is the material fact.
The same signature page contains “A. Shah” twice. One appearance is a director; the other is a witness. A conventional entity matcher may decide that the names refer to one person. Even if that identity guess happens to be correct, the capacities remain different. A director’s capacity cannot be inferred from a witness signature, and a shared name does not carry authority across roles.
This small file exposes the central burden of document intelligence. Perception must recover words, layout and regions. Extraction must propose fields and clauses. Identity resolution must connect mentions without collapsing capacities. Temporal reasoning must distinguish execution, effect and knowledge. Reconciliation must preserve support, supersession and contradiction. Rules must run against admitted evidence, while judgement remains with the authorised reviewer. The answer should be generated only after that state exists.
Figure 1. An evidence mosaic becomes an argument graph
Long-context vision-language models have improved the ability to locate text, tables and figures across PDFs. Yet a benchmark such as MMLongBench-Doc deliberately includes cross-page and unanswerable questions because evidence integration remains difficult even after perception succeeds.[3] M-LongDoc extends the challenge to documents with hundreds of pages and open-ended explanations.[4] These tasks show why longer context is useful. They do not establish that a generated answer preserves document identity, temporal effect or accountable judgement.
Evidence reconstruction is the missing systems layer. It converts a collection of mutable, multimodal files into a minimal case state that can be inspected, challenged, corrected and replayed. Summaries remain valuable, but only as disposable views over that state.
Part I
Evidence before prose
A document is more than its extracted text. Meaning may depend on page order, a table boundary, a struck-out phrase, a stamp, a footnote marker, a signature region or a chart legend. DocVQA was created because answering questions about document images requires visual and textual evidence together.[1] LayoutLMv3 similarly learns from text, image patches and their alignment rather than treating layout as decoration.[2]
The first architectural mistake is therefore to call OCR output “the document”. OCR is a transformation. It can be useful, cheap and searchable, but its errors become upstream causes for every later stage. OHRBench was designed to measure this cascade through retrieval and generation, demonstrating that OCR quality cannot be judged only by character accuracy.[5] A missing minus sign may change a figure. A broken row may move a value to the wrong entity. A lost negation may reverse a rule.
The one-word amendment thought experiment
Take two otherwise identical amendments. The first says, “The borrower may dispose of the asset with written consent.” The second inserts one word: “The borrower may not dispose of the asset with written consent.” Hold every other feature constant. If OCR loses the word “not”, both documents produce the same text stream. If a summariser compresses either into “asset disposal is subject to consent”, both produce the same narrative. The system has erased the causal feature that should change the decision.
A reliable pipeline retains the page image, the exact region, the OCR hypothesis, the parser version and the proposition that carries negation explicitly. The extracted proposition can be wrong, but it remains reversible. A reviewer can open the region that produced it and correct the earliest faulty object. Reversibility is more valuable than a polished explanation built on an invisible transformation.
Figure 2. Summarisation flattens the relation that reconstruction must preserve
The source manifest is the first evidence object
For every admitted file, record the native bytes, content hash, repository identity, version, access decision, page count, render output, text layer, OCR engine and parser build. Each page region receives stable coordinates. Derived text points back to the page and the exact transformation that produced it. W3C PROV provides a domain-neutral vocabulary for entities, activities and agents involved in producing a derived object.[11] A document system can specialise that model without pretending provenance alone determines evidential weight.
Evidence role must also stay explicit. “A mark is visible in the signature block” is an observation. “OCR reads A. Shah” is an extraction. “The identifier matches an approved record” is a verification. “The sender states that approval occurred” is an assertion. “The amendment probably narrows clause 8” is an inference. “The reviewer accepts this treatment” is an authorised decision. Combining these into one confidence score hides different kinds of uncertainty and different owners.
Figure 3. The transformation ledger keeps every interpretive step openable
Part II
The reconstruction pipeline
Reconstruction begins with an investigation question, not a generic request to “analyse these documents”. The case envelope names the subjects, document classes, time frame, jurisdictions or policy domains, reviewer role and downstream exclusions. These fields limit what the system is allowed to admit and which questions it may answer. A discovered document outside that envelope becomes a proposed scope change rather than silent context expansion.
The pipeline then performs seven separable operations: admit and fingerprint sources; render and segment pages; extract candidate spans; normalise spans into typed propositions; resolve entities and capacities; reconcile propositions across time and scope; and apply deterministic validation or decision rules before generating a review view. Models can assist several operations. No model output becomes authoritative merely because it appears later in the pipeline.
A proposition is smaller than a summary and richer than a field
A useful proposition contains at least a subject, capacity, predicate, value, qualifiers, negation, modality, source span, valid time, evidence role, confidence and review state. For example, “ORG-ACME, acting as supplier, has payment_days 45 from 1 July, according to page 1 of signed amendment 01, verified, superseding claim C1.” That object is verbose compared with a field-value pair. Its extra structure is exactly what permits comparison and correction.
The multiplication sign indicates that each dimension constrains the comparison frame. Two claims about the same subject and predicate may still be incomparable if they concern different capacities, assets, jurisdictions or periods. Removing a dimension increases apparent agreement by merging claims that should remain distinct.
| Claim | Typed proposition | Source state | Relation |
|---|---|---|---|
| C1 | ORG-ACME · supplier · payment_days = 30 · from 1 Jan | Master agreement, page 4, extracted | Superseded by C2 |
| C2 | ORG-ACME · supplier · payment_days = 45 · from 1 Jul | Signed amendment, page 1, verified | Explicitly supersedes C1 |
| C3 | ORG-ACME · supplier · payment_days = 30 · from 12 Aug | Purchase order, page 2, extracted | Conflicts with active C2 |
A summary-first system may choose C3 because it is most recent. A date-first rule may choose C3 for the same reason. Reconstruction refuses that shortcut. The amendment explicitly states what it replaces; the purchase order does not. Recency is evidence about sequence, not authority. The output is therefore a review question: does the order validly vary the agreement, or is it an operational error?
Document-level natural language inference provides useful machinery for identifying entailment, contradiction and evidence spans. ContractNLI asks whether hypotheses are entailed, contradicted or not mentioned in a contract and links decisions to supporting spans.[6] DocInfer builds a hierarchical document graph and selects evidence for document-level inference.[7] These are important components. A production investigation must add source admission, identity, temporal effect, rule ownership, reviewer authority and correction semantics.
Why the evidence graph is not just a knowledge graph
Enterprise knowledge graphs often aim to state durable facts about entities. An evidence graph carries competing claims, their provenance and their review status. It is comfortable with “unknown”, “conflicting” and “not searched”. It may contain two incompatible values for the same frame because suppressing one would destroy evidence. Its edges describe how propositions relate, while a separate decision object records which treatment an authorised reviewer accepted.
This distinction prevents a common failure: promoting a model inference into enterprise knowledge before the case is settled. A candidate identity match can exist as a proposal with a score and compared attributes. It should not silently merge records, transfer permissions or rewrite all dependent propositions. Promotion is a governed transition with an owner, rationale and reversible change set.
Recent work is beginning to measure this gap directly. DocScope frames long-document understanding around verifiable reasoning trajectories and precise evidence grounding, rather than answer text alone.[9] The broader hypothesis is that evidence reconstruction will become a reusable intermediate representation for such systems. That hypothesis remains open: a future end-to-end model may preserve equivalent structure internally and expose it reliably. Until that is demonstrated under mutation and review, explicit state is easier to test.
Part III
flowchart LR A["The amendment that a good<br/>summary swallowed"] --> B["Evidence before prose"] B --> C["The reconstruction pipeline"] C --> D["Identity, time and conflict<br/>are separate problems"] D -. "re-test" .-> A classDef first fill:#0a1424,stroke:#087A98,stroke-width:2px,color:#e6f3ff classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff class A first class B,C,D step
Identity, time and conflict are separate problems
Entity resolution asks whether two records refer to the same real-world entity. Systems such as Ditto cast matching as a learned comparison between record pairs and show how language models can improve matching quality.[8] That capability is useful, but a consequential document investigation needs two extra disciplines. Match uncertainty must survive downstream, and identity must remain distinct from capacity.
The same-name role-switch thought experiment
Imagine two bundles that differ in one feature. In bundle A, “A. Shah” signs an amendment as director and a second “A. Shah” signs as witness. In bundle B, a verified identifier proves both signatures belong to the same person. The identity answer changes. The authority answer may not. Even in bundle B, the witness signature does not become a director action. A system that stores only a person node and a generic “signed” edge cannot represent the distinction.
The minimum structure is mention → candidate entity → resolved entity, followed by a separate entity → capacity → instrument relation. Capacity can include director, trustee, attorney, beneficiary, customer, supplier or reviewer. It has its own evidence and valid interval. A correct person match can still support an incorrect authority claim when role evidence is collapsed.
Figure 4. Identity confidence does not carry capacity authority
Time has several clocks
Document collections carry execution date, stated effective date, filing date, receipt date, knowledge date and source-system version time. These clocks answer different questions. A later-discovered amendment may have changed the legal or contractual position at an earlier transaction date. The institution may nevertheless have acted on the documents it possessed then. Reconstruction should show both the historic effect and the historic knowledge state without treating either as the other.
Every proposition therefore needs a valid-time interval and a recorded-time stamp. Supersession should be an explicit relation with scope: amendment B replaces clause 8 of instrument A from 1 July for invoices under schedule X. A later document does not automatically supersede an earlier one. It may be a copy, an assertion, an ineffective draft or a statement about another scope.
Contradiction is a relation, not a bad score
Claims can be equivalent, corroborating, superseding, narrowing, conflicting, missing or incomparable. The classification depends on identity, capacity, predicate, scope and time. Two different values are not contradictory when they apply to different periods. Two identical sentences may conflict when their subjects only appear similar. “No evidence found” is different from “repository not searched”, “access denied” and “referenced schedule missing”.
Figure 5. A contradiction surface separates semantic incompatibility from temporal overlap
Part V
Evaluate the system by material failure
Field accuracy is necessary and insufficient. A pipeline can read every amount correctly and still merge two people, apply an amendment to the wrong period, hide a contradiction or cite a superseded page. Evaluation must follow the evidence chain from source admission to observed outcome. NIST AI RMF provides a general risk-management frame for mapping, measuring and managing system risks, while the concrete tests remain use-case specific.[16]
A release corpus should contain clean forms and difficult combinations: missing pages, duplicate files with subtle changes, handwriting, low-resolution scans, multi-column tables, inconsistent identifiers, same names in different capacities, amendments without originals, ineffective drafts, contradictory dates, conditional clauses, missing schedules and malicious embedded instructions. Gold state should include evidence spans, accepted propositions, legitimate alternatives and required review questions. Do not force one answer where experts reasonably disagree.
| Layer | Measure | Material failure |
|---|---|---|
| Source integrity | File, version, page and hash accuracy | Amended or rescanned page omitted |
| Perception | Region, text, table and reading-order accuracy by quality cohort | Negation or row association lost |
| Proposition | Predicate, qualifier, negation, condition and source support | Clause normalised into a broader claim |
| Identity | False merge, false split and uncertainty calibration | Same name transfers capacity |
| Temporal relation | Effective interval and supersession mutation tests | Later document treated as automatically controlling |
| Reconciliation | Conflict recall, false conflict and missing-evidence classification | Narrative chooses one unresolved claim |
| Rules | Input lineage, version and boundary tests | Rule runs on unadmitted or stale evidence |
| Review and outcome | Correction trace, replay, destination readback and reopen rate | Pack corrected but dependent action remains wrong |
Failure injection is the primary test
Mutation reveals whether the architecture uses the structure it claims to use. Delete a negation and expect the proposition or decision to change. Swap two same-name identities and expect a protected merge to fail. Move an effective date and expect the active claim set to change. Remove a referenced schedule and expect insufficient evidence, not a guessed default. Insert a hidden instruction and expect no tool or scope change. Simulate a destination timeout and expect an unknown outcome followed by readback, not a blind retry.
Figure 8. A failure-injection matrix tests causal use of the evidence state
The negative control: one clean invoice
A single verified invoice contains one gross total, one currency and no amendment chain. Direct extraction with a schema validator is the stronger baseline. Building a proposition graph, identity model and reviewer workflow adds latency and objects without changing the decision. This case matters because it prevents evidence reconstruction from becoming architecture theatre.
The mechanism earns its cost when the case crosses a threshold: multiple authoritative sources, material versioning, identity ambiguity, conditional rules, contradiction, privileged access or downstream consequence. The threshold should be measured against the simplest baseline. If an end-to-end model can preserve source support, conflict recall, identity separation and temporal behaviour under controlled mutations at lower total cost, the explicit graph should shrink or disappear.
Benchmark labels also need provenance
Evaluation data is another document collection. In August 2026, MMLongBench-Doc-V2 reported brittle answer normalisation and a concentration of wrong, ambiguous or incomplete annotations in the original benchmark.[10] The lesson is broader than one dataset. Preserve benchmark version, annotation history, adjudication and metric code beside every score. A model can appear to regress when the reference is wrong, or appear to improve by exploiting a metric that ignores the evidential question.
Production monitoring should stratify by document type, scan quality, language, amendment depth, entity ambiguity and case consequence. Track reviewer corrections at the layer where they occur, conflicts found after sign-off, reopened cases, stale-pack exposure, access denials, source latency and unknown outcomes. Pages processed and summaries accepted are activity measures. Better value measures include first-time-complete evidence packs, earlier conflict discovery, reduced duplicate review and fewer downstream defects.
Executable artefact
stateDiagram-v2 [*] --> Claim state "The one-word amendment thought<br/>experiment" as Claim state "The source manifest is the<br/>first evidence object" as Test state "A proposition is smaller than<br/>a summary and richer than a<br/>field" as Revision Claim --> Test: expose mechanism Test --> Decision: evidence survives Test --> Revision: rival survives Revision --> Claim: narrow or rebuild Decision --> [*]
A minimal investigation pipeline
The accompanying Python programme uses only the standard library. It represents source spans, typed claims and relations; requires explicit supersession; preserves identity and capacity in the comparison frame; and returns one of three states: SUPPORTED, REVIEW_REQUIRED or INSUFFICIENT_EVIDENCE. It is a teaching artefact, not a parser or legal decision engine.
The positive case contains a master agreement at thirty days, an explicit amendment to forty-five days and a later purchase order that repeats thirty. It also contains two “A. Shah” mentions with different stable identities and capacities. The pipeline marks the master claim superseded, preserves the later active conflict and returns review required. The negative control contains one verified invoice total and returns supported.
python3 document_evidence_reconstruction.py
Assumptions: source admission already occurred; stable entity identifiers are supplied; supersession must be explicit; comparison is limited to one subject-capacity-predicate frame.
positive_case.decision.state = REVIEW_REQUIRED
positive_case.active_values = ["30", "45"]
negative_control.decision.state = SUPPORTED
negative_control.value = "1250.00"The negative case is as important as the positive one. A mechanism that flags every file as complex has no placement discipline. Here the one-page invoice produces no relations and one unrebutted active proposition. The code therefore shows both when reconstruction changes the outcome and when it should remain thin.
Open the complete runnable Python programme
The programme is intentionally small enough to inspect. Replace the synthetic claims with outputs from a parser only after adding source admission, access controls and domain review.
#!/usr/bin/env python3
"""A small evidence-reconstruction pipeline using only the Python standard library.
The example keeps source spans, typed claims, explicit supersession, conflicts and
rule outcomes separate. It is deliberately not a legal interpretation engine.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass, field
from datetime import date
from enum import Enum
import json
from typing import Iterable
class RelationType(str, Enum):
CORROBORATES = "corroborates"
CONTRADICTS = "contradicts"
SUPERSEDES = "supersedes"
class DecisionState(str, Enum):
SUPPORTED = "SUPPORTED"
REVIEW_REQUIRED = "REVIEW_REQUIRED"
INSUFFICIENT_EVIDENCE = "INSUFFICIENT_EVIDENCE"
@dataclass(frozen=True)
class SourceSpan:
document_id: str
version: str
page: int
quote: str
sha256: str
evidence_role: str = "extracted"
@dataclass(frozen=True)
class Claim:
claim_id: str
subject_id: str
capacity: str
predicate: str
value: str
valid_from: date
source: SourceSpan
confidence: float
supersedes: str | None = None
@property
def frame(self) -> tuple[str, str, str]:
"""Claims can be compared only inside the same subject-capacity-predicate frame."""
return (self.subject_id, self.capacity, self.predicate)
@dataclass(frozen=True)
class Relation:
source_claim: str
relation: RelationType
target_claim: str
reason: str
@dataclass
class EvidenceGraph:
claims: dict[str, Claim] = field(default_factory=dict)
relations: list[Relation] = field(default_factory=list)
def add_claim(self, claim: Claim) -> None:
if claim.claim_id in self.claims:
raise ValueError(f"Duplicate claim id: {claim.claim_id}")
self.claims[claim.claim_id] = claim
def reconcile(self) -> None:
self.relations.clear()
ordered = sorted(self.claims.values(), key=lambda c: (c.frame, c.valid_from, c.claim_id))
for claim in ordered:
if claim.supersedes:
prior = self.claims.get(claim.supersedes)
if prior is None:
raise ValueError(f"{claim.claim_id} supersedes unknown claim {claim.supersedes}")
if prior.frame != claim.frame:
raise ValueError("Supersession must stay inside one identity-capacity-predicate frame")
self.relations.append(
Relation(claim.claim_id, RelationType.SUPERSEDES, prior.claim_id,
"Explicit amendment relation; date alone is not precedence")
)
for index, left in enumerate(ordered):
for right in ordered[index + 1:]:
if left.frame != right.frame:
continue
if left.value == right.value:
self.relations.append(
Relation(left.claim_id, RelationType.CORROBORATES, right.claim_id,
"Same typed proposition in the same comparison frame")
)
continue
explicit_pair = (
left.supersedes == right.claim_id or right.supersedes == left.claim_id
)
if not explicit_pair:
self.relations.append(
Relation(left.claim_id, RelationType.CONTRADICTS, right.claim_id,
"Incompatible values with no admitted supersession relation")
)
def active_claims(self, as_of: date) -> list[Claim]:
superseded_ids = {
relation.target_claim
for relation in self.relations
if relation.relation is RelationType.SUPERSEDES
and self.claims[relation.source_claim].valid_from <= as_of
}
return [
claim for claim in self.claims.values()
if claim.valid_from <= as_of and claim.claim_id not in superseded_ids
]
def decide(self, *, required_frame: tuple[str, str, str], as_of: date) -> dict[str, object]:
active = [claim for claim in self.active_claims(as_of) if claim.frame == required_frame]
if not active:
return {
"state": DecisionState.INSUFFICIENT_EVIDENCE.value,
"reason": "No active claim supports the required frame",
"claims": [],
}
active_ids = {claim.claim_id for claim in active}
conflicts = [
relation for relation in self.relations
if relation.relation is RelationType.CONTRADICTS
and relation.source_claim in active_ids
and relation.target_claim in active_ids
]
values = sorted({claim.value for claim in active})
if conflicts or len(values) > 1:
return {
"state": DecisionState.REVIEW_REQUIRED.value,
"reason": "Active evidence contains incompatible propositions",
"claims": sorted(active_ids),
"values": values,
"conflicts": [asdict(item) for item in conflicts],
}
return {
"state": DecisionState.SUPPORTED.value,
"reason": "One unrebutted active proposition supports the required frame",
"claims": sorted(active_ids),
"value": values[0],
}
def report(self, *, required_frame: tuple[str, str, str], as_of: date) -> dict[str, object]:
self.reconcile()
return {
"as_of": as_of.isoformat(),
"claims": [serialise_claim(c) for c in sorted(self.claims.values(), key=lambda c: c.claim_id)],
"relations": [asdict(r) for r in self.relations],
"decision": self.decide(required_frame=required_frame, as_of=as_of),
}
def serialise_claim(claim: Claim) -> dict[str, object]:
data = asdict(claim)
data["valid_from"] = claim.valid_from.isoformat()
return data
def build_positive_case() -> EvidenceGraph:
"""An amendment is explicit, but a later purchase order still asserts the old term."""
graph = EvidenceGraph()
graph.add_claim(Claim(
"C1", "ORG-ACME", "supplier", "payment_days", "30", date(2025, 1, 1),
SourceSpan("master-agreement", "v1", 4, "Payment is due within 30 days.", "sha256:a1"),
0.99,
))
graph.add_claim(Claim(
"C2", "ORG-ACME", "supplier", "payment_days", "45", date(2025, 7, 1),
SourceSpan("amendment-01", "signed", 1, "Clause 8 is replaced: payment is due within 45 days.", "sha256:b2", "verified"),
0.98, supersedes="C1",
))
graph.add_claim(Claim(
"C3", "ORG-ACME", "supplier", "payment_days", "30", date(2025, 8, 12),
SourceSpan("purchase-order-884", "v1", 2, "Terms: Net 30.", "sha256:c3"),
0.97,
))
# Same surface name, different stable identities and capacities. No automatic merge.
graph.add_claim(Claim(
"C4", "PERSON-ASHA-SHAH-001", "director", "signed_document", "amendment-01", date(2025, 6, 28),
SourceSpan("amendment-01", "signed", 2, "A. Shah, Director", "sha256:b2", "observed"),
0.86,
))
graph.add_claim(Claim(
"C5", "PERSON-ARUN-SHAH-017", "witness", "signed_document", "amendment-01", date(2025, 6, 28),
SourceSpan("amendment-01", "signed", 2, "A. Shah, Witness", "sha256:b2", "observed"),
0.82,
))
return graph
def build_negative_control() -> EvidenceGraph:
"""A clean one-page invoice where direct extraction is adequate."""
graph = EvidenceGraph()
graph.add_claim(Claim(
"N1", "INVOICE-1042", "document", "gross_total_gbp", "1250.00", date(2025, 8, 12),
SourceSpan("invoice-1042", "original", 1, "Gross total GBP 1,250.00", "sha256:n1", "verified"),
0.995,
))
return graph
def main() -> None:
cases: Iterable[tuple[str, EvidenceGraph, tuple[str, str, str], date]] = (
("positive_case", build_positive_case(), ("ORG-ACME", "supplier", "payment_days"), date(2025, 8, 20)),
("negative_control", build_negative_control(), ("INVOICE-1042", "document", "gross_total_gbp"), date(2025, 8, 20)),
)
output = {
name: graph.report(required_frame=frame, as_of=as_of)
for name, graph, frame, as_of in cases
}
print(json.dumps(output, indent=2, default=str))
if __name__ == "__main__":
main()
Open the complete synthetic JSON result
{
"positive_case": {
"as_of": "2025-08-20",
"claims": [
{
"claim_id": "C1",
"subject_id": "ORG-ACME",
"capacity": "supplier",
"predicate": "payment_days",
"value": "30",
"valid_from": "2025-01-01",
"source": {
"document_id": "master-agreement",
"version": "v1",
"page": 4,
"quote": "Payment is due within 30 days.",
"sha256": "sha256:a1",
"evidence_role": "extracted"
},
"confidence": 0.99,
"supersedes": null
},
{
"claim_id": "C2",
"subject_id": "ORG-ACME",
"capacity": "supplier",
"predicate": "payment_days",
"value": "45",
"valid_from": "2025-07-01",
"source": {
"document_id": "amendment-01",
"version": "signed",
"page": 1,
"quote": "Clause 8 is replaced: payment is due within 45 days.",
"sha256": "sha256:b2",
"evidence_role": "verified"
},
"confidence": 0.98,
"supersedes": "C1"
},
{
"claim_id": "C3",
"subject_id": "ORG-ACME",
"capacity": "supplier",
"predicate": "payment_days",
"value": "30",
"valid_from": "2025-08-12",
"source": {
"document_id": "purchase-order-884",
"version": "v1",
"page": 2,
"quote": "Terms: Net 30.",
"sha256": "sha256:c3",
"evidence_role": "extracted"
},
"confidence": 0.97,
"supersedes": null
},
{
"claim_id": "C4",
"subject_id": "PERSON-ASHA-SHAH-001",
"capacity": "director",
"predicate": "signed_document",
"value": "amendment-01",
"valid_from": "2025-06-28",
"source": {
"document_id": "amendment-01",
"version": "signed",
"page": 2,
"quote": "A. Shah, Director",
"sha256": "sha256:b2",
"evidence_role": "observed"
},
"confidence": 0.86,
"supersedes": null
},
{
"claim_id": "C5",
"subject_id": "PERSON-ARUN-SHAH-017",
"capacity": "witness",
"predicate": "signed_document",
"value": "amendment-01",
"valid_from": "2025-06-28",
"source": {
"document_id": "amendment-01",
"version": "signed",
"page": 2,
"quote": "A. Shah, Witness",
"sha256": "sha256:b2",
"evidence_role": "observed"
},
"confidence": 0.82,
"supersedes": null
}
],
"relations": [
{
"source_claim": "C2",
"relation": "supersedes",
"target_claim": "C1",
"reason": "Explicit amendment relation; date alone is not precedence"
},
{
"source_claim": "C1",
"relation": "corroborates",
"target_claim": "C3",
"reason": "Same typed proposition in the same comparison frame"
},
{
"source_claim": "C2",
"relation": "contradicts",
"target_claim": "C3",
"reason": "Incompatible values with no admitted supersession relation"
}
],
"decision": {
"state": "REVIEW_REQUIRED",
"reason": "Active evidence contains incompatible propositions",
"claims": [
"C2",
"C3"
],
"values": [
"30",
"45"
],
"conflicts": [
{
"source_claim": "C2",
"relation": "contradicts",
"target_claim": "C3",
"reason": "Incompatible values with no admitted supersession relation"
}
]
}
},
"negative_control": {
"as_of": "2025-08-20",
"claims": [
{
"claim_id": "N1",
"subject_id": "INVOICE-1042",
"capacity": "document",
"predicate": "gross_total_gbp",
"value": "1250.00",
"valid_from": "2025-08-12",
"source": {
"document_id": "invoice-1042",
"version": "original",
"page": 1,
"quote": "Gross total GBP 1,250.00",
"sha256": "sha256:n1",
"evidence_role": "verified"
},
"confidence": 0.995,
"supersedes": null
}
],
"relations": [],
"decision": {
"state": "SUPPORTED",
"reason": "One unrebutted active proposition supports the required frame",
"claims": [
"N1"
],
"value": "1250.00"
}
}
}
Compact glossary
- Evidence object
- A source or derived object with identity, version, provenance, quality and access state.
- Proposition
- A typed claim that preserves subject, capacity, predicate, value, scope, time and source span.
- Evidence role
- The status of a statement as observed, extracted, verified, asserted, inferred or accepted.
- Valid time
- The period during which a proposition applies in the represented world.
- Recorded time
- When the system learned or stored the proposition.
- Supersession
- An explicit relation stating that one proposition replaces another for a defined scope and period.
- Decision receipt
- A record binding purpose, evidence, rules, identity, authority and the accepted decision.
- Effect receipt
- Evidence that an authorised downstream action produced the intended state, or remains unknown.
Build the case state first
Document intelligence should make a difficult decision smaller without making its uncertainty disappear. The durable architecture begins with admitted source objects, preserves every transformation, turns spans into typed propositions, keeps identity separate from capacity, models time and contradiction explicitly, and records rules and reviewer decisions as different objects. A narrative comes last because it is one view of the case, not the case itself.
This changes the engineering decision. Do not ask first which model can summarise the bundle. Ask which evidence state must survive model replacement, correction, audit and dispute. Use direct extraction for clean, low-consequence forms. Introduce reconstruction when versioning, identity, conditions or conflict determine the answer. Keep human authority where interpretation remains consequential or legitimately contested.
The release gate is simple to state and hard to satisfy: can the system preserve the exact disagreement that should change the decision? If a one-word mutation, role swap or missing schedule does not move the structured state before it moves the prose, the system is reading documents without reconstructing evidence. Its summaries may be fluent. Its decisions are not yet defensible.
sequenceDiagram participant Q as Question participant M as Mechanism participant E as Evidence participant D as Decision Q->>M: The amendment that a good summary swallowed M->>E: The one-word amendment thought experiment E-->>M: compare rival explanations E->>D: Sources D-->>Q: revise scope or act
Sources
- Mathew, M., Karatzas, D. and Jawahar, C. V. “DocVQA: A Dataset for VQA on Document Images.” WACV 2021. Primary paper.
- Huang, Y. et al. “LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking.” 2022. Primary paper and model description.
- Ma, Y. et al. “MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizations.” NeurIPS Datasets and Benchmarks 2024. Primary benchmark paper.
- Chia, Y. K. et al. “M-LongDoc: A Benchmark for Multimodal Super-Long Document Understanding and a Retrieval-Aware Tuning Framework.” EMNLP 2025. Primary benchmark paper.
- Zhang, J. et al. “OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation.” ICCV 2025. Primary benchmark paper.
- Koreeda, Y. and Manning, C. D. “ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts.” Findings of EMNLP 2021. Primary paper and dataset.
- Mathur, P. et al. “DocInfer: Document-level Natural Language Inference using Optimal Evidence Selection.” EMNLP 2022. Primary paper.
- Li, Y. et al. “Deep Entity Matching with Pre-Trained Language Models.” PVLDB 2020. Primary paper.
- “DocScope: Benchmarking Verifiable Reasoning for Trustworthy Long-Document Understanding.” 2026. Recent primary preprint.
- Zhang, M. “MMLongBench-Doc-V2: A Corrected-Annotation, Semantics-Aware Revision of MMLongBench-Doc.” 2026. Negative and correction result.
- W3C. “PROV-O: The PROV Ontology.” Recommendation. Official specification.
- W3C. “PROV-DM: The PROV Data Model.” Recommendation. Official specification.
- W3C. “Shapes Constraint Language (SHACL).” Recommendation. Official specification.
- Object Management Group. “Decision Model and Notation.” Official specification family.
- OASIS. “LegalRuleML Core Specification Version 1.0.” Official standard.
- NIST. “Artificial Intelligence Risk Management Framework 1.0.” Official framework.
- Greshake, K. et al. “Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection.” 2023. Primary security paper.