The nearest document is not yet evidence

Information retrieval began with a useful compression: given a query, rank documents by their estimated relevance. The probability ranking principle formalised the intuition that, under stated assumptions, documents should be ordered by their probability of usefulness to the information need. Robertson (1977) Later methods changed the scoring machinery, not the basic interface. BM25 models lexical evidence. Dense passage retrieval maps queries and passages into learned vector spaces, then returns a small top-k set from a much larger collection. Karpukhin et al. (2020)

Retrieval-augmented generation added a model that conditions generation on those passages. The original RAG formulation showed how parametric and non-parametric memory could be combined for knowledge-intensive tasks, while preserving a route back to retrieved sources. Lewis et al. (2020) This was an important architectural move. It did not, however, turn similarity into evidence law. A retriever proposes items that may help. It does not establish that the set is complete, mutually coherent, current, independent or permitted for the decision now being made.

Four hidden assumptions enter when a search interface is reused as a decision interface. The first is item independence: each highly ranked passage is treated as another unit of useful evidence, even when five passages copied the same upstream memo. The second is monotonicity: more relevant text is assumed to make the decision better, even when added text introduces contradiction, stale instructions or distraction. The third is availability equivalence: if the system can fetch an item, it may use it. The fourth is query completeness: the user’s sentence is assumed to name every proposition that must be established before action.

The surface introduces a distinction that top-k ranking hides. A document can be relevant but unusable. It can be usable but redundant. It can be authoritative but irrelevant to the decision subject. It can be current but insufficient because it proves only one required condition. These are not edge cases added after retrieval. They determine which context may be released to the model in the first place.

flowchart LR
  A["The answer was plausible. The<br/>context was not admissible."] --> B["The nearest document is not<br/>yet evidence"]
  B --> C["Three surfaces, one release rule"]
  C --> D["Build a governed retrieval loop"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#087A98,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Three surfaces, one release rule

The central proposal is to treat retrieval as constrained inference over an evidence set. The query is more than a sentence. It is a decision contract containing the requester, decision subject, purpose, contemplated action, cutoff time, required propositions, acceptable source classes and risk tolerance. Candidate generation still uses lexical, dense, graph or hybrid retrieval. The change occurs after candidate generation: the system asks what each item can establish, whether it may be used, what remains missing and whether another retrieval step has enough expected value to justify its cost and exposure.

This separation extends earlier retrieval work rather than replacing it. Maximal marginal relevance introduced a trade-off between relevance and novelty, reducing redundant results. Carbonell and Goldstein (1998) Adaptive retrieval methods decide whether to retrieve once, iterate or not retrieve, often using query complexity or model confidence. Jiang et al. (2023) Jeong et al. (2024) RAG evaluators commonly separate context relevance, answer faithfulness and answer relevance. Saad-Falcon et al. (2024) Es et al. (2024) The missing control question is whether the evidence set is sufficient and entitled for the decision contract, rather than only helpful to a generated answer.

The smallest useful formalism

Let x denote the decision contract and C the set of propositions that must be resolved. Let R_x contain only candidate items that pass the current entitlement policy for x. Each item contributes signed support to one or more propositions, adjusted by source authority, freshness and dependence. The system selects an evidence set E from R_x.

This is not a claim that real decision loss can always be known. It is a design discipline. The formalism forces the team to name what another document is expected to change. If no required proposition remains open, and the next candidate only repeats an existing source, the marginal value is near zero. If one safety-critical proposition is uncovered, a low-similarity document targeted to that gap may have very high value.

Attribute-based access control makes the relevant dimensions explicit: authorisation may depend on attributes of the subject, object, requested operation and environment. NIST SP 800-162 Zero trust removes implicit trust based on location and treats authentication and authorisation as discrete functions before access to a resource. NIST SP 800-207 Large-scale authorisation systems such as Zanzibar also show why consistency across permission changes matters. Pang et al. (2019) The retrieval design implication is direct: policy evaluation must occur on the evidence path, outside the language model, with a versioned decision that can be rechecked.

Sufficiency is a stopping condition, not a feeling

A practical evaluator represents each required proposition as an evidence slot. For every slot, it records supporting and refuting items, their authority, freshness and provenance group. Evidence copied from one origin should not be counted as independent confirmation. Strong support and strong refutation should produce a conflict state, not an averaged confidence that looks calm. Missing evidence should remain visible rather than being silently replaced by model knowledge.

The release rule can then be stated plainly: stop when every mandatory slot has adequate entitled support, any material contradiction is resolved or escalated, and the expected marginal value of another authorised item falls below its retrieval cost and exposure risk. Continue with a targeted subquery when one slot remains open. Abstain when the needed evidence exists but is not entitled. Escalate when the policy cannot decide, the sources conflict, or the contract itself is underspecified.

Sufficiency is not completeness

No finite context can contain everything that might bear on a real decision. The useful target is therefore not complete knowledge but decision-relative closure. A weather forecast can be sufficient for choosing an umbrella without being a complete atmospheric model. A current approval table can be sufficient to establish an authority limit without explaining how the organisation designed its delegation structure. Sufficiency always inherits the scope and loss model of the decision contract.

This makes threshold design consequential. A claim may require one authoritative source, two independent sources, or a direct system readback. Some claims are conjunctive: policy permission and current screening must both hold. Others are disjunctive: either a signed completion certificate or an authorised operational readback may close the same slot. Negative claims require particular care. “No defect exists” is not established by failing to retrieve a defect unless the search scope, index completeness and access filters make that absence interpretable.

The evaluator should therefore expose its logic rather than collapse it into a single confidence score. For each claim, a reviewer should see the accepted source classes, the aggregation rule, the independence groups, the freshness window and the evidence that would reverse the result. A threshold without those semantics is only a number attached to an opinion.

Build a governed retrieval loop

The architecture begins before the vector search. A decision request is compiled into a typed contract. The contract separates intent, identity, world state, permitted context, model proposal, authority, typed action and evidence receipt. Retrieval receives only the fields it needs. The language model never decides whether it is allowed to read a document. A policy engine makes that decision from structured attributes and returns an allow or deny result with policy identifiers and reasons.

Externalising authorisation is not unusual. NIST’s ABAC model evaluates subject, object, operation and environmental attributes. Cedar and Open Policy Agent show how applications can delegate structured policy decisions to dedicated engines rather than burying permission logic in free-form model prompts. Cutler et al. (2024) Open Policy Agent documentation The novel step here is to make those decisions part of retrieval semantics and sufficiency evaluation.

Step 1: compile the decision into claim slots

A natural-language query such as “Can we release the hold?” is too small to govern retrieval. The compiled contract might require four propositions: the current policy permits the release path; the work or goods were accepted; screening is clear at the cutoff; and the proposed approver holds sufficient authority. Each proposition names acceptable source classes and a threshold. A free-form model may suggest these slots, but a deterministic schema and domain policy approve them.

Step 2: generate candidates without granting trust

Lexical search is valuable when exact identifiers, clauses or policy numbers matter. Dense retrieval is valuable when wording varies. Graph traversal is useful when the question depends on entities and relations. Candidate generation can combine all three. The result is still a proposal list. Candidate metadata should include source type, version, subject, owner, sensitivity, validity interval, provenance group and policy-relevant attributes before content is released downstream.

Discover without disclosing

A practical system often needs two entitlement stages. The first stage decides whether a principal may discover that an item exists and inspect safe metadata. The second decides whether its content may be released for the declared purpose. This prevents a broad vector index from becoming an accidental disclosure channel. Candidate generation can operate on policy-safe representations, partitioned indexes or pre-authorised collections, then request content only after a precise resource-level decision.

The distinction matters because metadata can itself be sensitive. A document title, subject identifier or similarity score may reveal an investigation, diagnosis or commercial event. Safe discovery therefore needs its own contract. Where even metadata is restricted, retrieval should be pushed behind the owning service, which can return a typed assertion or a ranked opaque identifier rather than raw content. The model should receive the minimum evidence object that preserves the required proposition, not the maximum record the infrastructure can technically fetch.

Step 3: gate, map and assemble

The policy gate evaluates each requested use. Allowed items are mapped to claim slots. The assembler prefers independent coverage over repeated wording. It keeps refuting evidence rather than suppressing it. It can request a focused search for “current screening result for supplier entity X” instead of adding another generic policy passage. This is where retrieval becomes inference: the next query is chosen because a specific uncertainty remains.

Step 4: recheck and issue a retrieval receipt

Before the proposal is released, the system re-evaluates entitlement against the current policy version and cutoff. The receipt records the decision contract, selected item identifiers and versions, denied candidates without their sensitive contents, claim coverage, conflicts, missing slots, provenance groups, policy decisions and the final state. It should be possible to replay the evaluator without asking the language model to reconstruct what happened.

The receipt is not a claim that the recommendation is correct. It proves a narrower but valuable fact: given the declared contract, source metadata, policy snapshot and evaluator rules, this is the evidence set that was admitted, this is what it covered, and this is why retrieval stopped or abstained. Operating evidence should be specific enough to challenge the release, rather than only reassure the reviewer.

{}

function evaluate(contract, evidence, threshold, conflictCeiling) {
  const selected = evidence.filter(item => item.selected);
  const blocked = selected.filter(item => !entitled(item, contract));
  const admitted = selected.filter(item => entitled(item, contract));

  const claims = contract.claims.map(claim => {
    const groups = groupBy(admitted, item => item.provenance);
    const strongestPerGroup = Object.values(groups).map(items => ({
      support: Math.max(0, ...items.map(i => contribution(i, claim.id))),
      refute: Math.max(0, ...items.map(i => -contribution(i, claim.id)))
    }));
    const support = noisyOr(strongestPerGroup.map(g => g.support));
    const refute = noisyOr(strongestPerGroup.map(g => g.refute));
    return { ...claim, support, refute, conflict: Math.min(support, refute) };
  });

  const missing = claims.filter(c => c.support < threshold);
  const conflicts = claims.filter(c => c.conflict > conflictCeiling);
  const state = blocked.length ? "ENTITLEMENT_BLOCK"
    : conflicts.length ? "CONFLICT"
    : missing.length ? "INSUFFICIENT"
    : "SUFFICIENT";

  return { state, claims, blocked, admitted };
}

{
  "id": "screening-current",
  "title": "Current screening result",
  "subject": "supplier-orion-ltd",
  "purpose": ["invoice-hold-review"],
  "actions": ["recommend-release"],
  "validFrom": "T-30d",
  "validTo": "T+1d",
  "provenance": "compliance-service",
  "authority": 0.98,
  "freshness": 1.0,
  "relevance": 0.72,
  "contributions": {"screening_clear": 0.96}
}

stateDiagram-v2
  [*] --> Claim
  state "Change only the document’s status" as Claim
  state "The retrieval surface" as Test
  state "Hold the answer fixed and<br/>replace the evidence path" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Where the framework breaks

The strongest objection is that a sufficiency evaluator can create false assurance. That objection is correct. A contract that omits a decisive claim can certify the wrong evidence set. A source marked authoritative can still contain an error. A policy engine can faithfully enforce a defective policy. A language model can ignore admitted evidence, misread a table or produce an unsupported synthesis. Context sufficiency is not answer truth, policy correctness or model faithfulness.

RAG evaluation work already separates context relevance from answer faithfulness and answer relevance because a good passage does not guarantee a faithful answer. Saad-Falcon et al. (2024) Research on conflicting evidence similarly shows that models can struggle when retrieved sources disagree. Wang et al. (2025) The framework therefore keeps a hard boundary: the sufficiency evaluator governs the evidence bundle; a separate claim-level verifier tests whether the generated proposal is supported by that bundle.

Failure case: authorised poison

Suppose an attacker publishes a document that the index is permitted to retrieve. Its text imitates an official instruction, includes phrases likely to match high-value queries and tells the model to ignore policy metadata. Entitlement passes because the document is public. Relevance passes because the wording is tuned. The item is still untrustworthy. Recent security work on retrieval pipelines has shown how document-authored control signals can exploit the path from external text into model behaviour; these results are active research rather than settled coverage guarantees. Zhu (2026)

The defensive architecture does not let retrieved prose define its own authority. Source identity, policy decisions, provenance and control instructions live in typed channels outside document content. The model may quote the document, but it cannot promote the document to “system policy” by reading a sentence that says it is policy. Source-quality checks, signatures, trusted-source classes and adversarial testing remain necessary.

Test the framework against a rival

The most credible evaluation compares the governed loop with a strong alternative, not a deliberately weak search baseline. The rival should use hybrid retrieval, reranking, generous context, source citations and a capable model. Both systems receive the same synthetic decision contracts and corpora. The test set then injects one causal property at a time: a superseded document, a wrong-subject record, duplicated evidence, a late permission revocation, a missing authoritative collection, a material contradiction and an authorised poisoned document.

Measure at least four outcomes. First, did the system release a proposal when a mandatory claim was missing? Second, did any denied item influence the answer or appear in the disclosed context? Third, did the system distinguish conflict, insufficiency and entitlement block? Fourth, how much latency, abstention and reviewer effort did the controls add? A result strengthens the proposal if it lowers impermissible and unsupported releases without making routine cases unusable. It weakens the proposal if a simpler citation-and-verification pipeline achieves the same control outcomes at materially lower cost.

A rival explanation must also be tested: perhaps the benefit comes only from better query decomposition, not entitlement-aware sufficiency. An ablation can keep claim slots while removing policy gating, then keep policy gating while replacing claim slots with top-k. The comparison locates which mechanism changes which failure. This is more informative than reporting one aggregate answer score.

When a simpler system is better

For low-stakes exploration, discovery or creative research, a strict claim contract may be unnecessary. Users may value surprise, breadth and unresolved perspectives more than a release decision. A conventional search interface with provenance and access controls may be enough. The three-surface framework should be applied when a system proposes a consequential conclusion, selects evidence for another decision-maker, or performs an action whose justification may later be challenged.

Even there, thresholds should reflect reversibility. A reversible draft may tolerate lower coverage and display explicit gaps. A payment release, eligibility decision or safety intervention should demand stronger, fresher and more independent evidence. The goal is not maximum bureaucracy. It is proportional assurance: the harder the action is to reverse, the stronger the evidence contract and entitlement proof should be.

sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: The answer was plausible. The context was not admissible.
  M->>E: Change only the document’s status
  E-->>M: compare rival explanations
  E->>D: Let retrieval stop only on governed sufficiency
  D-->>Q: revise scope or act

Let retrieval stop only on governed sufficiency

When designing a retrieval-augmented decision system, the first question should no longer be “Which embedding model and what top-k?” Those choices matter, but they begin in the middle. Start with: What must be established before this decision may be proposed, which evidence is permitted for this subject and purpose, and what receipt would let a reviewer challenge the basis?

That change moves three responsibilities. Product and domain owners define the decision contract and its claim slots. Security, privacy and policy owners define entitlement outside the model. Retrieval engineers optimise candidate generation and targeted follow-up against the unresolved slots. The model proposes language and queries. It does not grant itself context, declare that context sufficient or convert a missing record into confidence.

The smallest useful implementation is modest. Add structured metadata to each candidate. Call a real policy decision point before content release. Represent three or four mandatory claims for one consequential workflow. Discount duplicate provenance. Keep support and refutation separate. Recheck policy before release. Emit a receipt that names admitted items, blocked items, missing claims and the stop reason. Then inject a superseded policy, a wrong-subject record, a duplicate source and a revocation during generation.

A system that passes this test has not proved truth or wisdom. It has achieved something narrower and operationally important: it knows why it stopped retrieving, what evidence it was allowed to use, what that evidence actually covered, and where the remaining uncertainty must go. That is the point at which retrieval stops being a similarity service attached to a model and becomes a governed inference process.