AI Research Library

Retrieval as Inference Under a Budget

Rajesh Ranjan Mahapatra

A retriever should choose the next admissible observation by its expected reduction in decision loss, then stop when further evidence cannot justify its cost.

Start reading

AI Research Library Paper

AI RESEARCH LIBRARYEdition

Retrieval as Inference Under a Budget

RAJESH RANJAN MAHAPATRA

Retrieval as Inference Under a BudgetAll AI Research Library

Retrieval as Inference Under a Budget

A retriever should choose the next admissible observation by its expected reduction in decision loss, then stop when further evidence cannot justify its cost.

TLDR

Retrieval is a sequential decision about which observation to acquire next, rather than a one-off race to place similar text near a model.
The useful quantity is expected reduction in consequence-weighted decision loss, after hard checks for purpose, authority, entitlement and freshness.
Budgets must keep calls, latency, tokens, money, privacy exposure and human interruption visible, because one blended cost score can conceal a breached limit.
A static hybrid retriever remains the stronger baseline when queries are homogeneous, sources are uniformly admissible and document interactions add little conditional value.
Architecture changes when the stopping rule, evidence contract and counterfactual no-retrieval route are designed before embeddings, rerankers or agent loops.

Worked scenario, synthetic

The fifth document is less useful than the first refusal

A payment operations assistant receives a request: “Release today’s supplier payment to the newly supplied bank account.” The account appears on an invoice. A manager’s message says the change is fine. An old signed approval PDF agrees. A current approval registry can answer in about 150 milliseconds. A broad mailbox search might uncover more discussion, but the assistant has no permission to sweep unrelated messages.

A conventional retrieval pipeline can produce an impressive top five. The invoice, message and PDF all share the right supplier, amount and account number. Their lexical and semantic relevance is high. Yet none authorises release now. The invoice is an assertion from the party seeking payment. The message is informative, not dispositive. The PDF is outside the decision’s one-hour freshness limit. The mailbox query is inadmissible. The current registry returns pending.

The best retrieval path is therefore one authorised observation followed by a stop. It does not maximise recall. It does not fill a context window. It reduces the live decision to an accountable hold while spending one call, 120 tokens of returned evidence and a small privacy allowance. Fetching the other four items would add surface support while leaving the permissible action unchanged.

This case exposes the paper’s causal claim. The value of a source depends on the decision, the evidence already acquired, the action it can establish, and the budget still available. That value changes after every observation. Retrieval should therefore be implemented as a policy over observations, with hard admissibility gates and an explicit stopping rule.

Figure 1. The useful route is a path through an evidence landscape. Synthetic geometry. Position indicates illustrative marginal decision value and acquisition burden, not measured performance. Hatching marks a hard admissibility boundary; the teal path spends one call and terminates.

Part one

Relevance is only the first probability

Information retrieval began with a disciplined question: given an information need, which documents are likely to be relevant? Maron and Kuhns framed probabilistic indexing in 1960. Robertson’s probability ranking principle later stated the conditions under which ranking documents by decreasing probability of relevance is optimal. Those ideas remain foundational. They solve a ranking problem whose loss is largely expressed through relevant and non-relevant items.

A consequential agent faces a larger state space. It must ask whether a source concerns the proposition, whether it may be accessed for this purpose, whether it can establish the proposition, whether it is current enough for the action, and whether observing it can still change a permissible decision. A relevance score answers none of those questions by itself. Fuhr’s extension of the ranking principle to interactive retrieval is important here: once retrieval actions change the state of the interaction, the assumptions behind a single static ranking become fragile.

Consider the phrase “the bank-account change was approved”. An email, a meeting transcript, a signed form and an approval-registry event can all express it. Their words may be almost identical. Their evidence roles differ. The transcript may explain intent. The form may establish a past approval. The registry may establish current workflow state. A retrieved item can be true and still be unable to authorise the present action.

Four quantities therefore collide:

Four tests applied to every candidate observation
Test	Question	Typical failure	Control form
Relevance	Could this observation bear on the required proposition?	Topical similarity without decisive evidence	Probabilistic score or classifier
Authority	What may this source establish for this decision?	Informing text treated as an authoritative record	Typed evidence role and source policy
Freshness	Does its valid time cover the action’s decision time?	Correct historical fact used as current state	Effective-time and supersession rule
Cost	What does observing it consume or expose?	Cheap tokens masking privacy or human-attention cost	Visible budget vector and hard limits

These are not four interchangeable features for one learned score. Relevance is usually uncertain and graded. Authority and permission often contain hard constraints. Freshness can be proposition-specific: a company address might tolerate a month-old record while a sanctions disposition might expire within minutes. Cost has several owners. A product team can price model tokens; a privacy owner sets a separate exposure boundary; an operations team owns human interruption and latency.

Authority belongs to a proposition, not a logo

Calling a repository “trusted” is too coarse. A supplier master may establish the legal entity and approved beneficiary account, while the workflow engine establishes whether a particular change has cleared dual control. The same repository can be authoritative for one field, merely informative for another and prohibited for a third purpose. Evidence roles should therefore be assigned to proposition types and actions, then carried with the observation into the decision receipt.

Freshness has the same shape. Retrieval time tells us when the system observed an item. Ingestion time tells us when an index copied it. Neither proves when the underlying fact was effective or when it ceased to apply. A current search result can contain a superseded rule; an older event can remain the governing fact if no later event replaced it. The controller needs valid time, observed time and supersession semantics where the domain requires them.

This precision changes candidate generation. Instead of asking for “documents about account approval”, the controller asks for an observation capable of establishing approval status for change identifier X at decision time T. Search can still broaden recall, but the required evidence role constrains what counts as closure. Source reputation may raise a prior; only a typed evidence contract determines what the source is allowed to settle.

Figure 2. One source passes through four different tests. Conceptual figure. The tests are deliberately separated because an item can be highly relevant while unauthorised, stale or too costly for the remaining budget.

Thought experiment: the perfect trespasser

Imagine an oracle that answers every query correctly. In the first condition, the oracle is a service the user has authorised for the stated purpose. In the second, the same answers are extracted from a private mailbox outside that purpose. Accuracy, latency and wording are held constant. Only entitlement changes.

A utility score that trades privacy against correctness can still choose the trespassing oracle when the predicted accuracy gain is high enough. That is the wrong control shape. Some conditions define the feasible action set before optimisation begins. Permission is not a weak negative relevance feature that a sufficiently confident model may outweigh. Attribute-based access control, as formalised in NIST SP 800-162, provides one vocabulary for evaluating subject, object, operation and environment against policy. W3C PROV provides a separate vocabulary for recording derivation and attribution. Neither specification supplies a retrieval policy, but both show why admission and provenance belong outside a model’s free-form judgement.

The thought experiment also separates two claims often fused in RAG discussions. “This passage would improve the answer” is a counterfactual statement about model behaviour. “The system may fetch and use this passage” is an authority statement. The second cannot be inferred from the first.

Minimal worked example: the bridge status

A traveller asks whether a bridge is open. Three observations are available: a cached tourism page, a live transport-status API and a roadside camera. For drafting a weekend itinerary, the cached page may be sufficient. For dispatching an ambulance, its expected decision value collapses because the cost of a false-open answer rises sharply. The live API becomes valuable even if its relevance score is unchanged. If the API reports closure, the camera’s value may then fall to zero because no permissible action depends on visual confirmation.

The example varies one thing at a time. The query text stays the same. The corpus stays the same. Changing consequence changes the loss function. Observing the API changes the marginal value of the camera. Retrieval value is conditional on both purpose and acquired evidence.

Figure 3. Retrieval changes a decision state, not merely a prompt. Synthetic probabilities from the executable example. The registry observation lowers expected loss and supplies an authoritative state; its value comes from the action it resolves.

Part two

The argument in one passChart · flow

flowchart TD
  A["The fifth document is less<br/>useful than the first refusal"] --> B["Relevance is only the first<br/>probability"]
  B --> C["A policy over observations"]
  C --> D["The payment route, step by step"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#D95545,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

A structural route through Retrieval as Inference Under a Budget, from The fifth document is less useful than… to Sources. Reading use: follow the relation, then test it against the surrounding evidence and limits.

A policy over observations

Retrieval-augmented generation made external evidence available to a parametric model. REALM learned a retriever during pre-training, while RAG combined a generator with non-parametric memory. Later systems made acquisition more selective. FLARE retrieves while generating around low-confidence spans. Self-RAG learns reflection tokens for retrieval and critique. Adaptive-RAG routes queries among no retrieval, single-step retrieval and iterative retrieval. TARG uses a short no-context draft as a training-free gate. The shared movement is from unconditional context injection towards a decision about when and how much to observe.

The stronger formulation begins before generation. Let s denote a possible world state, b(s) the system’s current belief, u a permissible business action and L(u,s) the loss of taking action u when state s is true. The current Bayes risk is the loss of the best action available under present evidence:

R(b) = min_{u ∈ U(b)} Σ_s b(s) L(u,s)

U(b) matters. It is the set of actions currently authorised by evidence and policy, not every action the model can name. In the payment example, release does not enter that set until a current authoritative record establishes approval. A persuasive manager message may change belief without changing U(b).

Now consider an observation action o, such as calling the approval registry. It can return outcomes y. Each outcome updates belief to b_y and may change the permissible action set. The expected value of sample information is:

EVSI(o | b) = R(b) − Σ_y P(y | b,o) R(b_y)

This quantity asks a concrete counterfactual question: before paying to observe o, how much decision loss should it remove on average? A source can be relevant yet have zero EVSI because every possible outcome leads to the same action. Confidence is different too: an uncertain observation can be valuable when it separates two actions with asymmetric consequences. Information gain is also insufficient. A surprising fact can compress uncertainty while leaving the decision unchanged.

Lindley’s treatment of the information supplied by an experiment and Howard’s information value theory provide the older decision-theoretic spine. The engineering adaptation here adds typed evidence roles, hard authority gates and operational budget dimensions. That addition is practitioner synthesis, not a claim that classical value-of-information theory already specifies enterprise retrieval.

Decision value is not answer confidence

Suppose a model assigns 80 per cent probability to “approved”. If the assistant is drafting a note, that confidence might support tentative language. If it is releasing money, the same probability can be unacceptable because the loss of a false release dominates the cost of a short hold. Conversely, an observation that moves confidence from 80 to 90 per cent may have no value when both beliefs imply the same permissible action.

EVSI captures this through the loss table and action set. It also supplies a useful ceiling: no imperfect observation can be worth more than perfect information about the relevant state. If a controller claims large value for a source when even perfect knowledge would not change the decision, its estimator or action model is broken. That property is valuable for tests because it catches confident but causally irrelevant retrieval scores.

The numbers remain estimates. Outcome likelihoods can drift, rare harms can be badly sampled and stakeholders can disagree about losses. Report intervals or scenario bands where precision is weak. A conservative policy can use the lower bound of estimated value, or require a margin above cost before acquisition. Mathematical notation does not remove judgement; it makes the location and consequence of judgement inspectable.

For every candidate, construct a visible cost vector:

c(o) = [calls, latency, tokens, money, privacy exposure, attack surface, human interruption].

Some dimensions are hard budgets. Others can enter a declared utility function after their weights have an accountable owner. A controller should first reject observations outside purpose, entitlement, freshness or remaining hard budgets. Among the feasible set F, it can select the observation with highest positive net value:

o* = arg max_{o ∈ F(b,B)} [ EVSI(o | b) − C_w(c(o)) ]

B is the remaining budget vector. C_w is an explicitly owned cost function, not an invitation to hide every concern in one number. An observation that breaches a privacy limit never reaches the maximisation. An observation that fits all hard limits can be compared through declared trade-offs. The architecture preserves the difference between “forbidden” and “permitted but not worth it”.

One-step value is a baseline, not the whole policy

The equation above is myopic: it values the next observation through the decision available immediately afterwards. That is appropriate for the executable example, but it can miss option value. An identity-resolution call may not change the payment action at all; its benefit is that it produces the verified key required to query the approval registry. A one-step selector could wrongly discard it.

A multi-step controller adds the expected value of later feasible observations after each possible outcome. In effect, it compares short acquisition plans rather than isolated calls. This is closer to a partially observable planning problem, and it is where decomposition, tool preconditions and AND/OR dependencies enter retrieval. The gain comes with a cost: longer lookahead amplifies likelihood error, expands the search tree and can make a simple route harder to audit.

Use one-step EVSI as the controlled baseline. Add two-step lookahead only for measured prerequisite patterns, and compare it with a deterministic dependency rule. Planning depth should be purchased by observed option value, not by calling every retrieval loop an agent.

Thought experiment: the second copy

Suppose two search results quote the same regulatory notice. In the first condition, the second result was independently produced from a separate primary record. In the second, it copied the first notice through a syndication feed. Text, topical relevance and apparent agreement are held constant. Only lineage changes. Reading the first may sharply reduce uncertainty; reading the copied result adds almost no independent support.

A ranker that scores documents independently cannot represent that collapse in marginal value. A sequential policy can, provided it records provenance and updates after each observation. The experiment also shows why “more citations” is not a sufficient assurance target. Two links may represent one evidential event.

This is why the acquisition state needs more than a query string. It should contain the decision, required propositions, present beliefs, evidence already accepted, provenance links, evidence roles, effective times, remaining budgets and unresolved contradictions. The state is a compact model of what the decision still needs, not a transcript of everything encountered.

Figure 4. Document value is conditional on prior observations. Illustrative bars, not measurements. Provenance-aware state prevents duplicated lineage from masquerading as independent confirmation.

Stopping is part of retrieval

A retriever without a stop rule is an evidence accumulator. It continues until a fixed k, a context limit or an agent-loop cap. Those are resource ceilings, not epistemic stopping conditions. A defensible controller stops when one of four conditions holds:

The evidence contract is satisfied. Required propositions have acceptable evidence roles, validity and confidence for the proposed action.
No feasible observation has positive marginal value. More evidence cannot change the action enough to justify its burden.
A hard budget is exhausted. The system returns a bounded incomplete state rather than silently exceeding scope.
The next decision belongs to a person or another authority. Retrieval yields an escalation packet, not an invented resolution.

Stopping protects quality as well as cost. “Lost in the Middle” demonstrated that language models do not use relevant information uniformly across long contexts. Self-knowledge-guided retrieval experiments have also shown that retrieval can hurt when external context distracts a model that already knows the answer. Recent budget-aware active-RAG work sharpens the point. A 2026 preprint evaluates utility as the change in correctness relative to no retrieval. It reports non-trivial retrieval harm, router rankings that vary by dataset and budget, threshold policies that can miss nominal budgets, and simple baselines that often remain competitive. These are preprint results rather than a universal law. They still make a no-retrieval counterfactual mandatory.

A second 2026 preprint studies cost-aware evidence selection under explicit access budgets. It reports that no fixed selector dominates and that larger access budgets do not reliably improve answer quality. Its access costs are simulated proxies, which limits operational interpretation. The useful lesson is narrower: evidence composition and stopping can matter more than the nominal quantity retrieved.

Figure 5. Budgets form a constrained frontier, not one scalar score. Illustrative relationship. Point area represents token burden; axes represent latency and privacy exposure. The forbidden region is excluded before value optimisation.

Part three

The payment route, step by step

Return to the supplier payment. Assume beneficiary identity and current screening are already established. The unresolved proposition is narrower: Has the bank-account change been approved in the current workflow? The action set contains hold. Release becomes available only if a permitted, current, authoritative observation establishes approval.

The controller begins with a 55 per cent belief that approval is complete and a 45 per cent belief that it is pending. Those probabilities are synthetic. The loss table is also synthetic: releasing while pending costs 100 units; holding an approved payment costs eight; holding a pending payment costs one. Under current evidence, hold has expected loss 4.85. Belief alone cannot unlock release.

Candidate observations for the unresolved approval proposition
Observation	Role	Admission	Expected effect	Route result
Current approval registry	Authoritative record	Permitted and current	Can establish approved or pending	Selected first
Manager message	Informing source	Permitted	May change belief; cannot unlock release	Net value below zero
Signed approval PDF	Formerly authoritative	Stale	Could misstate current workflow state	Rejected before scoring
Mailbox sweep	Broad discovery	Forbidden	May find topical discussion	Rejected before scoring

The registry’s expected value of sample information is 3.911 loss units in the accompanying programme. Its explicit cost penalty is 0.253 under the example’s declared weights, leaving positive net value. The manager message has zero decision value because every possible message outcome leaves release unavailable. The stale PDF and mailbox sweep never enter optimisation.

The registry reports pending. The posterior probability of pending rises to approximately 98.8 per cent. Hold remains the action, now with expected loss about 1.085. No remaining permitted observation can create an approval event or change the permissible action. The controller stops. The stop is a successful inference result, not a retrieval failure.

Figure 6. One authoritative observation can terminate the route. Synthetic sequence. Rejected branches are visible so that restraint is auditable; the decision receipt records why further acquisition was unnecessary.

Negative control: the serious static baseline

Adaptive retrieval is unnecessary in an important regime. Suppose every query asks for one exact policy identifier. The corpus contains one current, authorised record per identifier. Evidence items do not interact. Consequence and budget are stable. A sparse exact match or well-tuned hybrid retriever returns the decisive record at rank one.

In that regime, a controller has no selection advantage. It estimates values, manages state and emits traces only to choose the same item. The executable negative control makes this explicit: static top one and adaptive selection return the approval registry; the quality gain is zero, while the controller adds illustrative orchestration latency.

This is more than a toy objection. NEST, presented in the ACL 2026 industry track, uses recall-first expansion followed by progressive evidence pruning rather than an open-ended agent loop. Its reported results show why strong non-agentic retrieval and pruning belong in the baseline set. The architecture decision should depend on measured heterogeneity and interaction, not the prestige of adaptation.

Failure boundary

The policy formulation loses value when observation outcomes cannot be calibrated, the loss function has no accountable owner, sources are uniformly admissible and independent, or latency is so tight that policy computation costs more than retrieval. In those conditions, use a static route and keep the decision contract outside the retriever.

The inverse boundary also matters. A learned controller can appear sophisticated while estimating EVSI from the same language model that generates the answer. Correlated error then infects proposal, confidence, likelihood and evaluation. Independent outcome labels, deterministic admission checks and human-owned consequence weights are needed before the score deserves operational authority.

Figure 7. Adaptation earns its overhead only in the upper-right regime. Practitioner decision map, illustrative rather than measured. Static retrieval is the preferred lower-left baseline; sequential control becomes plausible as heterogeneity, consequence and evidence dependence rise.

The retrieval decision card

A team can test whether it has designed inference or merely installed a search stack by completing the following card. Each answer should be machine-readable in the release manifest or evidence contract.

Budgeted retrieval decision card

Decision: What action, recommendation or claim will the evidence change?
Consequence: Which errors are asymmetric, irreversible or expensive to recover?
Propositions: What exact facts remain unresolved, and which may stay uncertain?
Evidence roles: Which source types may inform, corroborate or establish each proposition?
Admission: What purpose, identity, entitlement, field scope, freshness and provenance checks are hard gates?
Budgets: What are the separate limits for calls, latency, tokens, money, privacy, attack exposure and human interruption?
Selection: How is marginal reduction in decision loss estimated, calibrated and compared with the no-retrieval route?
Stop: What evidence-complete, no-positive-value, budget-exhausted and escalation states terminate acquisition?

Part four

Build an evaluator that can prove restraint

The controller is only as credible as the experiment that can reject it. Answer accuracy at one fixed k is insufficient. It cannot show whether retrieval was needed, whether the chosen source was admissible, whether a cheaper path existed, whether duplicated evidence inflated confidence, or whether stopping happened for the right reason.

Begin with five baselines: no retrieval; static top k; static hybrid retrieval with reranking and pruning; a retrieve-or-not confidence gate; and the sequential policy. Keep the generator fixed where possible. Evaluate across budget levels rather than at one cap. Recent work on budget-aware active RAG uses exact top-k and threshold frontiers for this reason. Query-decomposition research at EACL 2026 also formulates retrieval as sequential allocation of a fixed budget across sub-queries, offering evidence that interaction-aware allocation can improve the composition of retrieved evidence. Those results motivate the experiment; they do not remove the need for a domain-specific baseline.

Measure at least six outcomes:

Decision loss: the consequence-weighted error of the resulting action, including abstentions and holds.
Evidence sufficiency: whether each required proposition has the permitted evidence role, freshness and provenance.
Budget compliance: calls, tail latency, tokens, money, privacy exposure and human interruptions, reported separately.
Acquisition harm: cases where retrieving changes a correct no-retrieval decision into an incorrect or unauthorised one.
Route regret: additional loss and cost relative to the best admissible path known after replay.
Stop quality: premature stops, wasteful continuation and correct escalation.

CRUX argues that conventional retrieval-ranking metrics are insufficient for long-form RAG and evaluates whether context actually covers the claims a response must support. That distinction should be extended to decisions: rank quality is a component measure; verified evidence coverage and action quality are outcome measures. A retriever passes only when the evidence it acquires survives the decision contract.

Open hypothesis

A proposition-level policy that applies deterministic authority and freshness gates, then selects observations by conservative marginal decision value, will dominate fixed top-k retrieval on consequential tasks with heterogeneous sources and dependent evidence. It should fail to dominate on homogeneous exact-lookup tasks. That two-sided prediction is the experiment.

Calibration through replay, not introspection alone

A policy needs estimates for observation outcomes and their effect on decisions. Model self-confidence is one input, but it is a weak foundation when proposal, confidence and judgement share the same errors. Build estimates from logged cases in which candidate observations, outcomes and downstream decisions can be replayed. Where historical policy selected only some observations, account for that selection bias rather than treating missing outcomes as failures.

Calibration should be checked at the level used by the controller. If it chooses among tools, test whether predicted value orders tools correctly. If it gates retrieval, test utility and harm at each threshold. If it controls a privacy budget, verify constraint violations at the tail, not only average spend. A controller can be well calibrated for answer correctness and poorly calibrated for route value because the cost and action models are wrong.

Failure injection before optimisation

Construct paired cases in which one causal feature changes. Make the most relevant source forbidden. Make the authoritative source stale. Duplicate one document through three indexes. Return contradictory outcomes from two current systems. Hide the decisive source behind a slower permitted tool. Give the no-retrieval model a correct answer and observe whether weak context degrades it. Exhaust one budget dimension while leaving the others generous. Then test whether the controller rejects, continues, stops or escalates as specified.

The most important failures are not low scores. They are category errors: an informing source promoted to authority, a historical fact treated as current state, an unpermitted observation optimised into the route, or an unknown outcome narrated as success. These should be typed assertion failures in the harness.

Figure 8. Release on a utility frontier, not a single accuracy point. Illustrative curves. A sequential controller may lose at tiny budgets because orchestration has a cost; it earns release only if its frontier dominates strong static and gated baselines without breaching hard constraints.

Executable artefact: a minimal budgeted retriever

The accompanying Python programme implements the payment example on synthetic data. It calculates Bayes risk and expected value of sample information, rejects observations that fail permission, purpose, freshness or remaining budget, charges a visible cost penalty, chooses the highest positive net value, updates belief and stops. It also runs the homogeneous exact-lookup negative control.

The programme deliberately omits embeddings and a language model. That is a feature of the experiment. It isolates the control logic from the mechanism that proposes candidates or estimates likelihoods. In a larger system, dense or sparse retrieval can generate candidate observations; calibrated models or empirical replay can estimate outcome likelihoods; deterministic services enforce hard gates.

Inspect the implementation in this paper Python 3.10 or later; standard library only.

View the complete runnable programme

#!/usr/bin/env python3
"""A tiny, synthetic retriever that selects observations by expected decision value.

The example asks whether a supplier bank-account change has been approved.  It is
not a search benchmark and it does not connect to any real system.  Its purpose
is to make four pieces of retrieval logic executable:

1. hard admission checks for purpose, permission and freshness;
2. expected reduction in consequence-weighted decision loss;
3. an explicit multi-dimensional acquisition budget; and
4. a stopping rule when no remaining observation has positive net value.

Run with:  python budgeted_retriever.py
Expected result: the current approval registry is selected, a pending result
leads to HOLD, forbidden and stale observations are rejected, and the static
negative control chooses the same decisive record with less orchestration.
"""

from __future__ import annotations

from dataclasses import dataclass, replace
from enum import Enum
from typing import Dict, Iterable, Mapping, Optional, Tuple


class World(str, Enum):
    APPROVED = "approved"
    PENDING = "pending"


class Decision(str, Enum):
    RELEASE = "release"
    HOLD = "hold"


Prior = Dict[World, float]
Likelihoods = Mapping[World, Mapping[str, float]]


@dataclass(frozen=True)
class Budget:
    calls: int
    tokens: int
    latency_ms: int
    privacy_points: float

    def permits(self, cost: "Budget") -> bool:
        return (
            cost.calls <= self.calls
            and cost.tokens <= self.tokens
            and cost.latency_ms <= self.latency_ms
            and cost.privacy_points <= self.privacy_points
        )

    def spend(self, cost: "Budget") -> "Budget":
        if not self.permits(cost):
            raise ValueError("Observation exceeds the remaining budget")
        return Budget(
            calls=self.calls - cost.calls,
            tokens=self.tokens - cost.tokens,
            latency_ms=self.latency_ms - cost.latency_ms,
            privacy_points=round(self.privacy_points - cost.privacy_points, 4),
        )


@dataclass(frozen=True)
class EvidenceState:
    """Whether an authoritative, current record has established approval."""

    approval_established: Optional[bool] = None


@dataclass(frozen=True)
class Observation:
    name: str
    role: str
    likelihoods: Likelihoods
    cost: Budget
    permitted: bool
    purpose_match: bool
    age_minutes: int
    max_age_minutes: int
    establishes_approval: bool
    note: str

    def hard_rejection(self, remaining: Budget) -> Optional[str]:
        if not self.permitted:
            return "permission denied"
        if not self.purpose_match:
            return "purpose mismatch"
        if self.age_minutes > self.max_age_minutes:
            return "stale for this decision"
        if not remaining.permits(self.cost):
            return "budget exceeded"
        return None


# Loss is consequence-weighted, not a claim that these numbers are universal.
LOSS: Mapping[Decision, Mapping[World, float]] = {
    Decision.RELEASE: {World.APPROVED: 0.0, World.PENDING: 100.0},
    Decision.HOLD: {World.APPROVED: 8.0, World.PENDING: 1.0},
}

# The scalar penalty is owned policy.  Hard constraints are checked first.
CALL_PENALTY = 0.08
TOKEN_PENALTY = 0.0004
LATENCY_PENALTY = 0.0005
PRIVACY_PENALTY = 0.50


def normalise(prior: Mapping[World, float]) -> Prior:
    total = sum(prior.values())
    if total <= 0:
        raise ValueError("Probabilities must have positive mass")
    return {world: value / total for world, value in prior.items()}


def allowed_decisions(state: EvidenceState) -> Tuple[Decision, ...]:
    # A model's belief alone cannot authorise release.  Release becomes an
    # available action only after a current authoritative record says approved.
    if state.approval_established is True:
        return (Decision.RELEASE, Decision.HOLD)
    return (Decision.HOLD,)


def bayes_risk(prior: Mapping[World, float], state: EvidenceState) -> Tuple[Decision, float]:
    candidates = allowed_decisions(state)
    scored = []
    for decision in candidates:
        risk = sum(prior[w] * LOSS[decision][w] for w in World)
        scored.append((risk, decision))
    risk, decision = min(scored, key=lambda item: (item[0], item[1].value))
    return decision, risk


def outcome_probability(prior: Mapping[World, float], obs: Observation, outcome: str) -> float:
    return sum(prior[w] * obs.likelihoods[w].get(outcome, 0.0) for w in World)


def posterior(prior: Mapping[World, float], obs: Observation, outcome: str) -> Prior:
    weighted = {
        w: prior[w] * obs.likelihoods[w].get(outcome, 0.0)
        for w in World
    }
    return normalise(weighted)


def state_after(state: EvidenceState, obs: Observation, outcome: str) -> EvidenceState:
    if not obs.establishes_approval:
        return state
    if outcome == "approved":
        return replace(state, approval_established=True)
    if outcome == "pending":
        return replace(state, approval_established=False)
    return state


def observation_outcomes(obs: Observation) -> Tuple[str, ...]:
    return tuple(sorted({key for row in obs.likelihoods.values() for key in row}))


def expected_value_of_sample_information(
    prior: Mapping[World, float], state: EvidenceState, obs: Observation
) -> float:
    _, current_risk = bayes_risk(prior, state)
    expected_posterior_risk = 0.0
    for outcome in observation_outcomes(obs):
        probability = outcome_probability(prior, obs, outcome)
        if probability == 0:
            continue
        next_prior = posterior(prior, obs, outcome)
        next_state = state_after(state, obs, outcome)
        _, next_risk = bayes_risk(next_prior, next_state)
        expected_posterior_risk += probability * next_risk
    return current_risk - expected_posterior_risk


def visible_cost_penalty(cost: Budget) -> float:
    return (
        CALL_PENALTY * cost.calls
        + TOKEN_PENALTY * cost.tokens
        + LATENCY_PENALTY * cost.latency_ms
        + PRIVACY_PENALTY * cost.privacy_points
    )


@dataclass(frozen=True)
class CandidateScore:
    observation: Observation
    evsi: float
    penalty: float
    net_value: float
    rejected: Optional[str]


def score_candidates(
    prior: Mapping[World, float],
    state: EvidenceState,
    remaining: Budget,
    observations: Iterable[Observation],
) -> Tuple[CandidateScore, ...]:
    rows = []
    for obs in observations:
        rejected = obs.hard_rejection(remaining)
        if rejected:
            rows.append(CandidateScore(obs, 0.0, 0.0, float("-inf"), rejected))
            continue
        evsi = expected_value_of_sample_information(prior, state, obs)
        penalty = visible_cost_penalty(obs.cost)
        rows.append(CandidateScore(obs, evsi, penalty, evsi - penalty, None))
    return tuple(sorted(rows, key=lambda row: row.net_value, reverse=True))


def choose_next(scores: Iterable[CandidateScore]) -> Optional[CandidateScore]:
    eligible = [row for row in scores if row.rejected is None and row.net_value > 0]
    return max(eligible, key=lambda row: row.net_value, default=None)


def most_likely_outcome(obs: Observation, hidden_world: World) -> str:
    return max(obs.likelihoods[hidden_world], key=obs.likelihoods[hidden_world].get)


def make_observations() -> Tuple[Observation, ...]:
    return (
        Observation(
            name="approval_registry",
            role="authoritative record",
            likelihoods={
                World.APPROVED: {"approved": 0.99, "pending": 0.01},
                World.PENDING: {"approved": 0.01, "pending": 0.99},
            },
            cost=Budget(1, 120, 150, 0.10),
            permitted=True,
            purpose_match=True,
            age_minutes=0,
            max_age_minutes=5,
            establishes_approval=True,
            note="Current system of record for payment-change approval.",
        ),
        Observation(
            name="manager_message",
            role="informing source",
            likelihoods={
                World.APPROVED: {"sounds_approved": 0.80, "unclear": 0.20},
                World.PENDING: {"sounds_approved": 0.35, "unclear": 0.65},
            },
            cost=Budget(1, 80, 60, 0.05),
            permitted=True,
            purpose_match=True,
            age_minutes=8,
            max_age_minutes=1_440,
            establishes_approval=False,
            note="May inform the next query but cannot establish approval.",
        ),
        Observation(
            name="stale_approval_pdf",
            role="formerly authoritative record",
            likelihoods={
                World.APPROVED: {"approved": 0.95, "pending": 0.05},
                World.PENDING: {"approved": 0.25, "pending": 0.75},
            },
            cost=Budget(1, 300, 90, 0.05),
            permitted=True,
            purpose_match=True,
            age_minutes=4_320,
            max_age_minutes=60,
            establishes_approval=True,
            note="Correctly signed, but older than the decision's freshness limit.",
        ),
        Observation(
            name="mailbox_sweep",
            role="broad discovery",
            likelihoods={
                World.APPROVED: {"support": 0.90, "no_support": 0.10},
                World.PENDING: {"support": 0.30, "no_support": 0.70},
            },
            cost=Budget(1, 1_100, 850, 1.80),
            permitted=False,
            purpose_match=False,
            age_minutes=0,
            max_age_minutes=60,
            establishes_approval=False,
            note="Potentially relevant, but outside the granted field and purpose scope.",
        ),
    )


def run_positive_case() -> None:
    print("POSITIVE CASE: consequential supplier-payment decision")
    prior: Prior = {World.APPROVED: 0.55, World.PENDING: 0.45}
    state = EvidenceState()
    remaining = Budget(calls=2, tokens=500, latency_ms=500, privacy_points=0.50)
    observations = list(make_observations())

    decision, risk = bayes_risk(prior, state)
    print(f"Initial action: {decision.value.upper()} | expected loss: {risk:.3f}")

    scores = score_candidates(prior, state, remaining, observations)
    for row in scores:
        if row.rejected:
            print(f"  REJECT {row.observation.name}: {row.rejected}")
        else:
            print(
                f"  SCORE  {row.observation.name}: "
                f"EVSI={row.evsi:.3f}, cost={row.penalty:.3f}, net={row.net_value:.3f}"
            )

    chosen = choose_next(scores)
    assert chosen is not None
    assert chosen.observation.name == "approval_registry"
    print(f"Selected: {chosen.observation.name}")

    # Deterministic synthetic world for the worked run.
    outcome = most_likely_outcome(chosen.observation, World.PENDING)
    print(f"Observed: {outcome}")
    prior = posterior(prior, chosen.observation, outcome)
    state = state_after(state, chosen.observation, outcome)
    remaining = remaining.spend(chosen.observation.cost)
    observations = [obs for obs in observations if obs.name != chosen.observation.name]

    decision, risk = bayes_risk(prior, state)
    print(f"Updated action: {decision.value.upper()} | expected loss: {risk:.3f}")
    follow_up = choose_next(score_candidates(prior, state, remaining, observations))
    assert follow_up is None
    assert decision is Decision.HOLD
    assert any(row.observation.name == "mailbox_sweep" and row.rejected for row in scores)
    assert any(row.observation.name == "stale_approval_pdf" and row.rejected for row in scores)
    print("Stop: no admissible remaining observation has positive marginal decision value.")
    print("PASS: hard gates, value calculation and stopping rule behaved as intended.\n")


def run_negative_control() -> None:
    print("NEGATIVE CONTROL: homogeneous exact-lookup corpus")
    observations = make_observations()
    registry = next(obs for obs in observations if obs.name == "approval_registry")
    prior: Prior = {World.APPROVED: 0.55, World.PENDING: 0.45}
    state = EvidenceState()
    budget = Budget(1, 200, 250, 0.20)

    adaptive = choose_next(score_candidates(prior, state, budget, [registry]))
    static_top_one = registry
    assert adaptive is not None
    assert adaptive.observation.name == static_top_one.name

    # The controller cannot improve selection when the corpus has one permitted,
    # current and decisive record.  It only adds orchestration overhead.
    controller_overhead_ms = 20
    print(f"Static top-1: {static_top_one.name}")
    print(f"Adaptive choice: {adaptive.observation.name}")
    print(f"Quality gain: 0 | illustrative controller overhead: {controller_overhead_ms} ms")
    print("PASS: use the static route when heterogeneity and interaction are absent.")


if __name__ == "__main__":
    run_positive_case()
    run_negative_control()

View expected output and test interpretation

The positive case selects the registry and stops after pending. The negative control confirms that a static top-one route makes the same selection when there is only one admissible decisive record.

POSITIVE CASE: consequential supplier-payment decision
Initial action: HOLD | expected loss: 4.850
  SCORE  approval_registry: EVSI=3.911, cost=0.253, net=3.658
  SCORE  manager_message: EVSI=0.000, cost=0.167, net=-0.167
  REJECT stale_approval_pdf: stale for this decision
  REJECT mailbox_sweep: permission denied
Selected: approval_registry
Observed: pending
Updated action: HOLD | expected loss: 1.085
Stop: no admissible remaining observation has positive marginal decision value.
PASS: hard gates, value calculation and stopping rule behaved as intended.

NEGATIVE CONTROL: homogeneous exact-lookup corpus
Static top-1: approval_registry
Adaptive choice: approval_registry
Quality gain: 0 | illustrative controller overhead: 20 ms
PASS: use the static route when heterogeneity and interaction are absent.

From prototype to operating control

A production-shaped implementation needs four boundaries around the policy. First, a candidate generator returns typed observation actions, not raw passages. Each action declares source, fields, purpose, expected evidence role, effective-time semantics and cost estimate. Second, an authority kernel admits or rejects the action through identity and policy. Third, an observer returns a signed effect receipt or an explicit unknown outcome. Fourth, the controller updates its evidence state and writes a decision receipt.

Authority attached to a source must come from runtime policy, never the language model. Ingestion time cannot stand in for source validity. An HTTP success does not prove the requested effect. A blended reward must never hide privacy breaches. These controls belong to the runtime even when candidate selection is learned.

A reasonable release sequence is incremental:

Establish no-retrieval and static hybrid baselines.
Add deterministic admission, evidence roles and freshness without adaptation.
Add a retrieve-or-not gate and measure acquisition harm.
Add sequential selection only where route interaction and budget heterogeneity are demonstrated.
Release consequence-sensitive actions only after counterfactual replay, failure injection and calibrated stopping meet their thresholds.

The controller should earn every added degree of freedom against the simpler route. This keeps “agentic retrieval” from becoming a label for extra calls.

How the claim changes under pressureChart · state

stateDiagram-v2
  [*] --> Claim
  state "Authority belongs to a<br/>proposition, not a logo" as Claim
  state "Thought experiment, the<br/>perfect trespasser" as Test
  state "Minimal worked example, the<br/>bridge status" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

The revision loop begins with Authority belongs to a proposition, not… and keeps counterexamples inside the reasoning path. Reading use: follow the relation, then test it against the surrounding evidence and limits.

Compact glossary

Observation action: A permitted operation that can acquire a typed item of evidence, such as reading a record or calling a status API.
Evidence role: What a source may do for a proposition: inform, corroborate, establish, contradict or supersede.
EVSI: Expected value of sample information: the expected reduction in decision loss from one observation before paying for it.
Decision receipt: A replayable record of purpose, evidence, policy, chosen action, unresolved uncertainty and stop reason.

Build the stopping rule before the search stack

A retriever becomes decision-worthy when it can explain three things: why this observation was admissible, why it had greater marginal value than the alternatives, and why acquisition stopped when it did. Similarity search remains essential for proposing candidates. Reranking remains essential for ordering them. Neither supplies those three answers.

The architecture decision changes from “Which documents should enter the context?” to “Which permitted observation should reduce the remaining decision loss?” The design must also name the budget and the outcome that will cause a stop. This shift makes relevance, authority, freshness and cost comparable where trade-offs are legitimate, while preserving hard boundaries where they are not.

The research programme is equally concrete. Compare no retrieval, strong static retrieval, gating and sequential control on decision loss, evidence sufficiency, acquisition harm, budget compliance, route regret and stop quality. Inject forbidden, stale, duplicated and contradictory evidence. Reject the adaptive system when it cannot beat the static baseline. Release it only where conditional evidence value and asymmetric consequence make the extra control observable.

In the payment case, the right answer was one registry call and a hold. The system’s intelligence was visible in the evidence it declined to fetch.

The evidence conversationChart · sequence

sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: The fifth document is less useful than the first refusal
  M->>E: Authority belongs to a proposition, not a logo
  E-->>M: compare rival explanations
  E->>D: Sources
  D-->>Q: revise scope or act

Question, mechanism, evidence and decision exchange the burdens developed in Retrieval as Inference Under a Budget. Reading use: follow the relation, then test it against the surrounding evidence and limits.

Sources

Maron, M. E., and Kuhns, J. L. (1960). On Relevance, Probabilistic Indexing and Information Retrieval. Journal of the ACM.
Robertson, S. E. (1977). The Probability Ranking Principle in IR. Journal of Documentation.
Fuhr, N. (2008). A probability ranking principle for interactive information retrieval. Information Retrieval.
Lindley, D. V. (1956). On a Measure of the Information Provided by an Experiment. Annals of Mathematical Statistics.
Howard, R. A. (1966). Information Value Theory. IEEE Transactions on Systems Science and Cybernetics.
Guu, K. et al. (2020). REALM: Retrieval-Augmented Language Model Pre-Training.
Lewis, P. et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS.
Jiang, Z. et al. (2023). Active Retrieval Augmented Generation. EMNLP.
Wang, Y. et al. (2023). Self-Knowledge Guided Retrieval Augmentation for Large Language Models. Findings of EMNLP.
Asai, A. et al. (2024). Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. ICLR.
Jeong, S. et al. (2024). Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity. NAACL.
Liu, N. F. et al. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the ACL.
Controlled Retrieval-augmented Context Evaluation for Long-form RAG (2025). Findings of EMNLP.
Retrieval as a Decision: Training-Free Adaptive Gating for Efficient RAG (2026). Transactions on Machine Learning Research.
Query Decomposition for Retrieval-Augmented Generation: Balancing Exploration and Exploitation (2026). EACL.
NEST: Nested Evidence Survival for Retrieval (2026). ACL Industry Track.
When Should Active RAG Retrieve? A Budget-Aware Evaluation of Utility, Calibration, and Cost (2026 preprint).
When Knowledge Is Not Free: Cost-Aware Evidence Selection in Retrieval-Augmented Generation (2026 preprint).
NIST. SP 800-162: Guide to Attribute Based Access Control Definition and Considerations.
W3C. PROV-O: The PROV Ontology.

The fifth document is less useful than the first refusal

Relevance is only the first probability

Authority belongs to a proposition, not a logo

Thought experiment: the perfect trespasser

Minimal worked example: the bridge status

A policy over observations

Decision value is not answer confidence

One-step value is a baseline, not the whole policy

Thought experiment: the second copy

Stopping is part of retrieval

The payment route, step by step

Negative control: the serious static baseline

Failure boundary

The retrieval decision card

Budgeted retrieval decision card

Build an evaluator that can prove restraint

Open hypothesis

Calibration through replay, not introspection alone

Failure injection before optimisation

Executable artefact: a minimal budgeted retriever

From prototype to operating control

Related reading

Compact glossary

Build the stopping rule before the search stack

Sources