Part one

Locate the missing state before changing the model

Suppose a configured language system produces an output y from a request x. Its behaviour depends on at least five distinct inputs: model parameters, instructions, retrieved evidence, deterministic results and external world state. A compact representation is useful because it exposes the intervention points.

The equation is not a complete theory of language models. It is an architecture diagnostic. It asks which variable is absent, wrong or too expensive to supply at runtime. If the required policy clause is missing from e, changing θ may cause the model to repeat a likely clause more confidently without providing the current source. If the arithmetic in c is wrong, adding more prose to i still leaves the model performing approximate token prediction. If s is unavailable, the model cannot observe whether a payment cleared.

Prompting became a practical adaptation method because sufficiently capable models can perform tasks from instructions and examples without gradient updates. GPT-3 made this property visible at scale, while later work showed that demonstrations may help through label space, input distribution and format even when their labels are perturbed. That is a useful warning: a prompt can elicit a task shape without proving that the model has learned the intended causal rule. See Brown et al. (2020) and Min et al. (2022).

Lewis et al. (2020) formalised retrieval-augmented generation as a combination of parametric and non-parametric memory, motivated in part by the difficulty of updating model knowledge and preserving provenance. The formulation does not make every retrieved answer supported, but it identifies an inference-time surface where external evidence can change independently of the weights.

Fine-tuning has a different role. LoRA showed that parameter-efficient updates can adapt large models with far fewer trainable parameters than full fine-tuning, and LIMA provided evidence that a small, carefully curated supervised set can strongly reshape response behaviour. Those results support fine-tuning as a behavioural intervention. They do not establish that weights are the best place for current operational facts. Controlled studies have found that new factual information can be learned more slowly than knowledge-consistent examples and may increase factual errors elsewhere. See Hu et al. (2021), Zhou et al. (2023) and Gekhman et al. (2024).

The mechanism decision field

A flat comparison table tempts teams to assign one winner. The real choice has at least three dimensions. First, where does the required state live? It may already be latent in the model, exist in supplied text, reside in a governed corpus, require deterministic execution, or exist only in a live system. Second, what assurance is required? A drafting aid can tolerate probabilistic phrasing; a monetary calculation needs reproducibility; an external write needs identity, authority and an effect receipt. Third, how often will the same mapping recur? High repetition can justify moving stable behaviour into weights.

The field makes hybrids unsurprising. A policy assistant can retrieve the current rule and still use a prompt to produce a concise explanation. A financial analyst can retrieve source figures, calculate ratios in a deterministic service, and use the model to connect the verified values into an argument. A case worker can read world state through a tool, yet any consequential write should pass through a typed action contract, policy check, explicit authority and postcondition verification.

Fine-tuning is an overlay, not a substitute for the other surfaces. RAFT is a useful example. It trains a model to work better with retrieved documents and distractors in a fixed domain. Its gains do not imply that the documents should disappear at inference. They show that the repeated skill of reading the open book can itself be adapted. The paper's own framing compares studying for an open-book examination with merely opening the book on test day. See Zhang et al. (2024).

Use invariance tests to identify the surface

The most useful diagnostic is not “Which mechanism improved the average score?” It is “Which controlled change should alter this answer, and which changes should leave it invariant?” A mechanism is causally matched when the system follows the variable that matters and ignores nearby variables that do not. This turns architecture selection into a small scientific experiment rather than a vote among familiar techniques.

For a suspected instruction deficit, keep the facts constant and vary wording, order, demonstrations and output schema. A well-designed prompt intervention should repair the target behaviour across paraphrases and held-out examples, not only the prompt used during development. If performance collapses when irrelevant formatting changes, the system has learned a brittle surface cue rather than the decision rule.

For an evidence deficit, keep the request and instruction fixed while changing only the authorised source snapshot. The answer should track the supplied evidence, expose the source identity and abstain when required passages are absent or contradictory. An answer that remains unchanged after the governing document changes is not grounded merely because retrieval ran. The model may be ignoring the passage, the retriever may have selected the wrong version, or the context compiler may have omitted the decisive clause.

For a computation deficit, freeze the semantic parse and compare language-model arithmetic with a reference function. If the reference function repairs the result for the same validated inputs, calculation is the matched intervention. If the model extracted the wrong principal, unit or rounding rule, the error remains upstream. Adding a calculator then makes the wrong interpretation exact, which is a more reproducible failure rather than a correct answer.

For a world-state deficit, repeat the request before and after a controlled external change. The answer should move with a case-bound read and preserve the observation time. A model that says an invoice is paid because similar invoices usually clear has not observed the invoice. A tool that returns “accepted” without later readback may still leave an unknown outcome. Tool availability therefore tests access; an effect receipt and postcondition test establish what happened.

A mapping deficit is the residual pattern left after instructions, evidence, computation and state access are adequate. Errors recur across a stable input-output relation, labelled examples represent the future distribution, and a paired fine-tune improves the locked set rather than only the training-like slice. That is the surface fine-tuning can change directly. If the task definition or source truth changes every week, the apparent mapping may instead be a moving policy or data problem.

These tests can expose coupled failures. Retrieval may supply the correct clause while the model maps it into the wrong schema; a fine-tuned parser plus retrieval can then be justified. A tool may supply the current balance while a deterministic function computes the covenant ratio. The discipline is to earn each added mechanism on the residual errors left by the previous matched intervention, rather than assemble a maximal stack before learning what was missing.

Part two

flowchart TD
  A["Five requests, one assistant,<br/>five different failures"] --> B["Locate the missing state<br/>before changing the model"]
  B --> C["The mechanism decision field"]
  C --> D["Run matched interventions<br/>instead of arguing from<br/>technique names"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#B96D00,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Run matched interventions instead of arguing from technique names

A reliable choice begins with a failure set, not an architecture preference. Hold the base model, evaluation cases and output contract constant. Change one causal feature at a time. The intervention that repairs the target failure tells you more than a benchmark averaged across unrelated tasks.

Thought experiment one: the policy changes at midnight

At 23:59, the travel policy allows economy travel for journeys under six hours. At 00:00, an authorised amendment changes the threshold to five hours. The user asks the same question at 00:01, with the same model, wording and traveller details. Only policy freshness has changed.

A better prompt may tell the model to “use the latest policy”, but the phrase does not supply the amendment. Fine-tuning can inject the new threshold after data preparation, training, evaluation and release, but that process creates a window during which the system is confidently obsolete. Retrieval can expose the current document and its effective time. A policy service can go further by returning the applicable rule after evaluating jurisdiction, traveller class and booking time.

When the decisive variable changes outside the model, the evidence path must cross that external boundary at inference or at a controlled synchronisation point. The model may still explain the result. It should not be treated as the authority that made the rule current.

Thought experiment two: the prose is right and the penny is wrong

Now hold the policy, model and prompt constant. Give the system the exact principal, rate and rounding rule. Ask for the amount. The model explains the formula correctly but produces £147.56 instead of £147.57. Add a longer chain-of-thought prompt and it may repair this case while breaking another. The missing feature is no longer semantic understanding. It is reliable execution under a specified arithmetic regime.

Program-aided language-model research separates these roles. PAL asks the model to translate a natural-language problem into a program and delegates execution to a Python runtime. Program of Thoughts uses a similar division and reported gains over language-only chain-of-thought across numerical tasks. These studies do not prove that generated programs are always correct. They show why a deterministic runtime can remove one class of arithmetic error while leaving interpretation and code generation to be evaluated separately. See Gao et al. (2022) and Chen et al. (2022).

from decimal import Decimal, ROUND_HALF_UP

PENCE = Decimal("0.01")

def percentage_amount(principal: str, rate_percent: str) -> Decimal:
    if not isinstance(principal, str) or not isinstance(rate_percent, str):
        raise TypeError("Use decimal strings, not binary floats")
    amount = Decimal(principal) * (Decimal(rate_percent) / Decimal("100"))
    return amount.quantize(PENCE, rounding=ROUND_HALF_UP)

# Positive case
assert percentage_amount("843.27", "17.5") == Decimal("147.57")
print(percentage_amount("843.27", "17.5"))

# Negative case
try:
    percentage_amount(843.27, "17.5")  # type: ignore[arg-type]
except TypeError as exc:
    print(f"rejected: {exc}")

Tool calls add state and authority, not certainty

A calculator is a narrow tool whose effect is local and reproducible. Operational tools are harder. Toolformer showed that models can learn when and how to invoke APIs, and API-Bank decomposed tool use into planning, retrieval and calling challenges. More recent robustness work has shown that function-calling performance can degrade under natural query variations or when semantically similar tools are added. The lesson is not to avoid tools. It is to separate the model proposal from the deterministic authority kernel that validates the call. See Schick et al. (2023), Li et al. (2023) and Rabinovich and Anaby-Tavor (2025).

A typed contract can describe parameter names, locations, required fields, schemas and expected responses. OpenAPI 3.1 and JSON Schema Draft 2020-12 provide standard building blocks for that syntactic boundary. They do not determine whether the user is authorised, whether the purpose is permitted, whether the data is fresh, or whether a returned HTTP success means the intended business effect occurred. Those checks belong around the tool.

The evidence contract can change the mechanism even when the prose is identical

Imagine two systems that both answer, “The five-hour threshold applies.” One recalls the sentence from its parameters. The other retrieves policy version 8, records the effective time and links the threshold to the traveller's booking facts. The visible prose is identical, yet the second answer supports a different class of decision. It can show which rule was used, whether the rule was current and what must be corrected if the source is withdrawn.

This is why output accuracy alone can under-specify adaptation. A brainstorming assistant may need only a useful suggestion. An audited case note may require claim-level provenance. A percentage displayed in an informal draft may tolerate a rough estimate; an amount posted to an account needs a validated function and receipt. A “booking submitted” message may describe a request accepted by an API, while the business obligation is to prove that exactly one confirmed booking now exists.

A mechanism is adequate only relative to an evidence contract. State what must survive the answer: source and version, validated inputs, function identity, observation time, actor and subject, authority decision, effect identifier, readback or human disposition. Then select the mechanism capable of producing those objects. This prevents a fluent parametric answer from competing as if it were equivalent to a sourced claim, a deterministic result or a verified effect.

The negative control: when doing less is the correct result

Architecture teams often evaluate only interventions they hope to justify. A negative control asks whether the prompt-only baseline already meets the requirement. Consider a stable twelve-label email-routing task. A carefully specified prompt with three boundary examples reaches 97.4 per cent macro-F1 on a locked, representative test set. The release threshold is 97 per cent. Errors concentrate in genuinely ambiguous messages that human reviewers also split.

Adding retrieval fetches product documentation that is topically related but irrelevant to the label definition. It increases token cost and occasionally shifts the decision toward whichever product name appears most often in the retrieved passages. A fine-tune lifts the development set and changes the wording of explanations, yet the locked set remains within statistical uncertainty of the prompt baseline. A tool call adds no missing state. The correct engineering decision is prompt-only with drift monitoring and an escalation rule for low-confidence cases.

An intervention that cannot beat the adequate baseline on the failure class it claims to repair is architecture debt. The point is not that prompting is universally superior. It is that complexity must earn its place through a discriminating test.

Part three

Choose the least costly adequate mechanism, not the lowest token price

“Cheapest” is easy to misuse. A prompt may be cheap to build and expensive to repeat. Retrieval may save training cost while introducing ingestion, entitlement, freshness and evaluation work. Fine-tuning may reduce prompt length or enable a smaller model, yet every new dataset, base-model version and behavioural regression adds change cost. A tool call may be inexpensive per request and expensive to govern because an incorrect write has asymmetric consequences.

The formula changes the fine-tuning discussion. At low volume, a longer prompt can be rational because there is no training pipeline to amortise. At high volume, stable behaviour and a representative labelled set may justify a tuned smaller model. If policy changes weekly, repeated fine-tuning can dominate change cost. If claims must cite the applicable source, retrieval keeps provenance available even when a fine-tuned model could recall the answer. If a wrong answer can move money, expected failure cost may dominate every compute saving.

Adequacy is a gate, not another cost term

Some requirements cannot be purchased away by a lower value of J. Let Q denote task quality, L latency, E evidential sufficiency and A authority correctness. A design is feasible only when it meets the required thresholds for all four. Cost optimisation begins inside that feasible set. A cheap answer with no applicable source is disqualified when provenance is mandatory. A fast write with no authority check is disqualified even if its observed error rate is low.

This ordering blocks false economic comparisons. Fine-tuning and retrieval are substitutable only for the portion of performance where either can meet the same evidence requirement. They are not substitutes when the decision must show a current, withdrawable source. A generated calculation and a decimal service are comparable only when both satisfy the required numeric accuracy, rounding policy and reproducibility. A model's statement that an action succeeded and a verified postcondition do not belong in the same feasible set.

Use hard disqualifiers before weighted scoring: missing authority, unavailable provenance, unbounded stale-state exposure, inability to recover an unknown outcome, or failure on a severe protected case. Then compare total cost among the survivors. The cheapest adequate mechanism is often a composition because each component discharges a different obligation. Composition is justified by separated obligations, not by a general belief that more machinery is safer.

A practical break-even test

Do not estimate fine-tuning value from prompt-token savings alone. Measure paired task quality first. Then include data preparation, labelling, training, model hosting, release tests, rollback, monitoring and revalidation. For retrieval, include source ingestion, chunking or indexing, authorisation filters, freshness, recall evaluation and claim support checks. For tools, include contract design, sandboxing, identity propagation, idempotency, timeout handling, unknown outcomes and readback.

A tuned model deserves a production trial when four conditions hold together: the target mapping is stable; the prompt baseline misses a material threshold; representative labelled examples exist; and the expected volume or local deployment constraint can amortise the release lifecycle. Even then, keep facts, exact computations and external effects outside the weights when their provenance or state matters.

Part four

A production-shaped case: covenant monitoring without mechanism collapse

Consider a synthetic assistant that helps an analyst review an SME loan covenant. The assistant may prepare evidence and a draft recommendation. It cannot change the official risk grade, waive a covenant or contact the customer. The request sounds like one question: “Has Northstar Components breached its leverage covenant, and what should we do next?” In reality, the answer crosses every control surface.

Intent is to prepare an analyst review, not make a credit decision. Identity binds the analyst and customer case. World state includes the latest approved financials, drawn balance and facility status. Context contains the executed agreement, amendments and current policy. Reasoning connects definitions to evidence. Authority permits reading and drafting but not waiving or updating. Action is a draft case note. Evidence links every material claim to a source or calculation receipt. Outcome is the analyst's accepted, changed or rejected disposition. Release and recovery cover versioning, rollback and correction if a source or calculation proves wrong.

Prompting defines the analysis structure: identify the covenant, locate numerator and denominator, list exclusions, distinguish observed facts from assumptions, and abstain when evidence conflicts. Retrieval obtains the signed agreement and applicable amendments with page references and effective dates. A ledger tool reads the current drawn amount under case-bound identity. A deterministic calculation service computes the ratio using the agreement's definition. The model drafts an explanation from those verified objects. A human analyst retains decision authority.

Where might fine-tuning enter? Suppose the base model repeatedly fails to map idiosyncratic covenant language into a stable typed intermediate representation even after prompt examples and high-quality retrieval. The organisation has several thousand adjudicated clause-to-schema examples, a locked cross-document test set and sufficient volume. A fine-tune could improve clause parsing. It would still consume retrieved agreements at inference because the exact clause, amendment and evidence must remain inspectable.

The mechanism map also changes evaluation. Retrieval is tested for source recall, entitlement, freshness and contradiction handling. Calculation is tested against reference functions and boundary values. Tool use is tested for selection, argument validity, identity, timeouts and readback. The model is tested for claim support, clause mapping, abstention and omission of adverse evidence. Human review is measured by changes, disagreements and severe misses rather than by a vague “helpfulness” score.

This article is narrower than the companion discussion of trajectory assurance for agentic systems. The present question is where to place a capability change. Once tools and consequential workflows exist, the companion question becomes whether observable evidence, policy, approval and effect events support the route taken.

Two other adjacent articles begin after this placement decision. Shipping a Private Skill Model Foundry addresses the data, lineage, calibration and release system required when a fine-tuning candidate survives the paired test. Deterministic-First Agents addresses how deterministic controls and bounded model reasoning should be arranged once calculation or tool use enters a consequential workflow. Neither makes the upstream causal choice developed here.

Part five

stateDiagram-v2
  [*] --> Claim
  state "Prompt" as Claim
  state "Retrieve" as Test
  state "Calculate" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

The adaptation worksheet

The worksheet turns the argument into an executable design test. It does not score vendors or predict benchmark performance. It asks which state must cross the model boundary and whether a stable repeated mapping merits a fine-tuning experiment. Its output is a mechanism composition plus release evidence.

Read the output as an experiment plan

“Retrieve” means build a source path and test retrieval recall, freshness and claim support. “Calculate” means write or reuse a deterministic function, validate its inputs and preserve a receipt. “Tool” means define schemas, identity, authority, timeouts, idempotency, errors and readback. “Fine-tune candidate” means run a paired experiment against the prompt baseline and include change lifecycle costs. It is not permission to train on whatever documents happen to be available.

The thresholds inside the worksheet are deliberately conservative and transparent. Five hundred labelled examples and ten thousand monthly tasks are heuristics for demonstrating the logic, not universal industry cut-offs. Change them for local economics, model scale, data quality and risk. The invariant is the sequence: establish an adequate evaluation, locate the deficit, intervene on the matching surface, then compare total evidence-adjusted cost.

function recommendMechanisms(s) {
  const layers = ["Prompt"];
  const controls = [];
  const gap = s.targetScore - s.baselineScore;

  if (s.changingFacts || s.evidenceRequired) {
    layers.push("Retrieve");
    controls.push("source identity, entitlement, freshness, claim support");
  }
  if (s.exactComputation) {
    layers.push("Calculate");
    controls.push("typed inputs, reference function, calculation receipt");
  }
  if (s.liveState || s.sideEffect) {
    layers.push("Tool");
    controls.push("identity, authority, schema, timeout, receipt and readback");
  }

  const fineTuneCandidate =
    s.stableMapping &&
    gap > 0 &&
    s.labelledExamples >= 500 &&
    s.monthlyVolume >= 10000;

  if (fineTuneCandidate) {
    layers.push("Fine-tune candidate");
    controls.push("locked paired evaluation, regression suite, rollback");
  }

  if (gap <= 0 && layers.length === 1) {
    controls.push("retain prompt-only baseline; monitor drift");
  }

  return { layers: [...new Set(layers)], controls };
}

Compact glossary

Model proposal

A probabilistic suggestion, classification, draft or tool invocation that has not yet acquired external authority.

World state

Authoritative current facts outside the model, such as balances, inventory, case status or effective policy.

Evidence

A source or receipt that supports a material claim and can be inspected independently of the generated prose.

Typed action contract

A machine-checkable operation schema plus business preconditions, identity, purpose, authority and expected effects.

Readback

An independent observation after a tool call that establishes which state actually resulted.

Adequate baseline

The simplest mechanism that meets the agreed quality, latency, evidence and authority threshold on representative cases.

Put the change where the missing state lives

The original five requests no longer look like one adaptation problem. Tone starts with a prompt because the capability is present and the desired behaviour can be specified locally. Current policy uses retrieval because freshness and source identity matter. The percentage uses deterministic calculation because exact execution matters. Invoice status uses a read-only tool because the answer exists in live world state. The high-volume router begins with a prompt baseline and earns a fine-tuning experiment only if the residual mapping gap and economics justify a new model release.

The changed architecture decision is to fund interventions by failure class rather than by technique enthusiasm. Ask for a locked set of cases, a causal diagnosis, a matched intervention and the evidence that will survive operation. Reject designs that use weights as a policy database, generated prose as a calculator, retrieval as authorisation, or a successful API response as proof of a business outcome.

The strongest countercase remains a fixed domain with abundant labels, high repetition, constrained deployment and a supervised fine-tune that reliably outperforms runtime augmentation. Use it. Keep the boundary honest. Stable behavioural skill may move into weights; changing facts, exact functions, live state and consequential authority should remain in components that can be updated, tested, observed and withdrawn on their own terms.

That is the durable rule: choose the smallest adequate place for the change, then preserve the evidence needed to know that it worked.