Autonomy is a budget, not a setting

Every institution that deploys agentic AI eventually asks the same question, usually after an incident: how much should the machine do on its own? The common answers all fail the same way. "Full autonomy for low-risk tasks, human review for high-risk ones" sounds sensible until someone has to define, in advance, what makes a task low-risk. "The agent proposes, the human disposes" sounds safe until the human is disposing of four hundred proposals a day and has stopped reading them by Tuesday. "Set a confidence threshold" sounds rigorous until you learn that a model's stated confidence is a stylistic habit, and that the model is often most fluent when it is most wrong.

These answers share one mistake. They treat autonomy as a static property of the task, something you configure once in a policy document and revisit at the next annual review. In a real multi-step workflow, autonomy is consumed and replenished continuously. An agent that begins an investigation with good reason to act alone can lose that warrant halfway through, the moment it meets an entity it has never seen or a document that contradicts another. The reverse also happens. An agent that started under tight supervision can earn the standing to proceed alone as its evidence solidifies and its uncertainty collapses.

The organising idea of this essay: represent autonomy as a budget inside a constrained decision process. The state of that process, the thing the governing policy reads before every action, carries at least seven quantities. Four are assessments: the consequence of the contemplated action, its reversibility, the novelty of the situation relative to what the agent has been evaluated on, and the agent's calibrated confidence. Three are depleting resources: the remaining action budget, replan budget and cost budget.

The question at every step is no longer only "what should I do next?" It becomes: given what this step could cost, whether it can be undone, how strange the territory is, how sure I am, and how much runway is left, should I act, verify, replan, or hand this to a human?

Framed this way, escalation to a human stops being a failure mode. It becomes an ordinary action with a price and a payoff, chosen by the same policy that chooses everything else. Human oversight stops being a checkpoint bolted to the end of a pipeline. It becomes a resource the agent buys when the value of certainty exceeds the cost of interruption. Governance stops being a document and becomes a mechanism: hard constraints and depleting budgets that the runtime enforces on every trajectory, that auditors can replay, and that a learning system can be trained against.

The idea synthesises several converging strands of research from 2023 to 2026. One strand studies verbal reinforcement, where agents improve by reflecting on failures in plain language and carrying those reflections forward as memory. Another studies risk over long horizons, and shows that safety judged one action at a time misses harms that build across a sequence. A third builds verifiable shields that compile policy documents into rule circuits enforced outside the model. A fourth calibrates verbal uncertainty so that "fairly sure" corresponds to an actual error rate.

A fifth trains multi-turn, tool-using agents by reinforcement learning, and finds that the choice to search, verify or stop can itself be trained rather than hard-coded. Section 6 works through this literature paper by paper. The sections after it apply the framework to ten problem classes that cover most of what a large financial institution wants agents to do.

A note on scope. Everything here concerns enterprise agents inside a regulated institution: software that reads systems of record, assembles evidence, drafts analyses, and sometimes acts on accounts, cases and workflows. Nothing here concerns consumer chatbots. Regulatory themes appear as design pressure, not as legal advice for any jurisdiction.

Why static autonomy fails in regulated finance

It helps to be precise about what breaks without the budget view. The failure modes of static autonomy follow from three structural facts about financial-services work, and each shows up in incident reviews within months of a naive deployment.

2.1 The risk of a workflow is not the risk of its steps

Financial workflows are long and compositional. An investigation into a suspicious counterparty is forty or fifty retrievals, extractions, cross-checks and judgements, each conditioning the next. A per-step safety review can pass every step of a trajectory that is, in aggregate, unacceptable. The research on long-horizon risk makes the point directly: reactive, single-shot safety filters miss risks that accrue, because no individual step trips the wire. An agent pulling a customer's transaction history is doing something routine.

An agent that pulls transaction history, then location history, then counterparty identities, and then correlates them against an unrelated case has assembled something no single step disclosed. Static rules are written per action type ("agents may query the transaction store") and are structurally blind to accumulation. A budget is not. Every step spends, and a trajectory that wanders spends itself into a mandatory pause.

2.2 The distribution shifts underneath the evaluation

Novelty is endemic in this industry. Financial-crime typologies mutate specifically to evade whatever was last codified. Manipulation patterns migrate across venues. Document formats change whenever an originator updates a template. The evaluation battery your agent passed in March describes March's world. A static grant of the form "the agent scored 96% on the evaluation set, so it may act unsupervised" quietly converts an in-distribution performance figure into an out-of-distribution permission. The honest version is conditional: the agent may act unsupervised while it remains inside the territory it was evaluated on, with autonomy that degrades smoothly as it leaves that territory. That requires novelty to be measured at runtime and fed into the autonomy decision, which is exactly what a state dimension is for.

2.3 Human review is scarce, and static rules waste it

The third fact is economic. Every serious deployment in a bank has humans in the loop somewhere. Regulation demands it, and so does prudence. The real question is never whether humans review but which outputs consume the hours of the scarcest people. Static routing gives you the worst of both directions. Route too much and reviewers saturate: the queue grows, attention per item collapses, and approval degenerates into a rubber stamp that provides legal comfort and no control. Route too little and the one trajectory that needed a human sails through. The only routing rule that is both safe and affordable concentrates review where consequence, irreversibility, novelty and residual uncertainty are jointly highest. That combination changes step by step. No static rule can track it.

2.4 Regulation points the same way

The direction of regulatory travel is convergent with the budget view, even though no regulator uses the vocabulary. Model-risk guidance asks institutions to know the conditions under which a model's outputs can be relied on. That is a conditional statement. Consumer-protection regimes ask firms to evidence good outcomes, which are properties of whole customer journeys rather than of individual automated steps. Market-abuse and AML regimes require case-by-case explanations of why an alert was closed or escalated, and the explanation must reference the specific evidence trail.

Accountability regimes make named humans answerable for automated decisions, which only works if the human can see which decisions carried risk and show that those received attention. Each of these is a demand for instance-level, trajectory-aware, evidence-linked control. Static tiers cannot supply it. A constrained decision process, with per-step state and a replayable ledger, is close to the minimal structure that can.

The framework: a seven-dimensional decision state

The heart of the framework is deliberately small: a state vector the governing policy consults before every step, and a short repertoire of actions it can select in response. The difficulty lives in how the dimensions are measured, how the budgets are set, and how the policy over them is learned and assured. The skeleton fits in a diagram.

3.1 The action repertoire

Against this state the agent selects from a small, closed set. Act executes the next substantive step (a retrieval, an extraction, a draft, a case note, a workflow transition) and spends one unit of action budget plus its metered cost. Verify spends cost to buy confidence: re-retrieving from an independent source, cross-checking an extraction against the original document, running a second model as a checker. Replan abandons the current strategy and builds a new one, spending a unit of the deliberately scarce replan budget.

Unlimited replanning is how agents loop and thrash. Escalate packages the current state, with evidence, provenance, the budget ledger and a specific question, and hands it to a human at a price that reflects real reviewer scarcity. Abstain terminates safely. The agent records that the task could not be completed within constraints and why. In a regulated setting that record is itself a useful, auditable outcome.

Two properties deserve emphasis. First, verification is a decision, not a reflex. The retrieval literature is clear that agents can search too little and too much: skipping checks produces hallucination, while compulsive checking burns budget and buries the relevant evidence under marginal context, which measurably hurts accuracy. Verification should happen where its expected information gain justifies its cost. A budget-aware policy can learn that trade-off. Second, escalation carries information along with the work.

A good escalation is not "here, you finish it." It is a structured object holding the unresolved question, the evidence on both sides, the checks already run, and the budget state that triggered the handover. The human's decision is captured and becomes training signal. Done this way, escalation is the highest-bandwidth channel through which institutional judgement enters the learning loop.

3.2 Why three budgets

The budgets fail differently and are owned differently, so they stay separate. The action budget bounds trajectory length. Exhaustion signals a task larger than expected or an agent that is wandering; either way a human should look. Set it per task class from the observed distribution of trajectory lengths on evaluated cases. If successful trajectories historically take 20 to 35 steps, a budget of 50 puts the forced pause outside normal variation and inside pathology.

The replan budget bounds strategic churn. A replan is qualitatively different from a step: it discards sunk work, and it is the moment when success criteria can be quietly renegotiated. Multi-turn training research documents agents that loop through superficially different strategies or satisfy the letter of a goal while abandoning its substance. A hard cap of two or three strategy changes per episode, with exhaustion forcing escalation, is a cheap and effective control.

The cost budget denominates everything in one currency: model calls, tool invocations, retrieval volume, and human review minutes. A common unit is what lets the policy weigh "one more verification pass" against "escalate now." Making human attention a metered line item rather than a free exception path is the single most consequential choice in the framework. It forces the system to husband reviewer capacity for the cases that need it.

3.3 The autonomy frontier

The policy's behaviour is easiest to picture as a frontier in the state space. In the benign corner (low consequence, high reversibility, low novelty, high confidence, full budgets) the agent acts freely. As any dimension deteriorates, the policy first substitutes cheaper caution, then escalates, then abstains. The boundaries are not independent thresholds. They form a joint surface where the expected cost of proceeding alone crosses the expected cost of interruption, and the surface bends.

High reversibility buys tolerance for low confidence: you can afford to be wrong about things you can undo. High novelty poisons high confidence, because calibration is itself an in-distribution property; a well-built agent should distrust its own calibration in strange territory. A nearly exhausted replan budget makes even a confident act less attractive, since there is no runway left to recover from surprise.

The frontier picture clarifies what "raising autonomy" means in practice. It does not mean flipping a task from review to no-review. It means expanding the ACT region: demonstrating with evaluation evidence that, within a defined envelope of consequence, novelty and confidence, the agent's unsupervised error rate is acceptable, then moving the boundary. Each expansion is a measured, reversible, per-region policy change. That is how an institution should want to make such changes.

3.4 How the dimensions relate

The dimensions are not independent gauges. They form a small causal web, and the policy's quality depends on respecting its arrows. The diagram below makes the web explicit: which constructs feed which, where the discounts run, and where the loops close.

Formal treatment: the constrained decision process

The prose picture can be made precise with light machinery. Readers who skip this section lose rigour but not plot. The formalism earns its keep in three places: it explains why escalation pricing works, it connects budget exhaustion to standard constrained optimisation, and it states what the learning system actually optimises.

4.1 States, actions, constraints

Model each episode as a constrained Markov decision process. The state pairs the task context with the seven-dimensional autonomy vector:

Three modelling choices matter. First, the shield constraint has no probability attached. Forbidden transitions, the never-events (committing a customer-affecting change without a mandate, disclosing restricted data across a case boundary, executing an external action while a hold applies), are removed from the feasible set by a rule layer outside the learned policy. The policy never weighs a never-event against reward, because a badly specified reward will eventually find any harm that is merely expensive rather than impossible. Second, escalate is always feasible. No budget exhaustion, no shield and no state can remove it. There is no state from which the safe exit is closed. Third, harm appears both as a penalty and in the constraint set: the constraint handles the intolerable, the penalty shapes behaviour among the tolerable.

4.2 Escalation pricing

Why does pricing escalation produce sensible behaviour instead of an agent that never asks for help? At a decision point the policy can act with success probability p (its calibrated confidence, discounted for novelty) or escalate at cost κ. Acting risks an error with expected cost (1−p)·D, where D bundles consequence-weighted damage and the cost of repair given reversibility. Escalation buys near-certain correctness on this step at price κ plus latency ℓ. The policy prefers escalation exactly when:

Familiar governance intuitions fall out of this inequality. High-consequence, hard-to-reverse steps (large D) justify escalation even at high confidence. Trivial, reversible steps never justify it, which protects reviewers from noise. Poor calibration corrupts p, and with it every escalation decision, which is why §5 treats calibration as load-bearing infrastructure. Reviewer capacity becomes a pricing problem: if the queue saturates, κ was set too low relative to true scarcity, and raising it shifts marginal cases from escalation to verification or abstention. Institutions already reason about scarce resources through prices. The framework extends that reasoning to attention.

4.3 Budgets have shadow prices

Formally the budgets are constraints, and every constraint has a shadow price: the marginal value of one more unit. This is a monitoring instrument, not a theoretical nicety. If the shadow price of the action budget for a task class sits near zero, the budget never binds and can be tightened at no cost. If it stays high, the budget is strangling legitimate work. Either raise it or, better, ask why trajectories are long; the answer is often upstream, in a data source whose poor quality forces repeated verification. Watching shadow prices per task class turns budget-setting from an annual argument into a continuous control loop, and gives the platform team a shared language with finance. The cost of caution becomes a number, per class, per month.

4.4 Where the numbers come from

The formalism is only as good as its inputs, so it is worth stating plainly how each is obtained. Consequence and reversibility are mostly engineered rather than learned: the tool layer carries a taxonomy (read, draft, write, commit, external-effect) with per-tool metadata about compensating transactions, and case context supplies exposure and harm classifications from existing risk data.

Novelty is learned without labels: density or distance models over the embedded evaluation corpus, plus discrete tripwires such as an entity never seen, a schema that fails to parse, or a tool response outside its historical range. Confidence comes from a calibration layer, described in §5.4, validated per segment and revalidated on drift. Budgets come from evaluated trajectory distributions, adjusted through shadow prices. The policy itself starts as a conservative hand-written rulebook. Every institution's first version should be embarrassingly cautious.

The learning apparatus of §6 then refines it inside the envelope the shield and budgets define. The rulebook-first path matters because it delivers governance value from day one, before any reinforcement learning is trusted with anything.

The dimensions in depth

Each dimension hides enough subtlety to sink an implementation that treats it casually. This section takes them one at a time: what each measures, how it is estimated, how it interacts with the others, and how it typically goes wrong.

5.1 Consequence: severity belongs to the instance

Consequence answers a blunt question: if this step is wrong, how bad is the worst plausible outcome? The critical discipline is scoring it per instance and per step, never per task type. "Draft a case summary" sounds uniformly low-consequence until the summary in question will anchor a regulatory disclosure. "Update a customer record" sounds routine until the field being updated gates a payment hold.

Practical scoring composes ingredients the institution already has: monetary exposure on the case, the class of customer harm reachable from the step, regulatory reportability of a wrong outcome, and the sensitivity tier of any data touched. Compose them by maximum, not by sum. One catastrophic ingredient should never be diluted by many benign ones. Express the result in the currency the firm already uses for operational-risk events, so the autonomy system's judgements line up with the risk function's existing taxonomy.

The characteristic failure is consequence laundering through decomposition: a severe outcome reached through many individually mild steps, none of which trips a threshold. The defence is twofold. Score contemplated actions against the terminal uses of their outputs where those are knowable; a retrieval feeding a disclosure inherits some of the disclosure's weight. And let the trajectory-level shield reason about accumulation where they are not.

5.2 Reversibility: the cheapest safety property you can buy

Reversibility asks what it costs to undo the action, and whether undoing is possible at all. It is the most engineerable dimension, because it is determined mostly by how the tool layer is built. A tool layer designed for budgeted autonomy makes reversibility machine-readable: every tool declares its class on the ladder read → draft → write-internal → commit → external-effect, and every write-class tool declares its compensating transaction if one exists. The design goal is to push as much of each workflow as possible down the ladder. An agent that operates for forty steps in read-and-draft space and touches commit-space once, at the end, under the highest scrutiny, is categorically safer than one that interleaves commits throughout. Database engineers batch their writes for the same reason.

Two subtleties. Disclosure is irreversible even when data is not: a record can be corrected, but a fact revealed to the wrong audience cannot be unrevealed, so information flows to external parties or across confidentiality boundaries belong at the top of the ladder however read-only they look. And latency of reversal matters as much as possibility. A payment recallable within two hours is reversible only if monitoring would notice the error within two hours. Honest reversibility scores are conditional on the detection horizon of the surrounding controls, which quietly imports the quality of the observability stack into the autonomy calculus. That is where it belongs.

Reversibility also enables the single most useful behavioural pattern in the framework: the action downgrade. When the contemplated action is impermissible in the current state, the policy should look for a lower-impact substitute before failing the task. Instead of sending, draft. Instead of changing, stage. Instead of closing the case, recommend closure. Instead of committing to one identity, present the ambiguity set. The system degrades gracefully instead of jumping from full autonomy to refusal, and it keeps delivering value inside whatever envelope the state permits.

5.3 Novelty: the licence has a territory

Novelty measures how far the current instance sits from the territory in which the agent's competence was demonstrated. The reference corpus matters. Novelty must be measured against the evaluation distribution, the cases where error rates were actually established, and not against the training distribution, which is broader, unlabelled, and says nothing about demonstrated competence. An agent can have seen a pattern in pre-training and still never have been tested on it. For governance, the second fact is the one that counts.

Effective detection layers the statistical on the symbolic. The statistical layer embeds the current case and measures distance or density against the embedded evaluation corpus. The symbolic layer is a set of discrete tripwires needing no model at all: an entity type never encountered, a document whose schema fails to parse, a tool returning values outside its historical range, a jurisdiction or product absent from the evaluation set. The symbolic layer exists because embedding distance is a smooth and fallible signal, while some novelties, a new sanctions regime for instance, are discretely and knowably new.

The deepest interaction in the framework runs between novelty and confidence: novelty discounts confidence's warrant. Calibration is established on a distribution. Outside it, the calibration curve loses its evidentiary basis. A well-built policy treats high-novelty-high-confidence as a more alarming state than high-novelty-low-confidence. The latter is an agent that knows it is lost. The former is an agent that does not, which is the precondition for every confidently wrong action that ever reached a customer.

5.4 Confidence: calibration is infrastructure

The research finding that anchors this dimension is uncomfortable. A language model's verbalised confidence, the "I am fairly certain" it emits, is only loosely coupled to its internal uncertainty. Work on the linear-feature structure of verbal uncertainty shows the two are partly independent quantities. A model can be internally uncertain and verbally fluent at the same time, and that dissociation is the signature of confident hallucination. The practical consequence: no autonomy decision should ever consume raw verbalised confidence. The confidence feature must come from a calibration layer built and validated like any other model-risk artefact.

The working toolkit is standard by now. Self-consistency samples the reasoning several times and reads divergence as uncertainty; it is expensive, informative, and itself a budgeted verify-action. Checker separation has a second model score the first's claim against the evidence. Retrieval support measures how directly cited evidence entails a claim, a mechanical and auditable score. Post-hoc calibration fits the mapping from raw signals to realised correctness on labelled outcomes, per task segment, and refits on drift. Per segment matters. An agent can be well calibrated on one document type and badly calibrated on another, and one global curve will hide exactly the segment where it is dangerous.

5.5 The action budget

The action budget bounds episode length, and its exhaustion is designed to be informative rather than punitive. Set it from the observed distribution of successful trajectory lengths per task class, commonly the 95th percentile plus a margin, so that hitting the cap is by construction a statistically unusual event worth a human glance. The forced pause at exhaustion is not an error state. The agent packages its progress, its evidence and its view of what remains, and escalates with the ledger attached. The open-ended risk of a wandering agent becomes a bounded, legible artefact: the worst a runaway trajectory can cost is the budget, and it announces itself.

The usual mis-setting is generosity. A budget that never binds is a comment, not a control. If fewer than one episode in two hundred touches the cap, tighten it. The shadow-price telemetry of §4.3 makes the adjustment mechanical.

5.6 The replan budget

A replan discards the current plan and constructs a new one. It is where agents are most creative and most dangerous, because replanning is the moment success criteria can be quietly renegotiated. The multi-turn training literature documents the failure vividly: under outcome-only reward, agents loop through superficially different strategies, produce degenerate reasoning that games the check, or "succeed" by solving an easier task than the one assigned. A hard cap of two or three strategy changes per episode, with exhaustion forcing escalation, is a blunt instrument.

Bluntness is its virtue. Blunt rules are legible, and legibility is what audit requires. Each replan should also be logged as a structured object stating why the old plan failed, what the new plan changes, and what would falsify it. That discipline comes from the verbal-reinforcement literature, where articulated self-reflection is the mechanism by which agents improve across episodes. The replan log is a control artefact and training data at once.

5.7 The cost budget

The cost budget denominates everything the episode consumes in one currency: model tokens, tool calls, retrieval volume, third-party data fees, and reviewer minutes at loaded cost. Its purpose is commensurability more than cost control. One scale lets the policy weigh another verification pass against escalating now against abstaining. Pricing human review honestly is the keystone. Priced at zero, escalation dominates every close call and the queue saturates. Priced honestly, escalation is reserved for cases where human judgement earns its cost. There is a management dividend too: aggregate escalation spend becomes a legible number. This quarter, this portfolio consumed 340 reviewer-hours of escalations, concentrated in these two case types. That is the sentence an operating chief needs when deciding where to invest in better data or better evaluation next.

5.8 Auxiliary state: four variables worth adding in production

The seven core dimensions are the spine, and production systems benefit from four auxiliary variables that make the state richer without changing its logic. Evidence coverage tracks whether the factual questions the task requires have been answered: checklist completion, missing fields, unresolved contradictions. It gates consequential actions; a disposition with open contradictions should not be reachable. Provenance quality weights confidence by the trustworthiness and traceability of its sources: source class, recency, ownership, chain of custody.

High confidence derived from poor evidence is not high-quality certainty, and the state should be able to tell the difference. Policy sensitivity tags which rules, obligations and protected classes the current step implicates: jurisdiction, data class, vulnerable-person indicators. It routes hard constraints. Time criticality records how fast decision value decays: deadlines, market clocks, active threats. It reshapes search and escalation policy, and §14 shows it acting as a fourth depleting budget. None of these are decorative.

The complexity they carry already exists in the business; the state vector simply makes it explicit, testable and governable.

5.9 A budget ledger over one hard episode

5.10 The escalation ladder

Escalation itself is graded. A mature deployment distinguishes at least five rungs, each buying a different quantity of human attention at a different price, and the policy chooses a rung the way it chooses everything else: by expected value.

Rung design has a second-order effect that is easy to miss: it shapes the quality of the training signal. A rung-2 approval labels a finished artefact. A rung-3 consultation produces something richer, a human answer to a well-posed question at the exact point of maximal uncertainty, which is close to the ideal supervised datum for teaching the policy where its frontier should sit. Deployments that design rung-3 escalations to be answerable in one screen (question, evidence for, evidence against, what the agent would do and why) harvest better signal and better reviewer goodwill than those that dump transcripts.

One episode, end to end

A single worked trace, deliberately ordinary, makes the machinery concrete. The setting is an investigation-shaped task of the kind Part Two generalises: an alert about a counterparty, an agent assigned to assemble the picture, a human analyst at the end of the wire. Follow the dimensions and budgets through it.

Step 0, allocation. The episode opens with grants drawn from the registry for this task class: an action budget of 40, a replan budget of 2, a cost budget in the platform's internal unit, an escalation price reflecting current reviewer load, and a frontier parameterisation certified four weeks ago. The novelty detector scores the case on arrival. The entities sit comfortably inside the evaluated corpus; one attached document has a schema the classifier has seen rarely. Novelty: moderate, with a document-level flag. Nothing here is exceptional. Allocation is a lookup, not a ceremony.

Steps 1 to 9, cheap breadth. The agent spends its first actions on low-cost, fully reversible reads: case history, account profile, the transaction window around the alert, the standing knowledge objects for this alert typology. Each retrieval arrives with provenance and an authority tier. Confidence rises unevenly. The transaction picture is clean and well supported. The counterparty's stated business purpose rests on a single stale document, the odd schema from step 0, and the state assessor marks every claim derived from it with depressed confidence. No budget pressure yet.

Step 10, the fork. A cross-check surfaces a conflict: the business purpose implied by transaction behaviour does not match the documented one. Conflict is information. It raises the consequence score of any eventual disposition, since both directions now carry regulatory weight, and it triggers the hypothesis-pair discipline. The agent records two explanations, benign drift of a business model since documentation versus deliberate misrepresentation, along with the evidence each one predicts.

Steps 11 to 17, bought confidence. The policy now faces the verify-or-act comparison repeatedly, and the ledger shows it buying confidence exactly where the hypotheses disagree: a re-read of the stale document through a second extraction path, scored separately because image-derived and text-derived readings calibrate differently; a registry pull to date the business-model change; a peer comparison to establish whether the behaviour is sector-normal. The cost budget drops visibly here. Each purchase is logged against the hypothesis it was meant to discriminate. Two of three checks favour the benign explanation. The third is inconclusive.

Step 18, the replan. The inconclusive check exposes a dead end: the planned third-party source does not cover this entity category. The agent spends one of its two replans, logging the failed theory and the new route, triangulation through an adjacent covered entity. The log entry is four sentences a human can read. One replan remains. The frontier, recomputed, has narrowed slightly. Same evidence, less runway.

Steps 19 to 26, convergence with a residue. The triangulated route works. The benign explanation now carries strong support on every predicted point but one: a residual payment pattern neither hypothesis cleanly explains. Calibrated confidence in the benign disposition is high but not extreme, and the residue is flagged rather than smoothed. This is the framework doing its most characteristic work. The agent's fluent narrative could easily absorb the residue. Claim-level support scoring refuses to let it.

Step 27, the escalation decision. The terminal action would be a disposition with regulatory significance. The state at this moment: consequence high, reversibility low, novelty moderate with one unresolved flag, confidence high but not extreme with a named residue, budgets at roughly a third. The inequality of §4.2 lands on the escalate side. Not because any single dimension screams, but because the product of consequence and residual uncertainty exceeds the priced cost of a rung-2 review. The escalation object is assembled: recommended disposition, both hypotheses with their evidence, each item resolvable to source, the residue stated plainly, the checks performed, the ledger. One screen.

Step 28, the human. The analyst reads for six minutes. The queue is healthy, because pricing has kept it so. She agrees with the disposition and annotates the residue with a piece of institutional context no system held: the pattern matches a documented quirk of the sector's invoicing cycle. The annotation is captured, becomes a knowledge-object proposal (agent-drafted, owner-approved), and joins the training corpus, where it will quietly delete this escalation from the future. The next hundred cases with this residue will resolve it from the knowledge object at step 19, at reading cost, autonomously.

That last movement is the whole framework in miniature. The episode consumed 28 of 40 actions, one of two replans, most of a cost budget, and six minutes of a scarce human. Its by-product was a durable improvement to the institution's knowledge and to the policy's future frontier, produced not by a training run but by the ordinary operation of a system whose every component leaves a record. Multiply by ten thousand episodes a month. The compounding, not the automation, is the business case.

What the research says

The autonomy budget is a synthesis, and its credibility rests on the strands it synthesises: agent safety evaluation, risk-aware alignment, reinforcement learning for tool-using agents, and uncertainty-aware retrieval. One question runs through this section. What does each strand contribute to, or demand from, a budgeted design? Full citations sit in §21; papers appear here by arXiv identifier.

6.1 Safety evaluation: the licence must come from somewhere

The benchmark literature (2401.10019; 2412.14470) established that agent risk-awareness must be probed interactively. An agent's stated knowledge of what is risky diverges sharply from its behaviour when it actually meets a risky affordance mid-task, so questionnaire-style evaluation overstates safety. OS-Harm (2506.14866) extended this to computer-use agents and produced a finding that matters for enterprise deployment: harm arrives through three distinct routes. Deliberate user misuse. Prompt injection from the environment. Plain model misbehaviour. An evaluation regime that tests one route certifies nothing about the others, and the routes have different mitigations: policy for the first, input hardening and shields for the second, calibration and review for the third. Any institution granting autonomy must evaluate all three per task class.

AgentAuditor (2506.00641) contributes something constructive: an LLM evaluator, equipped with a structured memory of adjudicated cases and a reasoning procedure over them, can match human experts in judging whether a trajectory was safe. This changes the economics of the whole design. Trajectory adjudication at machine cost means the asynchronous review rung of the escalation ladder can be partly automated, with an auditor model triaging sampled trajectories and humans handling the ambiguous ones. The evaluated envelope that novelty is measured against is only as good as the evaluation that defines it, and this strand supplies the machinery to keep that evaluation continuous rather than annual.

6.2 Risk over long horizons: harm accrues along trajectories

The long-horizon simulation work (2506.20949) is the clearest statement of why per-step safety fails. Aligning a model against immediate harm leaves it blind to risks that materialise across a sequence of individually innocuous actions. Alignment must be trained and evaluated against simulated futures, rolling trajectories forward to see where they land. In budget terms, this is why budgets deplete over trajectories and why the shield evaluates sequences rather than steps.

Safety-RL alignment (2507.08270) addresses the complementary problem: an agent must be neither harmful nor useless, and both failures are training artefacts. Its central device treats risk awareness as a distinct, trainable capability that mediates between capability and compliance, so the agent learns to distinguish benign from malicious requests instead of refusing both. That maps directly onto the autonomy state. Consequence and novelty are the mediating variables that let a policy be aggressive where it is safe and conservative where it is not. Over-refusal, it should be said plainly, is not the safe default in a bank. An agent that abstains on everything pushes the work back onto humans, recreates the throughput crisis that motivated agents in the first place, and manufactures an illusion of safety while doing so.

ShieldAgent (2503.22738) supplies the enforcement architecture: safety policies extracted from written documents into a structured, verifiable rule model, checked at runtime against each contemplated action, with formal guarantees that permitted actions satisfy the encoded rules. Its key property is independence. The rules live outside the learned policy, so no amount of reward hacking or prompt injection can argue the shield out of the way. SafeWatch (2412.06878), though built for video guardrails, contributes the transparency requirement: guardrail verdicts need explanations attached to specific evidence. An unexplained "blocked" is operationally indistinguishable from a bug, and frustrated operators will route around it. That is the institutional failure mode of every opaque control ever deployed.

6.3 Agent learning: trainable, and reward design is where it goes wrong

Reflexion (2303.11366) is the seed of the learning story. Agents that convert failure into articulated, natural-language self-reflection, stored in episodic memory and consulted on later attempts, improve dramatically without any weight update. Its fingerprints are on the structured replan log of §5.6 and on the institutional memory where escalation outcomes accumulate. Verbal reinforcement is also the cheapest learning loop an enterprise can run. It needs no training infrastructure. It is prompt-space learning under version control, auditable line by line, a property compliance departments notice.

The survey of reinforcement learning for agents (2510.16724) frames the field's maturation: RL has moved from aligning single responses to training multi-turn, tool-using, memory-bearing agents end to end. RAGEN (2504.20073) is the cautionary tale inside that programme. Training agents by multi-turn RL, it documents a failure it calls the Echo Trap: reward variance collapses, gradients spike, and the agent falls into repetitive, locally rewarded reasoning loops. Stable self-improvement required filtering trajectories, stabilising the critic, and above all rewarding reasoning quality rather than outcomes alone.

WebAgent-R1 (2505.16421) demonstrates the positive result at scale: end-to-end multi-turn RL with simple task rewards substantially lifts success on realistic long-horizon web tasks. R1 (2501.12948) established the general point that extended reasoning can be elicited by RL. Read together: the act/verify/replan/escalate policy is trainable, but only inside guardrails the reward cannot renegotiate.

The process-versus-outcome study (2505.14069) sharpens reward design for agentic retrieval, and its findings carry weight here. Process rewards, which score the quality of intermediate steps and in particular whether retrievals were warranted and grounded, outperform outcome-only rewards for search agents. Sequencing matters too: a model that first learns outcome-hacking is hard to rehabilitate with process signal afterwards. Translated into budget terms: reward the agent for honest budget spending, for verifications that measurably reduced uncertainty and escalations that reviewers judged well-founded, and never merely for closed cases. An outcome-only reward on closed cases is a standing invitation to close cases wrongly.

6.4 Uncertainty and retrieval: confidence is measurable, and search has a price

The verbal-uncertainty work (2503.14477) shows that a model's expressed uncertainty is governed by a single linear feature, steerable independently of its actual semantic uncertainty. That is both diagnosis and treatment. Fluent overconfidence is a mechanistic fact rather than an anecdote, and steering the feature reduces confident hallucination. Search Wisely (2505.17281) documents the behavioural consequence in agentic RAG: confidence-unaware agents both under-search and over-search, retrieving when they should answer and answering when they should retrieve, and grounded uncertainty signals can train the search decision itself. This is the economics of the verify action, discovered independently from the retrieval side.

The retrieval-RL family fills in the machinery. DynaSearcher (2507.17365) trains search agents with multiple rewards at once, accuracy alongside efficiency and information-gain penalties, over dynamic knowledge graphs, showing that retrieval paths can be made accurate and frugal together; its graph grounding maps naturally onto entity-resolution work in financial crime. UR² (2508.06165) unifies retrieval and reasoning under one RL objective with difficulty-aware curricula, so the agent learns when retrieval is worth it per problem difficulty, which is the budget-allocation problem in miniature.

R-Search (2506.04185) adds multi-reward training over reasoning-and-search trajectories with explicit evidence extraction. Tool-Star (2505.16410) extends the logic to orchestrating several tools, with a hierarchical reward that bonuses correct multi-tool collaboration. VRAG-RL (2505.22019) pushes into visually rich documents, training region-selection and zoom actions so the agent attends to the right part of a page, which matters wherever the evidence is scanned statements, certificates and forms.

The modality-gap work (2602.07026) explains a failure mode every document-heavy deployment meets: visual and textual representations of the same fact occupy misaligned subspaces, so cross-modal confidence needs its own calibration rather than inheriting text-only curves.

One meta-observation closes the survey. Read together, the four strands describe a division of labour that the autonomy budget makes explicit. Evaluation defines where competence has been demonstrated. Shields make the intolerable unreachable. Learning optimises behaviour inside the remaining space. Calibration keeps the arrangement honest about what it knows. Institutions tend to fund these as four separate programmes and wonder why they never compose. The constrained decision process is the composition.

Ten problems, one framework

Part Two applies the constrained decision process to ten recurring problem classes in banking and financial services. Together they span the institution, from financial crime and credit to markets, operations, customer outcomes and the platform layer underneath. Each stresses a different region of the seven-dimensional state, and the same framework governs all ten under different parameters. That portability is the point. Each section describes the general problem rather than any firm's implementation, maps it onto the decision state, and draws out the design consequences specific to it.

Problem I: screening politically exposed identities

The first problem class is the oldest in financial-crime operations. Given a customer or prospect, determine whether they are, or are connected to, a politically exposed person. Resolve whether a "match" in a screening list is the same human being. Trace beneficial ownership through layered corporate structures. Weigh adverse media whose reliability ranges from investigative journalism to recycled rumour. The economics are brutal: the overwhelming majority of raw name matches are false positives, each consuming analyst minutes, while the rare true positive carries consequence far out of proportion to its frequency.

7.1 Why this is a budget problem

Identity inference is a chain of probabilistic judgements. Same name? Same person? Same person as the officeholder? Connected party? Each link's confidence should govern how much the next link is trusted, and the terminal action, clear or escalate to enhanced due diligence, has sharply asymmetric costs. The width of the investigation (how many registries queried, how many media sources pulled, how deep the ownership graph is traversed) is a budget-allocation decision. A fixed workflow queries the same five sources for every case. A budgeted agent spends little on the obvious mismatch, different birth decade, different continent, no overlap in associates, and concentrates spend on the genuinely ambiguous case. Analyst practice already works this way; procedure manuals just do not say so.

The state takes a characteristic shape here. Consequence is asymmetric by direction of error: a false clear on a true positive is a regulatory event, a false escalation is only cost. That asymmetry belongs in the state, not the prompt. The consequence score of a "clear" disposition should be strictly higher than that of an "escalate" on identical evidence. Reversibility is favourable, since everything before the disposition is read-and-draft work, and this is what makes meaningful autonomy defensible at all.

Novelty is chronic: new officeholders after every election cycle, new ownership-obscuring structures, new outlets of unknown reliability. The tripwire layer of §5.3 earns its keep here more than anywhere. Uncertainty is the working medium, since the whole task is uncertainty reduction, so calibration must be validated per evidence type. String-similarity confidence, biography-alignment confidence and media-credibility confidence are three different curves.

Ownership-graph traversal deserves its own note, because the retrieval literature lands on it directly. Tracing beneficial ownership is a walk over a dynamic knowledge graph of companies, officers, shareholders and registries. The multi-reward retrieval work (DynaSearcher, 2507.17365; R-Search, 2506.04185) describes almost exactly this setting: an agent trained to reach the answer while being penalised for redundant hops learns traversal strategies that are more accurate and cheaper than exhaustive expansion. The budget frame adds the governance wrapper. Graph depth is action-budget spend. A traversal that keeps finding new shell layers is a case whose novelty and consequence are rising together, and the frontier of Figure 3 responds correctly: it contracts, and the case lands on a human desk with the graph so far attached.

7.2 Escalation design

The right escalation object here is a disposition brief: the candidate identity chain with each link scored, the evidence for and against with provenance and a credibility note per item, the checks performed with their outcomes, and a recommended disposition with its calibrated error probability. The reviewer's decision, confirm, overturn, or send back with instructions, is captured against the specific links they disagreed with. That is what makes the feedback trainable. Institutions that capture only the final disposition throw away the supervision signal that matters most: which link in the chain the human judged differently.

Problem II: credit decisions the rules engine could not make

Every automated credit pipeline has an overflow pipe. Applications the deterministic engine cannot decide, thin files, irregular incomes, near-miss ratios, contradictory bureau data, are referred to human underwriters, who resolve them by doing what the engine cannot: gathering context. They read bank statements, reconcile income across documents, phone the broker, weigh an explanation. The referral queue is where underwriting capacity actually goes, and its unit economics are poor because each case is a small bespoke investigation across several systems.

8.1 The shape of the problem

The referred case is defined by missing evidence, not missing rules. The rules engine already encodes policy. What it lacks, case by case, is contextual fact. Is this three-month income dip a career break, a seasonal pattern, or distress? Is the large deposit a gift, a loan, or laundering? An agent working this queue is an evidence-gathering agent: it pulls statements, transaction categorisations, employment records and prior product history, reconciles them, and assembles a case file that either resolves the referral within policy or sharpens exactly what a human must judge. The judgement itself, the lend-or-decline on a case policy could not decide, remains a human act in most regimes and in any prudent design. The agent's autonomy ceiling in this class is the fully assembled recommendation, never the commitment.

That ceiling is not a limitation to be tolerated. It is what makes the class an ideal deployment. With the terminal act reserved, everything the agent does is read-and-draft, high reversibility across the whole trajectory, and the machinery concentrates on making the assembly excellent: verification spend where documents conflict, calibrated confidence on each reconciled figure, and an escalation object that turns a four-hour underwriter investigation into a twenty-minute adjudication. The value case is throughput and consistency. The risk case is quiet: underwriters may begin rubber-stamping well-formatted case files, which is the reviewer-saturation pathology of §2.3 in a nicer suit.

8.2 Anti-anchoring as a control objective

That risk deserves engineering rather than hand-wringing, and the framework supplies levers. Calibrated presentation: the case file leads with the agent's uncertainty, so figures it could not reconcile appear before figures it could. Deliberate friction on the margin: for cases near the policy boundary, the file withholds the recommendation entirely and presents balanced evidence, because anchoring does most damage exactly where independent human judgement is the control.

Seeded challenge cases: a small rate of honeypot files with known-wrong recommendations flows through the queue, and an underwriter population that stops catching them has measurably stopped reviewing. That is a far more honest metric than an approval-rate dashboard. The process-reward finding (2505.14069) reappears at the human layer: reward the process of review, not the throughput of approvals, or the review will optimise itself into decoration.

Novelty in this class is seasonal and structural rather than adversarial. New product types, new documentation standards, macro shifts that change what a normal statement looks like: these arrive on institutional timescales, so the evaluated envelope can be maintained deliberately. Each quarter's certification refresh folds the season's adjudicated referrals into the evaluation corpus, and the novelty detector's reference moves with the book. The class is therefore a natural proving ground for the autonomy-expansion loop. Start with everything at rung 2. Accumulate adjudicated outcomes. Demonstrate per-segment reliability. Expand the ACT region segment by segment, irregular-but-documented income first, thin files much later, each expansion an evidenced and reversible policy change.

Problem III: synthesis for client-facing advisers

The third class serves the people who serve clients. Before every client interaction, relationship managers and advisers must assemble a picture scattered across a dozen systems: portfolio positions, product holdings, covenant headroom on lending facilities, service history, prior meeting notes, market movements relevant to the client's exposures, and whatever the news has done to the client's industry overnight. The assembly is drudgery. The synthesis is judgement. The deadline is always this afternoon.

9.1 A low-stakes class, with two live wires

On the heatmap this row looks benign, and mostly it is. The output is a briefing consumed by a professional who applies their own judgement, the actions are almost entirely reads and drafts, and the novelty per query is modest. This is why it is many institutions' first agentic deployment. The framework's advice: embrace that. Set the budgets permissively, price escalation high (interrupting an adviser's day to review a briefing defeats the purpose), and let rung-1 sampled review carry the assurance load.

Two wires are live, though. The first is citation integrity. A briefing is synthesis, and synthesis is where language models fabricate most fluently: the blended sentence that is 90% sourced and 10% invented, with no visible seam. In this class the confidence dimension attaches to claims, not to the episode. Every material figure and factual assertion carries retrieval-support scoring. Claims that fail support are dropped or rendered visibly as unverified, never silently blended. The enforcing discipline is structural: the drafting step consumes only from the evidence set assembled earlier in the trajectory, and the shield rejects drafts containing material claims without provenance pointers. Provenance or silence is the never-event rule of this class.

The second wire is the suitability boundary. A briefing that says "the client's covenant headroom has narrowed to 1.2×" is description. One that says "you should propose refinancing" has crossed into advice, which carries regulatory weight in most jurisdictions. The boundary is a consequence cliff and should be engineered as one. Recommendation-shaped language is detectable, and the shield can hold drafts that cross from description into solicitation until the adviser explicitly adopts them. The adviser may take the suggestion, but the record shows a human adopted it rather than a machine issuing it. This is reversibility engineering applied to language: descriptive content is freely revisable by its reader; advice, once relied upon, is not.

9.2 The covenant-headroom pattern

One recurring sub-task shows the framework's texture well. Computing covenant headroom, how far a borrower sits from breaching the financial covenants on a facility, requires reading the facility agreement's definitions, which differ deal by deal, extracting the covenant formula, pulling current financials, and computing the ratio as the document defines it rather than as the textbook does. This is VRAG-RL territory (2505.22019): the evidence is a scanned legal document whose critical clause must be located, read precisely and interpreted. The budget texture: extraction confidence gates computation.

A covenant formula extracted at low confidence triggers a verify, a re-extraction against the clause image or a checker model reading the same clause, before any number is computed from it. A headroom figure that has narrowed sharply since last computed is consequence-elevated, because it is exactly the figure the adviser will act on. The general lesson: inside even a "low-stakes" briefing, individual claims carry their own consequence scores, and the state is evaluated at claim level where it matters.

The learning loop here is unusually rich because feedback is abundant and low-ceremony. Advisers correct briefings in the natural course of using them, and every correction, a figure amended, a section deleted as irrelevant, a missing exposure added, is a labelled datum about synthesis quality. Reflexion-style memory compounds it. "For this client, the adviser always removes the market commentary; stop generating it" is exactly the kind of verbalised lesson that improves the next hundred briefings without a single gradient step.

Problem IV: detecting manipulation in markets

Trade-surveillance functions sit on an alert torrent. Pattern engines flag order behaviour that resembles layering, spoofing, wash trading or front-running, and human analysts must reconstruct intent from order-book microstructure, execution timelines, communications and market context. False positives dominate by orders of magnitude. The true positive, missed, becomes an enforcement action with the firm's name on it. The investigation is hard because the same order pattern can be manipulation, legitimate market-making, or a hedging programme, and the difference lives in context no single system holds.

10.1 Intent reconstruction as budgeted investigation

Surveillance investigation is the most epistemically demanding class in this catalogue, because the target is intent: a hidden variable inferred from behaviour, communications and circumstance. The agent's work is hypothesis competition. Construct the innocent explanation and the abusive explanation, then spend budget gathering the evidence that discriminates between them. This structure should be explicit in the trajectory, for two reasons. It is how good analysts already think, which makes the agent's output legible to them.

And it is the structure regulators expect case files to have: the record must show the innocent explanation was considered and excluded by evidence, not merely omitted. An agent that emits "manipulation score 0.83" gives an enforcement lawyer nothing. An agent that assembles the timeline, aligns it with communications, quantifies the discriminating features and writes both explanations with their support has done analyst work.

The state has sharp edges here. Consequence is severe in both directions: a false dismissal risks regulatory failure, while a false accusation that leaks into an employee's file or a client relationship prematurely is its own harm. Both dispositions carry high consequence, unlike the screening class where the asymmetry ran one way. Novelty is adversarial and fast, since manipulation strategies mutate to evade the pattern engines by construction.

The surveillance agent lives permanently near the edge of its evaluated envelope, and its ceiling should reflect that: rung-2 or rung-3 adjudication indefinitely for any disposition with regulatory significance. Communications evidence adds a further wrinkle. It is the most probative evidence and the most privacy-sensitive to access, so the act of pulling communications is itself consequence-scored, and the budget frame forces the agent to record why the order-flow evidence warranted the escalating intrusion. Proportionality of investigation, a legal principle, becomes a mechanical property of the ledger.

The training story is correspondingly cautious. Outcome labels are scarce, slow and skewed. Few alerts become confirmed cases, and enforcement outcomes arrive years later. Outcome-only reward is therefore both starved and dangerous, and the throughput-hacking warning of §6.3 applies with maximum force to a queue whose easiest "resolution" is dismissal. Process reward is the only viable signal. Was the timeline reconstruction faithful? Were both hypotheses genuinely developed? Did verification target the discriminating features? Senior analysts, or an AgentAuditor-style adjudicator at rung 1, can judge all of this on the trajectory itself, without waiting for ground truth that may never arrive.

Problem V: fraud and push-payment scams

Fraud splits into two tempos. The fast tempo is interdiction: a payment is in flight, signals suggest a scam, and the institution has seconds to minutes to hold, challenge or release. The slow tempo is investigation: after the hold, an evidence-led reconstruction, was this a romance scam, an impersonation, a mule account, or a genuine payment that pattern-matched badly, leading to intervention, recovery attempts and reporting. Authorised-push-payment scams are the hard case because the customer authorises the payment. The fraud lives in the manipulation of the customer, and the evidence is behavioural and conversational rather than technical.

11.1 Two tempos, two parameterisations

The framework handles the split cleanly because tempo is just a budget schedule. The interdiction episode runs with tiny budgets: single-digit actions, sub-second cost ceilings, no replans, and an autonomy grant that is real but narrow. The agent may hold far more freely than it may release, and the reason is reversibility, honestly scored. A hold is undoable within the settlement window; the customer is inconvenienced, not harmed. A release is irreversible the moment funds settle in a mule account and cascade onward.

Note the inversion of the usual instinct: here the intervention is the reversible act and the non-intervention is the irreversible one. The customer-experience cost of false holds is then managed by making release-after-challenge fast, spending the saved reviewer minutes on rapid resolution of held payments rather than on pre-hold deliberation.

The investigation episode then runs at the slow tempo: reconstruct the payment chain, the device and session forensics, the conversational history where the manipulation happened (with the same shield-gated, justification-required access pattern as §10), the counterparty account's behaviour, links to known mule networks. This is graph traversal plus multimodal evidence again, frugal graph search in the DynaSearcher style over accounts and devices, VRAG-RL-style reading of the screenshots and payment confirmations victims provide. Its terminal products are consequence-tiered: internal case notes at rung 1, customer-affecting interventions at rung 2, and reimbursement adjudications on contested facts at rung 3, where mid-flight consultation lets the agent present the evidence gap precisely.

11.2 The vulnerability overlay

Scam investigation carries a duty-of-care overlay that reshapes consequence scoring. The victim is often still under the scammer's influence, believing the "investment" is real, coached to answer bank challenges, and may be vulnerable in other ways besides. Interventions that would be routine elsewhere, a blunt challenge question, a cold notification of loss, can cause real harm or tip off the manipulation.

Consequence scoring for customer-touching actions must therefore include the customer's inferred state, and the shield should hold certain communications, loss notifications above thresholds, challenge scripts for suspected coached victims, for human sign-off regardless of the agent's confidence. The institution's duty here is one no calibration curve discharges. The general pattern deserves a name: some consequence floors are deontological rather than statistical.

They are set by what the institution owes people, not by error rates, and the framework hosts them as shield rules precisely so that no amount of learned confidence can erode them.

Problem VI: document-heavy secured lending

Secured-lending origination, with the mortgage as the canonical case, is a pipeline of documents: applications, payslips, bank statements, tax records, valuations, insurance evidence, legal reports, each arriving in whatever format the issuer favours, each needing extraction, cross-checking and policy assessment. The work is affordability arithmetic, income reconciliation, fraud screening and policy conformance, threaded through gates where a human underwriter must own the decision. It is the highest-volume document-AI problem most retail institutions have.

12.1 The reconciliation lattice

The organising structure of this class is the reconciliation lattice. The same underlying fact, the applicant's income, their existing obligations, the property's value, is attested by multiple documents that should agree and frequently do not, for reasons ranging from timing (a payslip predating a raise) to formatting (gross versus net) to fraud (an edited statement). The agent's core loop: extract each attestation with per-field confidence, reconcile attestations of the same fact, and treat reconciliation deltas as the primary signal driving everything downstream.

A clean lattice supports high autonomy through the assembly stages. A delta is a fork. Timing and formatting explanations are investigated within budget, one more statement month pulled, a payslip date re-read. Unexplained deltas above materiality are what human underwriters exist for. Deltas whose shape matches tampering signatures trigger the fraud path, with its own elevated consequence scoring and its own specialists.

Per-field calibrated confidence keeps the lattice honest, and the modality-gap finding (2602.07026) is operationally real here. Confidence in reading a figure from a scanned image and confidence in parsing it from structured data are different quantities with different failure modes. Pooling them into one score hides exactly the cross-modal disagreements that matter. Trained visual attention in the VRAG-RL style earns its keep on the long tail of layouts. A payslip format the extractor has never seen is a novelty event at the document level, and the correct response is the frontier's: drop to low autonomy for that document, human-verified extraction, without dropping the whole case.

12.2 Where the budget pays for itself

The economics of this class are dominated by the rework loop. A case that reaches the underwriter with gaps bounces back, re-queues and ages, and case age is the metric customers feel. The budget framework attacks rework at its source by making completeness-before-escalation a trainable objective. The escalation object is specified: every reconciled fact green or explained, every delta either resolved or presented with its competing explanations. The underwriter's bounce-backs become the process-reward signal, and the agent learns to spend its verification budget where bounce-backs historically originate.

Institutions that have run assembly-line document AI without this loop will recognise the failure it fixes: extraction accuracy looks excellent while underwriters drown in almost-complete files, because nothing in the system was optimised for the reviewer's decision-readiness. That, in the end, is the only metric the pipeline exists to serve.

Problem VII: perpetual customer due diligence

The industry is midway through a quiet inversion: from periodic KYC review, every customer re-examined on a one-, three- or five-year cycle whether or not anything changed, to perpetual, event-driven review, where a standing process watches for material change and opens focused reviews when it occurs. The inversion is rational. Periodic review spends almost all its effort re-confirming the unchanged. But it replaces a bounded, schedulable workload with an unbounded, event-driven one. That is an autonomy-budget problem in its purest form.

13.1 A standing process is a portfolio of budgets

Perpetual review is not one episode. It is a standing allocation problem: a stream of change events of wildly varying materiality competing for finite investigation capacity. The framework scales up naturally, with budgets at three nested levels. The event level asks whether a change is material enough to open a review at all; a director's re-election is not, a new 30% shareholder in a new jurisdiction is. The review level is the familiar episode of §7, budgeted by customer risk tier and change materiality.

The portfolio level is new: a monthly investigation-capacity budget per book, whose depletion rate is itself a monitored signal. A book whose events persistently exhaust portfolio budget is a book whose risk profile has changed, and that fact should surface to the accountable owner as a portfolio alert rather than stay buried in per-case queues. The shadow-price telemetry of §4.3 turns strategic here.

The marginal value of one more analyst-hour, per book, per month, is exactly the number a financial-crime function needs for capacity planning, and the ledger produces it as a by-product.

Novelty plays a distinctive double role. In perpetual review, novelty is the trigger as well as a caution, since the whole design watches for the unfamiliar. The two roles need different instruments. A novelty signal tuned for triggering is sensitive, cheap and high-recall. A signal tuned for autonomy governance is specific and calibrated against the evaluated envelope. The trigger asks whether the world has changed near this customer. The governor asks whether this review sits inside the territory where the agent's competence was demonstrated. Conflate them and you get either a paranoid trigger or a permissive governor, and the failure of each stays invisible until it is expensive.

13.2 The audit dividend

Perpetual review has a compliance-narrative problem that the ledger happens to solve. Under periodic review, coverage was trivial to evidence: every file touched on schedule. Under event-driven review, the question "why was this customer not reviewed last year?" needs a defensible answer. The answer is the triage record: every event scored, every below-threshold determination logged with its evidence and calibrated confidence, every threshold traceable to an approved policy.

The budget ledger, built for autonomy governance, turns out to be the coverage evidence, the demonstration that attention was allocated by a consistent, examinable policy rather than by drift. This pleasant surprise recurs across all ten classes and is worth stating as a principle: the instrumentation that budgeted autonomy requires is a superset of the instrumentation regulators increasingly request. Building the first discharges much of the second for free.

Problem VIII: complaints and duty of care

Complaint handling looks operational and is actually constitutional. It is where the institution's duty to deliver good customer outcomes is tested case by individual case, under statutory deadlines, with an ombudsman behind every mishandled file. The work spans classification (what the complaint is actually about, often not what its subject line says), vulnerability recognition, root-cause analysis, conduct-risk detection, remediation design and response drafting, with accountability for the outcome resting on named humans.

14.1 The vulnerability gate and the systemic tripwire

Two features distinguish this class. The first is the vulnerability gate. Recognising that a complainant is vulnerable, financially, cognitively or circumstantially, changes everything downstream: communication style, remediation calculus, deadlines, and the harm of getting it wrong. Vulnerability inference is exactly the kind of judgement where model confidence should be consumed asymmetrically. A low-confidence positive inference, this person may be vulnerable, should flow freely into the handling posture, since treating a non-vulnerable person gently costs nearly nothing. A low-confidence negative should never gate protections off. One-directional consumption of an uncertain signal is a small pattern with outsized duty-of-care consequences, and it generalises: for any protective classification, wire the uncertainty so that doubt defaults to protection.

The second feature is the systemic tripwire. A single complaint is a case. A pattern of complaints is a conduct-risk event with reporting implications and, possibly, a remediation programme behind it. The agent working individual cases doubles as the sensor network for the pattern: complaint embeddings clustering where no cluster existed, a product suddenly over-represented, a demographic skew in outcomes. Pattern detection is a portfolio-level observation, like the budget-depletion alarms of §13, and its escalation is lateral, to the conduct-risk function rather than up the case-handling chain. The escalation design must support that explicitly. Some of the most valuable things an agent can notice are things no individual case owner is responsible for hearing.

Deadlines give this class its distinctive rhythm. Statutory clocks mean remaining time functions as a fourth depleting resource beside the canonical three, the time-criticality variable of §5.8 at full strength, and the policy must degrade gracefully under it. A case approaching its deadline with investigation incomplete should escalate earlier than its evidence state alone would warrant, because a late perfect answer is a regulatory failure while an early escalation is merely a cost. Encode the deadline as budget rather than as prompt text, for the usual reason: budgets are enforced and audited. Instructions are weather.

Problem IX: the knowledge and context fabric

Beneath every class so far sits the same dependency. The agent's evidence comes from somewhere, and the quality, governance and provenance of that somewhere determine everything above it. The knowledge and context fabric is the institution's answer: governed knowledge objects with owners and review cycles, intent-aware routing that sends a query to the right sources, context assembly that composes retrievals into a working set, provenance that survives from source to citation, and entitlement enforcement that respects every confidentiality boundary the institution has. It is a problem class in its own right, and the autonomy budget applies to it twice.

15.1 Retrieval is an agent too

The first application is direct. Assembling context for a downstream task is itself a budgeted episode. The retrieval layer decides how many sources to query, whether to expand a search that returned thin results, whether to reconcile two knowledge objects that disagree, and when the assembled context is good enough to hand over. Every pathology the agentic-RAG literature documents lives here: under-retrieval that starves the downstream task, over-retrieval that buries the relevant in the marginal and measurably degrades accuracy, unwarranted trust in stale or low-authority sources.

The remedies are the ones §6.4 catalogued, confidence-gated search decisions, difficulty-aware retrieval effort, multi-reward frugality, wrapped in the budget so that retrieval spend is visible, bounded and attributable. A useful metric falls out: context cost per decision, per task class, trended over time. It tells you whether your knowledge estate is getting easier or harder for machines to use, a leading indicator most institutions currently cannot see at all.

The second application is at the write path, and it matters more. The fabric's content is itself agent-maintained: summaries refreshed when sources change, knowledge objects proposed from recurring answers, staleness flagged, contradictions surfaced. Writes into the substrate that every other agent trusts are high-consequence by definition, and reversible only if versioning is impeccable. So the write path runs at low autonomy: agent-proposed, owner-approved, always versioned, never silent. The never-event of this class is unattributed mutation. No knowledge object changes without a recorded author, human or agent, a recorded basis, and a diff a human can read. Institutions that let convenience erode this rule discover the failure months later, when nobody can say why every agent in the building believes something that stopped being true in March.

15.2 Provenance as the load-bearing wall

Every earlier section leaned on a phrase, "with provenance attached," and this layer is what makes the phrase real. Provenance means every atom of context carries its source identity, version, authority tier, entitlement scope and retrieval timestamp, and the chain survives composition: when a draft cites a figure, the citation resolves through the assembled context back to the governed object and its source system. Tedious engineering, three compounding payoffs. It makes retrieval-support confidence computable, since a claim's support can only be measured if its evidence is identifiable.

It makes escalation objects self-verifying, since the reviewer clicks through to sources instead of trusting the summary. And it makes trajectory replay possible at all. The fabric is also where entitlements become autonomy constraints. A case boundary, an information barrier, a jurisdiction wall: these are shield rules enforced at retrieval time, so a downstream agent cannot leak what it was never allowed to assemble. Cross-boundary retrieval requests are not errors. They are recorded, justified, consequence-scored events, the proportionality pattern of §10 generalised to the whole institution.

Problem X: the institution-wide agent platform

The final class makes the other nine sustainable: the shared platform on which agents are registered, skilled, identified, routed, evaluated, observed and certified. Without it, each deployment rebuilds governance bespoke, controls drift apart, and the institution's tenth agent costs as much as its first. With it, the autonomy budget stops being an architectural pattern and becomes an operating system.

16.1 The platform is where budgets become real

Everything treated so far as a design element becomes, at platform level, a service with an owner. The registry holds each agent's identity, its certified task classes, and the addition this framework demands: its autonomy grants per class, the frontier parameters, budget schedules and escalation prices it has earned, versioned like any release artefact. Runtime identity gives every action a non-repudiable actor: this agent, this version, this episode. Without it the ledger is a story rather than a record.

The budget accountant meters and debits in real time and owns the ledger. The shield service evaluates every contemplated action against the compiled rule base, centrally maintained so a policy change propagates to every agent at once instead of through a hundred prompt edits. Model routing becomes budget-aware: cheap models for low-consequence steps, expensive deliberation purchased where the state justifies it, the cost budget made architectural. Observability is the read side of all of it: trajectories, ledgers, escalations and outcomes, queryable per agent, per class, per book.

16.2 Reuse without governance dilution

The platform's economic promise is reuse: a document-extraction skill certified once and consumed by lending, financial crime and complaints alike. Reuse creates the platform's characteristic governance problem, because a skill's certification does not transfer across consequence contexts. Extraction at 97% field accuracy is high-autonomy in an adviser briefing and human-verified in a fraud case file, since the downstream consequence differs by orders of magnitude. The registry must therefore represent grants as (skill, context) pairs. The composition rule for multi-agent workflows follows the same logic.

When agents compose, the composite's autonomy is not the maximum or even the minimum of its parts' grants. It is a fresh assessment of the composite trajectory, because composition creates exactly the accumulated, sequence-level risks that §2.1 showed per-step analysis cannot see. The platform also hosts inter-agent budget flows: an orchestrating agent delegating to a specialist passes down a sub-budget, never an open account, so the ledger stays conservative under delegation and every unit of spend is attributable up the call chain.

Delegation without budget inheritance is how an institution rediscovers, expensively, why finance departments invented cost centres.

Finally, the platform carries the institution's learning economy. Escalation outcomes, reviewer adjudications, replan logs and honeypot results flow back through it as the training corpus for the next policy version. The platform's data governance is therefore the training-data governance, and the certification loop of Figure 13 is the deployment gate for learned changes. The RL machinery of §6.3 enters production through this gate or not at all: a policy update, like any other autonomy expansion, is a versioned, evaluated, reversible artefact. Institutions that internalise this stop asking whether to "let the agents learn." The system learns continuously through its cheapest loops, memory, prompts, budgets and frontier parameters, while weights change only through the certification gate, and both kinds of change leave the same audit spoor.

A reference architecture

Part Three turns from what and why to how. This section lays out the minimal set of components an institution needs for the constrained decision process to be enforced rather than merely described, and the boundaries along which those components should be owned. The architecture is deliberately conservative. Every component is buildable with current technology, and most institutions already own fragments of each. What is usually missing is the connective tissue and the discipline of a single ledger.

17.1 Ownership boundaries

The architecture's political content is in who owns what, and the framework has firm opinions. The worst available design is the plausible-sounding one: a single "AI team" owning all four decision-layer components, which recreates in miniature the unsegregated-duties problem a century of financial control exists to prevent. The decision process is powerful because its components check each other. Concentrated ownership dissolves the checks.

17.2 The ledger as the system of record

If one component deserves over-investment it is the trajectory and ledger store. Its schema is the framework's contract. Every episode is a sequence of records: state assessed with all dimensions as scored at the time, options shield-filtered with the rules that fired, action selected with the policy version that selected it, rejected alternatives, spend debited, outcome observed. Append-only, versioned, replayable. Replayability is the property that pays.

An incident review can re-run the episode against the state as the agent saw it, distinguishing "the state was mis-scored" from "the policy chose badly from a true state," two failures with entirely different remediations. A proposed policy change can be evaluated counterfactually against last quarter's ledger before it touches production. And the audit function inherits something it has never had for human decision-making: the complete decision basis of every consequential act, captured at the moment of decision, uncontaminated by hindsight.

One boundary deserves emphasis: the model's hidden reasoning is not a suitable audit record. It may be unavailable, unstable, verbose or misleading, and nothing guarantees it reflects the causal basis of the action. The ledger must capture externally verifiable control facts, claims, evidence references, candidate actions, policy checks, budget movements, decisions and outcomes, so the record can be reconstructed without trusting the model's account of itself. Auditors audit the ledger. The model's prose is testimony, not evidence.

Evaluation, assurance and the audit trail

Budgeted autonomy is a claim: within this envelope, this agent's unsupervised behaviour is acceptable. Claims require evidence regimes. This section describes the regime: what is measured, how the three harm routes are covered, how gaming is caught, and which numbers deserve a place on the accountable executive's dashboard.

18.1 Certification: the envelope is the product

Certification's output is not a score. It is an envelope: a region of the state space, described in the same dimensions the runtime scores, within which measured error rates support a specified grant. This imposes healthy discipline on evaluation design. The corpus must be stratified along the dimensions that will govern runtime autonomy, consequence tiers, novelty bands, document modalities, difficulty, because an aggregate accuracy figure supports no envelope at all. Following the OS-Harm taxonomy (2506.14866), each envelope is tested against all three harm routes.

Misuse: can an operator weaponise the grant? Injection: do adversarial payloads in retrieved content or uploaded documents move the agent off-policy? In financial services the attack surface is documents, which arrive from customers and adversaries alike. Misbehaviour: the ordinary error distribution. And the probing must be behavioural (2401.10019; 2412.14470): the agent is evaluated by being run in a sandboxed replica against seeded cases, not by being asked how it would behave.

18.2 Continuous assurance: sampling, honeypots, drift

Between certifications, three mechanisms keep the envelope honest. Sampled adjudication takes a fixed fraction of autonomous trajectories and adjudicates them as if they had been escalations. The measured disagreement rate between adjudicator and agent is the live estimate of unsupervised error, and its trend is the single most important assurance series the system produces. AgentAuditor-style evaluator models make the economics work: the model adjudicates everything sampled, humans adjudicate where the model is uncertain plus a fraction of where it is certain, auditing the auditor.

Honeypots are seeded cases with known correct dispositions, including cases designed to tempt each documented gaming pattern: the dismissible-looking true positive, the case closable only by skipping a required verification, the reward-attractive wrong answer. Honeypot failure rates are the anti-gaming metric, for agents and for the human review layer alike. Drift tripwires watch the novelty distribution of live traffic against the evaluated corpus, per class.

When live cases drift out of envelope, the contraction loop of Figure 13 fires automatically, and the next certification refresh inherits a ready-made corpus of exactly the cases that drifted.

A further monitoring insight costs nothing and catches problems early: watch the state distributions themselves. If average consequence scores drift upward for a task class, the work is changing. If confidence distributions shift while accuracy holds, calibration is going stale. If shield blocks spike for one workflow, a policy or a prompt changed somewhere upstream. State distributions are often better early-warning signals than task accuracy, because they move first.

18.3 The audit trail as narrative

Assurance data answers regulators' questions only if it composes into narratives, and the framework's artefacts compose unusually well. For any decision, the institution can produce the state as assessed (why the system believed what it believed), the shield rules consulted (what was forbidden), the action chosen and the policy version choosing it, the budget position (what pressure the decision was under), the evidence with provenance, and, where a human was involved, the escalation object as presented and the adjudication as given. The last item addresses accountability regimes directly.

The perennial gap in "human oversight" claims is that the record shows a human approved but not what they reviewed. An escalation architecture that captures the presented evidence closes that gap structurally. The institution can demonstrate not merely that a human was in the loop, but that the loop contained the information a reasonable reviewer needed. That demonstration, repeatable for every consequential decision, is the audit dividend of the whole apparatus.

A maturity model and roadmap

No institution builds Figure 14 in one budget cycle, and the framework does not require it. Its components are valuable in strict sequence. Each stage delivers standalone value and lays the substrate for the next, which is what makes the roadmap credible where grand target-state architectures are not.

19.1 What each stage buys

Stage 1, instrument. Build the ledger and class the tools. Change no behaviour. The deliverable is visibility: real trajectory-length distributions, real cost per case, real reviewer-time consumption, the empirical basis every later parameter needs. Institutions discover at this stage, cheaply, that their assumptions about their own workflows are wrong, rather than discovering it at stage 4, expensively.

Stage 2, constrain. Compile the never-events into a shield. Set static budgets at generous percentiles. Run the hand-written rulebook policy with rungs 1 and 2. This stage delivers most of the governance value, bounded worst cases, mandatory pauses, evidence-carrying escalations, using no machine learning beyond the agents themselves.

Stage 3, calibrate. Stand up the confidence layer and novelty detection. Introduce priced escalation and rung 3. The frontier becomes real: autonomy now responds to instance-level state.

Stage 4, adapt. Let the ledger tune the parameters: budgets from observed distributions, κ from queue economics, frontier boundaries from sampled-disagreement evidence. Add episodic memory and verbal learning in the Reflexion style. The system now improves without weight changes, under version control.

Stage 5, learn. Train the policy itself, process-rewarded, shield-constrained, honeypot-tested, and admit it through the certification gate as a versioned artefact. Expect years between stages 1 and 5, not quarters. Nothing forces the pace, because every stage stands on its own economics.

19.2 Sequencing across the problem classes

The ten classes of Part Two also imply an adoption order, driven by where the prerequisites already exist and where failure costs are gentlest. Adviser synthesis (§9) and the knowledge fabric (§15) come first: high reversibility, professional consumers, and they build the provenance substrate everything else needs. Document-heavy lending (§12) and referred credit (§8) follow: bounded domains, abundant ground truth, human gates already in place. The financial-crime investigation classes (§7, §13) come next, exploiting mature evidence machinery under conservative dispositions.

Surveillance (§10), scams (§11) and complaints (§14), the classes with adversarial novelty, deontological floors and statutory clocks, deserve the most mature apparatus. The platform (§16) is not a phase but a thread: it begins as the standards the first deployment follows and hardens into shared services as the second and third arrive. Resist the tempting inverse, leading with the highest-drama use case because it commands sponsorship.

That deploys the least mature apparatus against the least forgiving problem, and tends to buy the incident that sets the programme back two years.

Objections and open problems

A framework earns trust by stating where it strains. Five objections deserve honest treatment, and two problems remain genuinely open.

20.1 "The scores are subjective, so the rigour is theatre"

The sharpest objection: consequence, novelty and confidence are estimates produced by fallible components, so the seven-dimensional state launders judgement calls through numeric costume. The response is comparative, not absolute. The alternative to explicit, versioned, auditable estimates is the same judgements made implicitly, buried in prompt phrasing, in a product manager's tier assignment, in a reviewer's mood, with no record, no calibration and no improvement loop. The framework does not eliminate judgement. It locates it, attaches evidence to it, and exposes it to correction. A mis-scored state is a diagnosable, fixable failure precisely because the score was recorded. The objection lands one real blow, though: institutions must resist decorating weak estimates with strong decimal places. Coarse bands honestly derived beat precise scores confidently invented, and the framework works identically over bands.

20.2 "This will strangle the agents into uselessness"

The over-refusal worry of §6.2, applied at system level: budgets, shields and escalation prices could compound into an agent that does nothing cheaply. The design answer is that the framework contains its own corrective. Shadow prices and escalation-precision metrics make over-constraint visible and quantified, which hand-wavy governance never is. The honest caveat: the corrective only works if someone reads the telemetry with authority to act. A budget regime without an owner who relaxes constraints when the evidence says so will calcify. Not because the framework strangles, but because unattended controls always do.

20.3 "Humans will still rubber-stamp"

True, and no architecture abolishes it. The framework's claim is narrower. Escalation pricing keeps review volumes inside the attention budget. Evidence-first escalation objects lower the cost of genuine review. Honeypots measure whether review is genuine. Attention telemetry catches saturation early. The residual risk, a well-rested reviewer genuinely reviewing and still deferring to machine fluency, is the hard core of the automation-bias literature, and it is managed rather than solved. Withheld recommendations near boundaries (§8.2) and one-directional protective defaults (§14.1) are mitigations, not cures.

20.4 "Nobody solves CMDPs in production"

Correct, and the framework never requires it. The CMDP is the specification. It defines what the enforcement layer guarantees (constraints), what the ledger records (state, action, spend), and what any policy, rulebook or learned, is accountable to. Stage-2 institutions run it with if-statements. The formalism's value is that the if-statements, the telemetry and the eventual learned policy all refer to the same objects, so maturity upgrades never require re-specification. Precision of contract is the point, not sophistication of solver.

20.5 "Novelty detection is unsolved, so the envelope leaks"

Partially conceded. Out-of-distribution detection over embedded case representations is imperfect and will miss semantically novel cases that embed unremarkably. The layered design, statistical plus symbolic tripwires, narrows the leak. Sampled adjudication bounds its consequences, since leaked cases are still sampled at the base rate. The honest statement of the guarantee is probabilistic: the envelope reduces out-of-competence autonomous action by a large, measurable factor. It does not reduce it to zero, and certification documents should say so in those words.

20.6 Open problems

Two problems this essay does not solve. First, multi-agent budget game theory. When budgeted agents transact with each other, an orchestrator purchasing a specialist's verification, or two institutions' agents negotiating a document request, budget schedules become strategic objects. Can an agent burn a counterparty's budget? Do sub-budget auctions clear efficiently? The mechanism-design questions have no settled answers yet. Second, the calibration of calibration. The confidence layer is validated against realised outcomes, but the validation has a sample-size frontier. For rare, high-consequence segments, exactly the ones that matter most, outcome data accumulates too slowly to validate calibration tightly. The interim answer is conservative floors plus synthetic evaluation, which is scaffolding rather than solution. Both problems reward research attention. Neither blocks adoption of stages 1 through 4.

The budget view's deepest claim is cultural rather than technical. It asks institutions to treat machine autonomy the way they already treat every other scarce, risky, valuable resource: granted against evidence, metered in use, audited after the fact, and repriced as the world changes. Banks have run on that grammar for centuries. The framework's contribution is to notice that autonomy conjugates.

Sources

The papers below are the primary sources synthesised throughout, listed by arXiv identifier. Section 6 discusses each in context.

1. 2303.11366 · Shinn et al., Reflexion: Language Agents with Verbal Reinforcement Learning.
2. 2401.10019 · R-Judge: Benchmarking Safety Risk Awareness for LLM Agents.
3. 2412.06878 · SafeWatch: An Efficient Safety-Policy-Following Video Guardrail Model with Transparent Explanations.
4. 2412.14470 · Evaluating the Safety of LLM Agents (Agent-SafetyBench).
5. 2501.12948 · Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (DeepSeek-R1).
6. 2503.14477 · Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations.
7. 2503.22738 · ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning.
8. 2504.20073 · RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning.
9. 2505.14069 · Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learning.
10. 2505.16410 · Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning.
11. 2505.16421 · WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning.
12. 2505.17281 · Search Wisely: Mitigating Sub-optimal Agentic Searches by Reducing Uncertainty.
13. 2505.22019 · VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning.
14. 2506.00641 · AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agents.
15. 2506.04185 · R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning.
16. 2506.14866 · OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents.
17. 2506.20949 · Beyond Reactive Safety: Risk-Aware LLM Alignment via Long-Horizon Simulation.
18. 2507.08270 · Agent Safety Alignment via Reinforcement Learning.
19. 2507.17365 · DynaSearcher: Dynamic Knowledge Graph Augmented Search Agent via Multi-Reward Reinforcement Learning.
20. 2508.06165 · UR²: Unify RAG and Reasoning through Reinforcement Learning.
21. 2510.16724 · A Comprehensive Survey on Reinforcement Learning for LLM-Based Agents.
22. 2602.07026 · Modality Gap-Driven Subspace Alignment Training Paradigm for Multimodal Large Language Models.

Titles are given as cited in the working notes for this essay. Treat the arXiv identifier as authoritative and consult the latest revision of each paper. Nothing in this essay is legal, compliance or investment advice. Regulatory references are illustrative design pressures, not interpretations.

A pattern language for budgeted autonomy

Frameworks are remembered as patterns. This appendix distils the essay into a working vocabulary, twenty named patterns and anti-patterns that recur across the ten problem classes, meant to be quotable in design reviews. Each entry states the pattern, the force it resolves, and where its full treatment lives.

A.1 Patterns

Escalation is an action, not an exception

Model the handover to a human as an ordinary member of the action repertoire, with a price, chosen by expected value. Exception-path escalation is unowned, unmeasured, and either overused into saturation or underused into silence. Priced escalation makes the demand for human judgement legible and optimisable. (§3.1, §4.2.)

Separate capability from authority

A model may be able to perform an action without being permitted to perform it in the current state. Capability is a property of the model; authority is a property of the state. Confusing them is how "the demo worked" becomes a production incident. (§3.3, §16.)

Downgrade before refusing

When the contemplated action is impermissible, search for a lower-impact substitute before failing the task. Instead of sending, draft. Instead of changing, stage. Instead of deciding, present the ambiguity set. Graceful degradation keeps value flowing inside whatever envelope the state permits. (§5.2, Figure 6.)

Provenance or silence

A claim without a resolvable evidence pointer does not appear in agent output. It is dropped or marked unverified, never blended. Synthesis is where fabrication hides best, and a structural rule is enforceable where a stylistic exhortation is not. (§9.1, §15.2.)

Novelty vetoes autonomous clearance

Whatever the confidence score says, a case flagged as outside the evaluated envelope may not be closed autonomously in a protective direction. Calibration is an in-distribution property, so confidence is the one signal that cannot be trusted where novelty is high. (§5.3, §7.2.)

Doubt defaults to protection

For protective classifications such as vulnerability or coercion, consume uncertain signals one-directionally. An uncertain positive activates protections. An uncertain negative never deactivates them. (§14.1.)

The intervention can be the reversible act

Score reversibility honestly per action, not per connotation. In payment interdiction the hold is undoable and the release is not, so the frontier should be trigger-friendly toward the protective act. (§11.1.)

Both hypotheses before further spend

In investigation classes, require the innocent and the adverse explanation to be articulated before evidence-gathering deepens, and aim verification at the features that discriminate between them. Confirmation bias is a trajectory shape, and trajectory shapes can be gated. (§10.1.)

Justify the intrusion in the ledger

Access to sensitive evidence is consequence-scored and requires a recorded justification from previously gathered evidence. Proportionality of investigation becomes a mechanical property of the trajectory. (§10.1, §15.2.)

Withhold the recommendation at the boundary

Where an assembled case sits near a policy boundary, present balanced evidence without a recommendation. Anchoring is most harmful exactly where independent human judgement is the control. (§8.2.)

Seed the queue with honeypots

Flow known-disposition cases through both agents and human review at a low rate. The catch rate is the only honest measure that review is real and reward is unhacked. (§8.2, §18.2.)

Budgets deplete; frontiers contract

The same evidence that justifies acting with a full budget justifies only escalating with an empty one. Remaining runway is part of the state, not an operational afterthought. (§3.3, §5.5.)

Deadlines are budgets

Statutory and service clocks enter the state as a depleting resource, so lateness pressure produces earlier escalation, enforced by the accountant rather than suggested by the prompt. (§5.8, §14.)

Sub-budgets flow down the call chain

Delegation between agents passes a bounded allocation, never an open account, keeping the ledger conservative under composition. (§16.2.)

Expansion is slow; contraction is automatic

Autonomy grants widen only through evidenced, human-approved certification changes, and narrow automatically on drift or incident. The asymmetry is the safety property. (Figure 13, §16.1.)

A.2 Anti-patterns

The task-tier illusion

Assigning autonomy by task type at design time, then discovering that the risky instances of "safe" task types carry the incidents. The remedy is instance-level state. (§2.1.)

The rubber-stamp spiral

Free escalation floods reviewers. Attention per item collapses. Approval becomes decoration, and the control exists only on the org chart. The remedy is pricing plus attention telemetry. (§2.3, §20.3.)

Fluency as confidence

Consuming verbalised certainty as if it were calibrated probability. The linear-feature results make this a mechanistic error, not a stylistic one. (§5.4, §6.4.)

Outcome-only reward on closable cases

Rewarding closure rates in queues where the cheapest closure is the wrong one: dismissal, skipped verification, degraded success criteria. The remedy is process reward plus honeypots. (§6.3, §10.1.)

The glamour-first roadmap

Deploying the least mature apparatus against the highest-drama, least forgiving problem class because it commands sponsorship, and buying the incident that stalls the programme. The remedy is the sequencing of §19.2.