Deterministic-First Agents: The Certainty Gradient for Governed Banking Automation

The cases combine recurring patterns from regulated-bank delivery. Figures are illustrative and need to be recalibrated for each institution.

A misrouted complaint and a missed deadline

The composite case that crystallised how I place workflow steps was, on paper, a small one. Drawn from recurring patterns in regulated-bank delivery, it concerns a complaint intake pipeline that read every incoming written complaint, interpreted what the customer was upset about, and classified it into a regulatory category before routing it to the right queue with the right service-level clock attached. One of those categories was Regulation E: unauthorised electronic fund transfer.

Under the current CFPB error-resolution rule, an institution generally has ten business days to investigate; if it needs the permitted extension to as long as forty-five days, it must generally provide provisional credit within the initial period, subject to stated exceptions. A missed classification can therefore create customer-remediation and compliance consequences, but the precise reporting and liability outcome depends on the facts and the institution's obligations.

A customer wrote in describing a transaction he had not made, noting that his card had never left his possession. The complaint text used the word "dispute" rather than "unauthorised", mentioned a merchant name the customer recognised from a previous, unrelated billing question, and was, in the way that real customer writing usually is, a little rambling.

The classification step was a single call to a frontier language model, given the full complaint text and a list of about eighteen possible regulatory categories, and asked to pick the best one with a concise rationale linked to cited complaint evidence. It picked "card servicing dispute", a general category with no regulatory clock attached. In the composite, the complaint remained in the general queue until a quality-assurance sample found it after the relevant clock had been missed.

The control issue was then remediation, compliance assessment and reopening a case the workflow had treated as resolved.

The postmortem did not support the expected conclusion. The first instinct was to blame the model: use a stronger one, add examples, or add a verification prompt. The worked reconstruction instead examines the structured information available at classification time: the transaction type already held in the servicing system, the dispute reason selected before the customer wrote anything, and roughly forty phrases that acted as near-deterministic markers for a regulatory category.

Under the composite assumptions, those fields and markers agree with audit-verified labels above 99 percent of the time and resolve 82 percent of volume without open reasoning. A small calibrated classifier handles a further 11 percent. Only the remaining 7 percent contains the mixed or ambiguous cases for which the original reasoning pipeline was actually needed. These percentages are not a benchmark; they show the kind of volume decomposition a team should calculate from its own labelled cases.

The design error was using an expensive, slow and occasionally unreliable reasoning engine for a lookup problem and for the difficult residual, without telling the system which population it was facing. Classification does not have one fixed level of difficulty. Its cases form a distribution. The architect's job is to place each portion as low on the certainty gradient as the evidence permits and reserve open reasoning for the residual.

The certainty gradient defined

Every step in an automated workflow sits somewhere on a line running from fully deterministic, provably correct code at one end, to fully open-ended reasoning at the other. I find it useful to name five zones along that line, not because reality respects clean boundaries, but because naming them forces an explicit conversation about which zone a step is actually in, rather than letting it default to whichever zone the engineer building it happened to reach for.

Zone zero is deterministic code and rules: if-else logic, regular expressions, checksum and reference matching, lookup tables. Given correct inputs the output is provably correct, with no distribution of outcomes, only a distribution of input quality. Cost per call is fractions of a cent, latency single-digit milliseconds. This is where matching a payment reference against an internal case number belongs when the format is structured, the great majority of the time.

Zone one is validated heuristics: hand-tuned scoring rules and weighted combinations of signals, deterministic in execution, with thresholds fitted against historical outcomes rather than first principles. "Auto-approve if the dispute reason code is in this set and the amount is under five hundred dollars" is a zone one rule. It is deterministic once written, but depends on an assumption about the world holding, and needs periodic revalidation, unlike zone zero's logical necessity.

Zone two is calibrated small-model classification: a trained classifier, whether a gradient-boosted tree, logistic regression, or small distilled transformer, outputting a probability rather than a hard rule. The defining property is that the probability has been checked against reality and can be trusted as one, not just a score. A classifier costing a fraction of a cent per call can carry volume a rule engine cannot reach, because the pattern is real but not expressible as a short list of conditions.

Zone three is cascade escalation to a larger model: cases where the zone two classifier's confidence falls into a band neither high enough to trust nor low enough to need a human get a second look from a larger model, often with retrieved context the smaller model had no room for. This zone handles the boundary population and should be a minority of volume by design.

Zone four is full open reasoning with mandatory human review: a frontier model reasons over the case with as much context as it needs, but its output is never auto-actioned, only a recommendation a human signs off. This is where genuinely novel, high-stakes, or irreducibly ambiguous cases belong, and the most expensive zone per case, one that should carry the smallest volume share if the earlier zones are placed correctly.

Naming these zones is not about building a rigid classification system for its own sake. It gives a team a shared vocabulary for a question otherwise answered by default: what zone does this step currently sit in, and is that the lowest zone the evidence actually supports. Most over-engineering I see in banking workflows is a step sitting one or two zones higher than it needs to be, because reaching for a model call felt safer or faster to build than gathering the evidence for a cheaper zone.

Where a step belongs: measuring true ambiguity

The most common architectural mistake I see is confusing assumed ambiguity with true ambiguity. A task looks ambiguous to the engineer designing the workflow because they cannot immediately see the rule that would resolve it. That is a statement about the engineer's visibility into the problem, not about the problem's actual structure. The way to tell the difference is to measure it, not guess it.

The measurement I use is a blind inter-annotator agreement study, run before any classifier or reasoning step is built. Take a representative sample of two to five hundred cases believed to require judgement. Ask two or three domain experts to label them without seeing the workflow output or one another's answers. Then compute Cohen's kappa, or Fleiss' kappa for more raters.

Agreement above roughly 0.8 suggests the task has enough structure for a rule set or calibrated classifier. Low system agreement paired with high expert agreement indicates assumed ambiguity: the humans know the answer, but the system lacks the signal. Expert agreement below roughly 0.5 suggests irreducible ambiguity. That is evidence for zone three or four, not proof that every case needs open reasoning.

In the composite, a retrospective agreement study gives two compliance specialists the raw complaint text and structured servicing fields independently. They agree on category 96 percent of the time, kappa 0.91. That result would indicate low true ambiguity and support redesigning the open-reasoning default. It is an illustrative result, not a threshold to import without local labels.

Ambiguity alone does not set the zone. Two more factors matter as much, weighed together rather than optimised in isolation.

Cost of a wrong answer is the probability of being wrong multiplied by the severity of the consequence, not the probability alone. A 4 percent error rate producing a mildly annoying email is a different risk from a 1 percent error rate producing a missed regulatory deadline. I ask teams to write severity as one of three tiers before arguing about probabilities: operational nuisance requiring rework, customer detriment requiring remediation, or regulatory consequence requiring self-disclosure. A step cannot sit in a low zone purely because its error rate looks acceptable in isolation; the rate has to be acceptable given what a wrong answer costs.

Reversibility of the action is the third factor, and it interacts directly with the first two. A wrong classification affecting only an internal routing queue, correctable the moment a human notices, tolerates a cheaper, faster zone because the system has a second chance built in. A wrong classification that starts a statutory clock, releases funds, or closes an account cannot rely on being caught later, because by the time it is caught the damage is already done.

I treat reversibility as a hard gate rather than a soft factor: if an action is irreversible, or its reversal itself carries regulatory consequence, the step is not eligible for full automation regardless of how good the classifier's numbers look, and it must retain a human sign-off gate, which is regulator-bounded autonomy applied at the level of a single workflow step rather than an entire agent.

Put together, placement is a function of three measured quantities, not one guessed quantity: kappa-measured true ambiguity, severity-weighted cost of a wrong answer, and reversibility of the resulting action. A step with high agreement, low severity, and full reversibility belongs at zone zero or one. A step with low agreement, high severity, and low reversibility belongs at zone four regardless of how well a prototype classifier scores on a small sample, because a small sample cannot be trusted to have surfaced the tail of genuinely hard cases yet.

Calibration: making confidence mean something

Calibrated cascade routing only works if the confidence score at each stage means what it claims. A classifier outputting 0.9 should be right about 90 percent of the time across cases where it outputs 0.9, not 70 percent, not 99 percent. Most teams skip measuring this and trust the reported number, the single most common cause of the overconfidence failures I have had to clean up in production.

The two measurements I insist on before any threshold is set are the Brier score and expected calibration error. Brier score is the mean squared difference between predicted probability and actual binary outcome across a labelled sample, zero is perfect, one is worst. It blends calibration and discrimination, so I never use it alone. ECE buckets predictions into confidence bands, commonly deciles, and compares average predicted confidence in each bucket against observed accuracy there, weighted by bucket size. A well calibrated classifier has an ECE near zero across the range. A model can show a respectable Brier score overall while hiding a dangerous overconfidence tail in one narrow band, and only a bucketed reliability diagram shows that tail.

The sanctions calibration example assumes an overall Brier score of 0.11 and ECE of 6.8 percent before recalibration. A reliability diagram then reveals that the 0.90 to 0.99 band has 98 percent mean predicted confidence but only 87 percent observed accuracy, an eleven-percentage-point gap in the band used for auto-closure. After isotonic recalibration on a held-out set stratified by case type, the worked figures improve to a Brier score of 0.04 and ECE of 1.3 percent. The important control is the bucket-level check; the numbers simply show how an aggregate can hide an overconfident tail.

Setting the auto-route threshold is a policy decision informed by the calibration data, not a fixed constant copied between projects. Plot observed accuracy against confidence threshold using the held-out sample, then pick the threshold where accuracy first exceeds the target set for that step's severity tier: 99.5 percent for a step whose wrong answers carry regulatory consequence, perhaps 97 percent for one whose wrong answers are cheaply reversible. In the complaint classification rebuild, the threshold for auto-routing into a regulatory-clocked category was set at 0.985 or above, where the held-out sample showed 99.6 percent accuracy, comfortably above target with margin for drift.

Calibration is not a one-time exercise. Input distributions shift: a product launch changes complaint vocabulary, a sanctions list update changes base rates, a script change alters how customers phrase requests. I schedule calibration checks monthly using a fresh audit sample drawn independently of the training data, with the auditing team separate from the team tuning thresholds, to avoid the blind spot that lets a model's self-reported token probability pass as a calibrated confidence score. A raw token probability is not calibrated confidence unless checked against outcomes; skipping that check is the fastest way to build an overconfident tail into production.

Calibrated cascade routing mechanics

Calibrated cascade routing turns the certainty gradient from a static diagram into something a live workflow uses on every request. Every request first hits the cheapest stage capable of handling it, escalating to a more expensive stage only when the current stage's calibrated confidence says it should, rather than every request being sent to whichever stage the engineer originally wired up.

Every request enters at the deterministic check, zone zero, because it is nearly free and instantaneous, and there is no reason to skip a stage that costs a fraction of a cent on all volume. If a rule matches with full logical certainty, for instance a payment reference parsing cleanly against the case number format with an exact match, the action is taken immediately and logged with a provenance tag noting which rule fired, supporting audit and feeding provenance-enforced generation further up the workflow. If no rule matches, or the engine flags the input as outside known patterns, the request passes to the calibrated classifier at zone two.

The classifier returns a label and a calibrated probability. If it clears the high threshold, typically 0.95 to 0.99 depending on severity tier, the action is taken automatically. If it falls into a middle band, commonly 0.80 up to the high threshold, the request escalates to the larger model at zone three. The handoff carries the cited evidence, applicable policy results, relevant tool events and calibrated score components, so the larger model and a later human reviewer can see why the smaller model hesitated.

If the larger model's calibrated confidence clears its threshold, the action proceeds. If not, or the case trips an uncertainty flag such as disagreement across independently sampled labels or cited evidence, it escalates to zone four: full reasoning with mandatory human sign-off, nothing auto-actioned from that point.

The cost and latency profile across stages is not linear, closer to an order of magnitude jump at each escalation, which is why volume distribution across stages matters so much to a workflow's economics. If the deterministic and classifier stages are correctly placed and genuinely absorb the bulk of volume, blended cost per case stays close to the zone zero and zone two numbers. If those stages are placed too conservatively, volume backs up into zone three and four by default, and the economics start to resemble a full-reasoning system even though most cases never needed one.

What triggers escalation is worth being precise about. I have seen teams define the trigger as simply "the classifier said it wasn't sure," eyeballing a raw score rather than a calibrated one. The trigger needs three components: a calibrated probability crossing a validated threshold, an explicit disagreement signal where relevant, such as a rule engine and classifier producing different labels for the same case, and a hard override list of attributes forcing escalation regardless of confidence, for instance a vulnerable customer, a politically exposed person, or an amount above a materiality threshold. That last component matters because a high-severity case can, by chance, score high confidence at an earlier stage, and severity should override confidence, not only the reverse.

Concrete failure modes

The failure modes I encounter repeatedly in production banking workflows are specific enough to name, each with a recognisable signature.

Threshold drift is the quietest. A classifier is calibrated correctly at launch, ECE at 1.3 percent, thresholds set with margin. Six months later a shift in customer behaviour has moved the input distribution, and accuracy in the previously safe band has degraded, but the dashboard tracks only overall accuracy or volume, both of which can look healthy while a specific bucket quietly slides. The signature is stable headline numbers until an audit reveals a bucket-level problem accumulating for months. The fix is structural: track ECE by bucket on a rolling basis as a first-class metric with its own alert threshold, not an occasional audit.

Rule explosion is the mirror image on the deterministic end. A rule set can grow from forty rules to several hundred as every edge case receives another branch. The signature is declining comprehensibility and a rising defect rate even as nominal coverage improves. Set a forcing function in advance: crossing an agreed size or complexity measure triggers review of whether the task needs a calibrated classifier instead of another exception.

Cascade short-circuit is a subtler infrastructure failure. In one workflow, a timeout on the zone three call had a fallback defaulting to the highest confidence value, on the reasoning that timeouts were rare and escalating by default would be safe. In practice the opposite happened: an intermittent network issue caused a spike of timeouts over six hours, each silently routed as if returning maximum confidence, and cases that should have gone to human review were auto-actioned instead. The signature is subtle: logs show no errors, only a cluster of maximum-confidence results correlated with an unrelated infrastructure incident. The fix is to make timeout states return the lowest possible confidence, never the highest, and log a distinct "timeout fallback" tag monitored separately from genuine model output.

Provenance loss on escalation happens when a case moves stages and only the raw input and a bare label are carried forward. The reviewer then cannot see the observable basis for hesitation and effectively starts from zero, sometimes reproducing the same misjudgement because the same surface features mislead a person skimming under caseload pressure. Preserve the cited evidence, policy decisions, tool events, calibrated score components and a concise decision rationale at every escalation. Assurance should not depend on hidden chain-of-thought: the durable record is what the system observed, checked, proposed and did. That evidence-preserving handoff is what makes a cascade a cascade rather than a series of disconnected retries.

Confidence laundering is the failure that started the opening incident, generalised. A raw model output, a token probability or an unvalidated heuristic score, is treated as calibrated confidence without the bucketed reliability check described earlier. The signature is a workflow reporting high confidence alongside a surprising override rate never reconciled against the confidence claimed. Any number gating an automated action needs to be checked against ground truth before it is trusted as a probability, not merely produced by a process that outputs something that looks like one.

Worked example: re-placing a step in a KYC refresh workflow

A periodic KYC refresh workflow is a good test case because it spans a genuine range of difficulty, from pure arithmetic to genuinely hard judgement calls, and one specific step in it is a close cousin of the complaint classification incident.

Step one, trigger detection, decides whether a customer is due for a periodic refresh based on risk tier and last completed refresh date. Pure date arithmetic against a lookup table of refresh cycles by tier, firmly zone zero, with no ambiguity to measure at all.

Step two, document completeness check, confirms every required document for the customer's type and jurisdiction is present, checked against a document requirement matrix. A checklist match, zone zero, with a small zone one component where acceptable substitutions exist, for instance a utility bill in place of a bank statement, governed by a validated substitution table rather than free judgement.

Step three, document data extraction, pulls structured fields such as name, date of birth, and address from scanned passports and utility bills. Optical character recognition has a genuine, measurable error rate that varies by scan quality, making this a legitimate zone two case: a calibrated model, or an OCR engine paired with a calibrated confidence layer, extracts each field and flags anything below threshold for a quick human glance rather than blind trust.

Step four, sanctions and politically exposed person alert adjudication, is the one that had been placed wrong. Before re-placement, every screening hit went to a full open reasoning call: the model read the hit details and the customer's profile and produced a match or no-match recommendation. At forty thousand alerts a month, at roughly $0.20 per call and six seconds median latency, the step cost about $8,000 a month and added queue time.

What forced a rethink was the override rate: analysts overrode the model 61 percent of the time, a sign it was an expensive rubber stamp rather than a genuine judgement engine. A retrospective audit found that on the 85 percent of hits resolvable by deterministic transliteration and date-of-birth rules already used elsewhere in the screening engine, model accuracy was statistically indistinguishable from those rules, meaning it was doing expensive work to reach answers the deterministic layer already had.

In the worked model, 82 percent of alerts resolve through zone-zero and zone-one matching at 99.7 percent retrospective precision. A further 9 percent, mainly cross-script transliteration cases, go to a calibrated classifier with 96 percent agreement against adjudicated labels. The remaining 9 percent escalate to mandatory human review. The modelled blended cost falls from twenty cents to roughly three cents per alert. These values make the allocation testable; they are not promised production outcomes.

Step five, source of funds and source of wealth narrative assessment for higher-risk customers, correctly belongs at zone four and should stay there. Assessing whether a stated narrative about the origin of wealth is plausible given transaction history genuinely requires judgement that resists rules, the cost of a wrong answer is severe, and reversibility of a wrong onboarding decision is low. This is not a step to push down the gradient, it is the step the whole cascade exists to protect, keeping its volume small enough that human reviewers can give each case real attention rather than being buried under alerts that never needed them.

Step six, risk rating recalculation, recomputes the customer's overall risk score from updated inputs using a weighted formula already validated by model risk governance, zone zero. Step seven, scheduling the next review date and closing the case, is the same.

Under the stated volume and unit-cost assumptions, re-placing step four reduces monthly automation cost by roughly $6,760 and cuts median processing time for the bulk flow from six seconds to fifteen milliseconds. The business case should therefore expose its volume, unit-cost, error and review assumptions rather than present one benefit figure without a denominator.

The deterministic envelope

The certainty gradient becomes more useful when it is treated as an architectural envelope rather than as a routing diagram. An agent does not need to be deterministic in every internal operation. It needs a deterministic envelope around the parts of its behaviour that create obligations, move value, change a system of record, represent the institution to a customer, or determine who receives a regulated service. The envelope specifies what the agent may observe, which decisions it may recommend, which actions it may attempt, which policy checks must pass, which evidence must be written, and which states require human approval.

This distinction matters because “deterministic-first” is easily misunderstood as an argument for replacing language models with rule engines. It is not. The language model remains valuable precisely where the task contains linguistic variation, incomplete information, competing hypotheses or the need to generate a coherent explanation. Deterministic-first means that an architect does not spend probabilistic reasoning on facts a system already knows, permissions an identity service can decide, thresholds a policy service can evaluate, calculations a conventional program can perform, or process transitions a workflow engine can enforce. The model reasons inside a structure whose load-bearing members do not depend on the model remembering to behave.

The envelope has six boundaries.

The principle is intentionally asymmetric. A deterministic service may veto a model-proposed action. A model should not override a deterministic denial merely because it can generate a plausible explanation. If an override is a legitimate business requirement, the override is a separate, named action with its own authority, reason code and approval path. Otherwise an organization has not created an exception process; it has allowed natural-language persuasion to bypass policy.

The architecture makes the model one participant in a decision, not the decision boundary itself. That is a stronger design than asking the model to “follow the policy,” because policy compliance is observed at the point of enforcement rather than inferred from an answer. It also makes model substitution easier. A stronger or cheaper model can be introduced without silently changing identity, authority, state management or evidence semantics, because those contracts remain external to the model.

From user journey to decision inventory

Banks usually begin agent programmes with journeys: resolve a servicing request, refresh KYC, prepare a credit memo, investigate a fraud alert, or help a relationship manager prepare for a meeting. Journeys are the right unit for business value but the wrong unit for assigning autonomy. A journey contains many decisions with different ambiguity, materiality and reversibility. “Automate the KYC refresh” is therefore not an actionable autonomy decision. “Extract the expiry date from a passport,” “decide whether the evidence satisfies the jurisdictional requirement,” and “approve the risk-rating change” are three different decisions and should sit at different points on the certainty gradient.

The practical design move is to turn the journey map into a decision inventory. For each decision, record the input state, authoritative sources, allowed outputs, consequence of error, reversibility, expected ambiguity, evidence requirement, owner and escalation destination. Only then select a mechanism.

The inventory exposes a common pattern: most volume is structurally simple, while most risk is concentrated in a thin residual. That residual deserves the strongest reasoning and review. It does not follow that the same expensive mechanism should process the simple majority. A well-designed agentic journey is often a conventional workflow with several narrow islands of probabilistic interpretation, not a language model controlling a conventional workflow from above.

This is consistent with the direction of current banking analysis. McKinsey’s 2026 discussion of consumer financial agents identifies intent, identity and liability as central friction points once an agent can act rather than merely advise. Those are not prompting concerns. They are boundaries in the deterministic envelope. The bank needs to know whether the customer intended the transaction, whether the agent has valid authority, and which party bears responsibility if execution produces harm. The architecture must answer those questions before an external action is made available to a model.

The autonomy budget

Autonomy is frequently discussed as a level attached to an application: assistive, semi-autonomous or autonomous. That is too coarse for production governance. An agent may be safe to search broadly, summarize moderately, recommend within a constrained set, and execute almost nothing. The useful unit is an autonomy budget assigned to an action class in a particular context.

An autonomy budget has four dimensions:

1. Scope: which resources, customers, products and jurisdictions may be touched.
2. Magnitude: the maximum financial, operational or customer consequence of one action and of cumulative actions over a period.
3. Duration: how long delegated authority remains valid before reauthorization.
4. Uncertainty: the minimum evidence and calibrated confidence required for the action class.

An additional risk multiplier captures coupling: the number of downstream systems or decisions that consume the action. Updating an internal draft has limited coupling. Writing a risk rating into a system used by pricing, monitoring and regulatory reporting has high coupling even if the field itself looks small.

The budget is consumed by actions, not by tokens. Repeated low-value actions can create high cumulative exposure, so the gateway should meter both per-action magnitude and rolling exposure. An agent permitted to issue a goodwill credit below a small threshold may still create unacceptable loss if it can issue the credit thousands of times without a daily cap. The budget therefore resembles a combination of transaction limit, rate limit and risk appetite, enforced at the action gateway.

This model also resolves a persistent governance argument: whether human-in-the-loop is always required. The correct answer is not universal. Human approval should be attached to the action classes whose consequence exceeds the machine principal’s budget, not inserted performatively into every step. Requiring a human to click through thousands of low-risk, high-confidence cases produces automation bias and queue pressure. Removing humans from high-coupling, irreversible decisions because the average benchmark score is impressive produces a different but more serious failure. The action budget makes the dividing line explicit and reviewable.

Policy as an executable contract

Natural-language policies remain essential because laws, standards and business rules are written for people. They are insufficient as the enforcement mechanism for an acting agent. A production system needs an executable policy contract that translates relevant obligations into machine-testable predicates while retaining the source clause, owner, effective date and exception process.

A policy decision should return more than allow or deny. It should return a structured record:

decision_id
policy_version
principal_id
customer_or_case_id
action_type
resource_scope
purpose
input_evidence_ids
result: permit | review | deny
reason_codes[]
obligations[]
approver_role_if_required
expiry

Obligations are especially important. A policy may permit an action only if a disclosure is included, a second source is checked, an approval is obtained within a time window, or the customer is notified through a specified channel. Returning these obligations as data lets the workflow prove completion instead of relying on the model to remember a sentence embedded in a policy document.

The contract should be evaluated twice. The first evaluation occurs before the tool is exposed or called, using the proposed action and current context. The second occurs immediately before commitment, using the fully populated parameters, current authority and any approval. This protects against the gap between an abstractly acceptable plan and a concretely unacceptable transaction. “Arrange a transfer” may be permitted in principle; a transfer of a particular amount to a newly created beneficiary may require review.

Short-lived capability tokens are preferable to giving the agent a durable credential. The policy service issues a capability for one specific action, resource and expiry after checks pass. The gateway verifies the capability independently. If the agent’s prompt, memory or model is compromised, it cannot expand the token’s scope through a persuasive tool call. This is the practical difference between asking an agent to exercise restraint and making excess authority unavailable.

Failure semantics are part of the architecture

Many agent designs specify the successful path in detail and treat failure as an implementation concern. In banking, failure semantics are a control design. Every external action needs a declared response to timeout, duplicate submission, partial completion, stale state, conflicting approval, policy-service unavailability and downstream rejection.

The safest default is not always “fail closed.” A blanket failure can itself create harm if it prevents a time-critical regulatory action or customer protection. The design needs decision-specific fail states:

The state-version check is particularly important for long-running agent tasks. The agent may begin with a customer state snapshot, spend minutes gathering documents and reasoning, and then act after the balance, risk flag or case ownership has changed. The action gateway must compare the version used for the decision with the current version. A mismatch is not automatically an error, but it invalidates the decision’s evidential basis and forces reconstruction or review.

Compensation should also be modeled as a first-class workflow, not improvised after an incident. For each reversible action, define the inverse, the maximum compensation window, the owner if automated compensation fails, and the evidence linking original and compensating transactions. For irreversible actions, label them as such and set the autonomy budget accordingly. Architecture diagrams often show a neat arrow from tool call to success. Production diagrams need the arrow back.

A production reference architecture

The following reference architecture separates the probabilistic plane, deterministic control plane, enterprise data plane and evidence plane. The separation is logical rather than vendor-specific; services may be implemented together while retaining independent contracts.

The orchestrator coordinates reasoning but does not hold unrestricted enterprise credentials. The context assembler obtains an authorized, decision-specific view of enterprise state. The model gateway records model and configuration identity, applies routing policy and normalizes provider differences. The verifier can challenge claims or plans, but verification never replaces policy enforcement. The workflow engine holds durable state, timers, retries and compensation. The tool gateway is the only route to side effects. The evidence plane joins the proposal, context snapshot, model configuration, policy decisions, approvals, tool parameters and observed effect into one replayable record.

This structure aligns with where hyperscaler guidance is converging. AWS’s 2026 prescriptive guidance separates use-case maturity, governance and enterprise architecture, and emphasizes registries, access control and audit as deployments become customer-facing. Google Cloud’s 2026 production-agent guidance similarly emphasizes long-running state, delegated approvals, identity, registry, gateway controls, anomaly detection and orchestration patterns. Microsoft’s current agentic maturity guidance emphasizes environment separation, managed identities, governed connectors, inventories, reusable components, observability and evaluation. The services differ; the control responsibilities are remarkably stable.

Applying the pattern across banking domains

The deterministic-first split changes by domain, but the method does not.

In credit, for example, a model can interpret management commentary and identify inconsistencies, but it should not calculate ratios from prose if the financial statements are already structured. It can propose risk factors, but policy thresholds and exposure aggregation should be computed outside the model. It can draft a recommendation, but approval authority remains bound to a named mandate. The result is not less intelligent. It is intelligence concentrated where interpretation adds value.

In financial crime, the same distinction prevents a common category error. Entity resolution is probabilistic because names, dates, addresses and networks can conflict. But list version, customer identity records, prior dispositions, materiality rules and case status are deterministic facts. The agent may assemble and weigh evidence, while the final disposition path is constrained by policy, confidence, case type and accountable review.

In servicing, a customer’s intent is sometimes ambiguous, but product eligibility, transaction limits and complaint-service clocks are not. A model may ask a clarifying question or explain the consequence of an option; it should not invent eligibility or calculate authority from conversation history. The clean split improves customer experience because the agent can be flexible in language while being exact in obligation.

Evaluation for a deterministic-first estate

Evaluation must reflect the mixed architecture. A single “agent accuracy” number hides whether the rules, classifier, reasoning, policy or execution layer failed. The scorecard should be decomposed along the same boundaries used in design.

Selective risk is more informative than average accuracy for cascade systems. It asks: among the cases the system chose to automate at a threshold, what error rate occurred? Coverage asks what share of cases crossed that threshold. Plotting selective risk against coverage reveals the actual trade-off between autonomy and quality. A system that automates 95 percent of volume at 92 percent accuracy is different from one that automates 70 percent at 99.7 percent accuracy, even if both report an attractive aggregate benchmark.

The evaluation set should be stratified by severity and boundary conditions, not sampled only in proportion to volume. Rare vulnerable-customer cases, new beneficiaries, cross-border conditions, sanctions matches and policy exceptions may be statistically small but operationally decisive. A risk-weighted suite deliberately over-represents these cases and reports them separately. Passing the average does not excuse failing the tail.

The engagement model: from demonstration to governed production

Consulting programmes often separate strategy, platform and use-case delivery into different workstreams. Agentic AI punishes that separation when decision rights and evidence are left until late. A twelve-week path can keep them connected without pretending that every journey will reach unrestricted production in one quarter.

Weeks 1–2: decision and risk framing

Select one end-to-end journey with measurable value. Build the decision inventory, identify systems of record, classify action consequences, define the accountable owner and agree the initial autonomy budget. Establish baseline cost, cycle time, quality, loss and review effort. The output is not a generic use-case scorecard; it is a signed statement of which decisions may be automated under what evidence.

Weeks 3–4: deterministic spine

Implement or expose the workflow state machine, identity, policy, tool contracts, source queries, idempotency and evidence schema. This may look slower than starting with a conversational prototype. It is usually faster than retrofitting controls after stakeholders have become attached to a demo architecture that cannot be approved.

Weeks 5–6: residual intelligence

Add models only to the sub-decisions whose ambiguity justifies them. Establish routing and fallback. Build a small but risk-stratified evaluation set before tuning prompts against it. Separate development examples from final validation cases.

Weeks 7–8: trajectory and failure testing

Test tool parameters, action order, stale state, approval expiry, retries, duplicate events, policy unavailability, hostile content, partial completion and compensation. Rehearse both safe refusal and safe degradation. A system that works only while every dependency is healthy is not production-ready.

Weeks 9–10: shadow and controlled release

Run against live or production-representative traffic without consequential action. Compare recommendations with human outcomes, inspect disagreements, recalibrate thresholds and measure review burden. Then introduce a narrow reversible action class with a low autonomy budget and explicit kill switch.

Weeks 11–12: operating handover

Deliver the evaluation suite, control catalogue, runbooks, ownership map, change classification, evidence pack, service objectives, cost model and next-budget recommendation. The programme succeeds when the bank can operate and extend the system without depending indefinitely on the original build team.

The dates in the diagram are illustrative anchors, not a claim about a particular engagement. The point is sequencing: earn action rights after the deterministic spine and assurance evidence exist, not before.

Operating model and accountability

The architecture requires a corresponding operating model. A central AI platform team can own shared identity patterns, model gateways, agent and tool registries, observability, evaluation infrastructure and evidence schemas. Domain teams own journeys, source semantics, policy interpretation, outcome metrics and day-to-day exception handling. Independent risk and validation functions define challenge expectations and review high-consequence changes. Cybersecurity owns threat models and control assurance. Operations owns service recovery. The accountable business executive owns the decision to automate a specific action class.

The most effective pattern is federated delivery on a governed platform. Pure centralization turns the platform team into a bottleneck and distances design from domain reality. Pure federation recreates identity, tool access, logging and evaluation differently in every journey. The shared platform should make the safe path faster: a pre-approved typed tool contract, evaluation harness and evidence pattern should reduce delivery work, not add a compliance ceremony after the build.

Change control should follow impact rather than component type. A prompt change can be material if it changes action selection. A model change can be minor if it is proven equivalent for a bounded extraction task. A policy threshold change can be more consequential than either. Classify changes by potential effect on decision boundaries, action rights, evidence semantics and population, then attach the appropriate validation depth.

The forum should not approve “the agent” once. It should approve populations, action classes, evidence thresholds and change envelopes. That lets the system evolve without repeating a full approval for every low-impact improvement while preserving explicit control over expansions of autonomy.

What executives should ask before funding scale

Senior leaders do not need to adjudicate model libraries, but they should be able to ask questions the architecture can answer precisely.

1. Which parts of this journey are genuinely ambiguous, and how was that measured?
2. What is the highest-consequence action the system can take without a person?
3. Which deterministic service can veto the model, and can the model bypass it?
4. What current state does a decision depend on, and how do we prove the state had not changed before execution?
5. What evidence joins the request, context, model, policy decision, approval, tool call and effect?
6. How does the system behave when the model, policy service, source system or evidence store is unavailable?
7. What share of volume is automated at each observed error rate, including severe tails?
8. Which change can expand customer or regulatory exposure, and who approves it?
9. What capability is reusable in the next journey, and what remains domain-specific?
10. Can the institution operate, test and unwind the system without the delivery partner?

These questions shift governance from abstract declarations of responsible AI to observable architecture. They also improve the economics. When simple volume is processed by deterministic and calibrated components, expensive reasoning is reserved for cases where it changes the decision. When tool contracts, evidence and policy are reusable, the second journey starts further ahead than the first. The compounding asset is not a collection of prompts. It is a governed delivery spine.

The placement decision record

A workflow map says where a step runs today. It does not preserve why the step was placed there or what evidence would justify moving it. For a consequential process, that reasoning belongs in a placement decision record: a small, versioned control artefact attached to the workflow step and reviewed whenever its evidence changes.

The record starts with the decision population, not the model. It names the case type, inclusion and exclusion rules, expected monthly volume, severity tiers and the downstream action. This prevents a familiar testing error: demonstrating strong performance on the common, reversible population and then silently applying the same threshold to rare cases with a different consequence. A complaint classification step may be one box on a process diagram, but complaints involving vulnerability, fraud or a statutory clock may require separate placement decisions.

Next comes the evidence for the present zone. For a deterministic rule, retain its authoritative input fields, rule tests, exception rate and source ownership. For a classifier, retain the labelled population, inter-rater agreement, calibration plot, selective-risk curve and slice results. For a reasoning step, retain the ambiguity evidence, evaluation scenarios, human-review design and the reason a lower zone failed. “The task needs judgement” is not sufficient evidence; the record should show where experts disagree or where lower-zone methods fail on a representative sample.

The record should also distinguish the prediction from the action. A model may predict a category with 99 percent calibrated confidence while the action remains ineligible for automation because it is irreversible or creates a regulatory clock. The placement decision therefore records two thresholds: the confidence at which the prediction is accepted, and the policy condition under which the corresponding action may proceed. They are related, but they are not the same control.

Every material change is evaluated against this record. A model update matters, but so does a new complaint form, a renamed source-system field, an acquisition that changes the customer population, or a policy revision that alters materiality. If the change touches an assumption used to place the step, the step returns to shadow mode until the affected evidence has been refreshed. This is more precise than sending every change through the same governance ceremony and safer than treating non-model changes as operational trivia.

Re-placement should use a challenger period rather than an immediate cutover. Run the proposed lower zone beside the current path without acting on its output. Compare coverage, selective error, severe-tail performance, subgroup behaviour, latency, cost and escalation quality. The important number is not merely how much volume the challenger absorbs. It is how much it absorbs within the agreed risk bound, including cases the current path gets right for reasons the challenger cannot reproduce.

The record also needs an expiry. A zone-two classifier supported by last year's sample is not automatically still a zone-two control after vocabulary, base rates and source data have moved. High-consequence steps may need quarterly evidence review; stable, reversible internal routing may justify a longer cadence. The interval should follow the rate and consequence of change rather than an enterprise-wide calendar chosen for administrative convenience.

This artefact improves executive discussion because it converts “Can we automate more?” into a narrower question: which population has earned a different placement, on what evidence, with what retained veto and rollback? It also improves delivery discipline. A future team can see why a threshold exists, what would invalidate it and which unresolved cases were deliberately left for human judgement. Without that record, a sound architectural decision gradually becomes an unexplained number in configuration, and unexplained numbers are eventually copied into places where their original evidence does not apply.

Sources and limits

The architecture draws on public research, current industry guidance and experience designing systems for regulated financial institutions. The examples combine recurring delivery patterns rather than describe one named client. Thresholds and cost figures require validation against each institution’s data, policy and risk appetite.

The wider market direction is supported by several current sources. The World Economic Forum’s 2026 AI Playbook for Financial Services frames the next stage around strategy, data and technology foundations, governance, workforce and responsible scale. KPMG’s 2026 Global AI in Finance report argues that value is concentrating where governance, measurement, assurance and workforce are built into the operating system. McKinsey’s 2026 banking work emphasizes rewiring end-to-end work rather than accumulating pilots. AWS’s enterprise guidance, Google Cloud’s production-agent architecture series, and Microsoft’s agentic maturity guidance provide implementation-specific evidence that identity, registries, controlled integration, lifecycle management, evaluation and observability are becoming shared production concerns.

The thesis goes one step further: those controls should not surround a fully probabilistic process as an afterthought. They should determine which parts of the process are probabilistic at all.

Notes for practitioners

Run the inter-annotator agreement study before writing a line of classifier or prompt code, budgeting two to three real weeks including expert availability. Treat a kappa below 0.5 as your only legitimate justification for defaulting a step to full open reasoning; treat anything above 0.8 as a mandate to keep looking for the deterministic or calibrated solution even if the first attempt fails.

Instrument every cascade stage with its own confusion matrix from day one, broken out by case type, not just an end-to-end accuracy figure. An end-to-end number can look healthy while hiding a stage quietly wrong on a specific slice, invisible unless each stage is measured separately.

Put the high-confidence-wrong rate on the same dashboard as overall accuracy and ECE, with its own alert threshold, rather than folding it into a blended metric. This number catches an overconfident tail before it causes an incident, and behaves quite differently from an average error rate, so averaging it away defeats the point of tracking it.

Recalibrate on a fixed cadence, monthly for high-volume steps, using an audit sample labelled by a team that had no hand in setting the thresholds being checked. Independence matters more than frequency; a monthly check performed by the people who tuned the model tends to confirm what they already believe rather than surface what has actually drifted.

Set a rule-count trigger in advance for every zone zero or one step, a number at which crossing it forces a review of whether the step belongs elsewhere, rather than simply adding the next rule. I use one hundred and fifty as a rough default and adjust by domain, but the number matters less than agreeing one before growth starts, because that is exactly when the temptation to add one more rule is strongest.

Keep a placement register: a living document, reviewed at the same cadence as your model risk governance process, listing every workflow step, its current zone, the evidence that put it there, the date last reviewed, and who owns the decision. Without this, placement decisions live only in the memory of whoever built the step, and re-placement never happens because nobody remembers there was a decision to revisit.

Separate model confidence from action confidence explicitly. A calibrated probability tells you how often a prediction is correct; it does not by itself tell you whether that error rate is acceptable for a given action. Multiply the calibrated probability of error by the severity tier of the action before setting an auto-action threshold, and use different thresholds for different severity tiers rather than one number copied across the whole workflow.

When moving a step down the gradient toward more automation, stage the change as a shadow deployment first: run the cheaper zone in parallel with the existing zone for four to six weeks, or enough volume to see the tail cases, without acting on its output, then compare before cutting over. This is the same discipline adversarial twin verification applies to individual outputs, applied instead to a whole re-placement decision, the difference between evidence-based change and one merely hoped to be safe.