A credit memo is not one task
A commercial credit paper may look like one document, but it is the visible result of several activities with different evidence standards. Financial statements are extracted. Figures are normalised and reconciled. Policy is selected for the customer, product and date. Risks are compared with mitigants. A narrative is drafted. An authorised person decides whether the proposal should proceed and on what terms.
The unsafe design treats the entire chain as summarisation. Documents enter a model context and a polished memo emerges. That approach hides whether a number was read, calculated or inferred. It makes a policy exception look like a writing detail. It also invites the drafting model to become the practical decision maker, even when the user interface still contains an approval button.
The architecture should split the work wherever the meaning of “correct” changes. Extraction is correct when a value is faithfully located. Arithmetic is correct when a reproducible calculation balances. Policy treatment is correct when the applicable rule and exception path are used. Narrative is correct when it represents the admitted evidence without altering it. A credit decision is correct only within the institution's delegated authority, risk appetite and governance.
The five-lane control model
The five lanes are not five agents by default. They are five accountability domains. A lane may contain deterministic services, retrieval, conventional models, language models and human work. The separation matters because each lane needs a different contract and failure response.
| Lane | Primary question | Suitable machinery | Required evidence | Authority boundary |
|---|---|---|---|---|
| Evidence | What did the source actually contain? | OCR, document parsing, schema extraction, source retrieval | Page, region, system record, version, quality state | Cannot infer a credit conclusion |
| Arithmetic | Do the numbers reconcile under the approved definition? | Code, rules, calculators, controlled spreadsheets | Inputs, formula version, units, exceptions, cross-footing result | Cannot change accounting treatment silently |
| Policy | Which rule applies to this entity, product, purpose and date? | Policy service, codified rules, governed retrieval | Policy version, applicability attributes, cited clause, exception route | Cannot approve an exception |
| Narrative | How should admitted facts and analysis be expressed? | Template engine, language model, style validator | Claim-to-evidence map, protected values, unresolved gaps | Cannot create facts or change a verdict |
| Decision | Should the exposure be approved, declined, conditioned or escalated? | Delegated human workflow supported by approved tools | Complete pack, recommendation, exceptions, approval identity | Reserved to authorised credit roles |
This model is deliberately different from a chain of specialist personas that talk to one another in free text. A “financial analyst agent” can still mix source reading, calculation and judgement. The control model instead defines typed outputs. The evidence lane emits propositions and gaps. The arithmetic lane emits calculations and failed invariants. The policy lane emits applicability and exceptions. The narrative lane emits a draft whose material claims point back to those objects.
A lane can fail without forcing the whole case into an invented answer. A table extraction can route to review while the policy search continues. A missing market benchmark can remain an explicit gap. A policy conflict can stop recommendation generation without discarding completed financial analysis.
Evidence must survive the document pipeline
Commercial lending uses annual accounts, management accounts, bank statements, facility letters, valuations, ownership records, covenant schedules and correspondence. A PDF is only a container. It may hold scanned pages, selectable text, tables, signatures, annotations and embedded spreadsheets. The system should preserve those forms before reducing them to text.
Each extracted value needs a source pointer, extraction method, unit, reporting period, currency, sign convention and quality state. “Revenue 14.2” is not a usable proposition. The system needs to know whether that means 14.2 million pounds for the year ended on a stated date, whether it was restated, whether it came from audited accounts and which table cell supports it.
The W3C PROV-O recommendation offers a useful vocabulary for entities, activities and agents in a derivation chain. It does not define banking evidence sufficiency, but it helps distinguish a source page from an extraction activity and a normalised proposition. NIST AI 600-1 also treats provenance, content integrity, privacy and testing as linked risk concerns for generative systems.
| Evidence state | Meaning | Permitted use | Required next action |
|---|---|---|---|
| Observed | A source contains the stated mark, text or value | Display with source pointer | Validate meaning and context |
| Extracted | A parser or model converted the observation to a field | Candidate for mapping | Check schema, unit and confidence |
| Reconciled | Cross-footing, identity and period checks pass | Approved calculation input | Retain reconciliation receipt |
| Conflicted | Two admitted sources disagree | Show both; block silent selection | Route to named reviewer or precedence rule |
| Missing | Required proposition not found in authorised sources | Preserve as a gap | Request evidence or apply approved fallback |
| Inferred | A model or analyst derived a proposition | Use only for the stated analytical purpose | Keep distinct from source fact |
A confidence score should not erase an evidence state. A model can be highly confident about the wrong period or a subtotal presented as a total. Deterministic checks and reviewer rules operate on the extracted object, not on confidence alone.
Arithmetic belongs outside the language model
Financial spreading is not merely copying tables. It includes sign normalisation, period alignment, currency conversion, reclassification, consolidation treatment, exceptional items and reconciliation across statements. Those choices must be explicit. Once a mapping is approved, code should reproduce it.
The calculation service accepts only reconciled propositions or an authorised exception. It returns the exact input IDs, formula version, result, rounding rule and validation outcomes. Ratios such as leverage or debt-service coverage need a defined numerator and denominator. The phrase “standard leverage” is not a definition.
The distinction is reinforced by the April 2026 US interagency model risk guidance. The revised guidance defines its model scope carefully and states that generative and agentic AI are outside that scope, while other governance should still determine appropriate controls. That exclusion should not be misquoted as permission to treat agentic systems casually. It means the institution needs a system-level governance design rather than assuming one familiar model-validation document covers the whole workflow.
Stress and sensitivity analysis should also remain reproducible. A model can propose scenarios from admitted evidence, but an approved engine calculates their effects. The reviewer sees the assumptions separately from the results. If the model suggests a margin decline because a source reports supplier pressure, that relationship is a hypothesis until accepted. The calculation does not smuggle it in as a fact.
Policy selection is a compilation problem
Credit policy is not one searchable manual. Applicability may depend on legal entity, booking location, segment, product, facility type, amount, collateral, purpose, jurisdiction and effective date. Exceptions may have separate authority. A semantic retriever can find a similar paragraph while missing the rule that governs the case.
The policy service should compile a case envelope before retrieval. It resolves the relevant entity and product identifiers, decision type, dates and delegated role. Deterministic predicates select the permitted policy set. Retrieval then locates clauses within that set. A language model may help translate a user question into approved search concepts, but it cannot widen scope or choose an obsolete version.
| Policy outcome | What the system may say | What it must not say |
|---|---|---|
| Applicable and satisfied | The cited condition is met by named evidence | The proposal is approved |
| Applicable and not satisfied | The cited condition is not met | The customer is unsuitable in general |
| Exception possible | The policy provides an exception route and authority | The exception will be granted |
| Conflicting sources | Two admitted policy artefacts disagree | The model chose the one that “looks current” |
| No applicable rule found | Search scope, sources and gap are recorded | No policy exists |
| Out of authority | The requested determination requires a different role | A draft workaround that bypasses delegation |
Policy codification should start with high-value, stable decisions where definitions and exceptions can be made explicit. It should not attempt to convert every paragraph into brittle executable rules. Some clauses are evidence for human interpretation. The architecture can still record the clause, relevant facts and unresolved question without pretending the interpretive work is deterministic.
Narrative generation is a constrained projection
The drafting stage should operate on a protected case model, not a folder of raw documents. The case model contains admitted propositions, calculations, policy outcomes, gaps, conflicts and reviewer annotations. The draft template defines required sections and permitted speech acts. Each material sentence must either point to evidence or be labelled as analysis, proposal or reviewer judgement.
The generator should not be asked to “make the case persuasive.” Credit papers need clarity and proportionality, not advocacy disguised as analysis. Strengths and weaknesses should be derived from the same evidence model. Missing information should remain visible. If a risk conclusion depends on judgement, the text should name the accepted judgement rather than simulate certainty.
Recent finance-agent benchmarks support caution. Finance Agent Benchmark found substantial limitations on research tasks even with search and filings tools. FinToolBench moves evaluation toward executable financial tools and regulatory alignment. BankerToolBench reports that cross-artifact consistency remains a major obstacle on long professional workflows. These studies do not test a commercial credit approval process, but they challenge the assumption that a capable model plus tools produces client-ready work.
The draft is a view over governed case state. It is not the case state itself. A reviewer can change wording without changing the calculation. A corrected source invalidates affected propositions and draft claims. The system can then regenerate only the impacted sections while retaining the decision trail.
Human authority must be more than a button
A nominal approval screen does not preserve human authority if the reviewer cannot inspect the material evidence, if the system anchors them with one confident recommendation, or if workload makes approval automatic in practice. The control must address identity, time, competence, information and the ability to disagree.
The reviewer receives a compact evidence pack: the proposal, material calculations, policy treatment, exceptions, conflicts, missing evidence and changes since the previous review. The interface foregrounds conditions requiring judgement. It allows the reviewer to approve, decline, request evidence, change an analysis conclusion or escalate. Every override has a reason and retains the machine proposal rather than overwriting history.
| Review control | Weak implementation | Decision-grade implementation |
|---|---|---|
| Identity | Shared team login | Named role, delegation and workload identity |
| Evidence | Memo only | Openable sources, calculation receipts and policy clauses |
| Time | Approval after an unknown delay | Evidence cut-off, freshness state and changed-item alert |
| Choice | Approve or reject model recommendation | Independent outcome, conditions, evidence request and escalation |
| Workload | Queue size ignored | Capacity, ageing, concentration and sampling monitored |
| Override | Free-text edit replaces history | Original proposal, changed field, reason and approver retained |
The FCA's July 2026 Mills Review expects AI to reshape consumer journeys and firm operations, while emphasising risk-tiered approaches and existing accountability frameworks. Commercial credit is a different domain, but the underlying point carries: automation level should follow consequence, evidence and accountability rather than the broad label “agentic.”
Evaluation follows the five lanes
One end-to-end accuracy number is insufficient because it cannot locate failure. Evaluation should run at component, trajectory and case-pack levels. It should include negative cases where the right result is abstention, conflict or escalation.
| Test family | Example measure | Consequential failure |
|---|---|---|
| Evidence | Field accuracy by document class; page-region citation accuracy | Wrong period admitted as current evidence |
| Arithmetic | Cross-foot pass; formula equivalence; protected-value integrity | Narrative amount differs from calculation receipt |
| Policy | Applicability precision; supersession handling; exception recall | Retired clause used for current case |
| Narrative | Claim support; contradiction coverage; speech-act compliance | Recommendation represented as approval |
| Trajectory | Tool choice, stop condition, retry and escalation | Missing evidence bypassed through repeated generation |
| Human workflow | Review time, override quality, automation bias indicators | Material exceptions routinely approved unread |
Evaluation data should be partitioned by product, document form, customer type, language, exception class and decision consequence. Aggregate improvement can hide regression on a rare but material segment. Production monitoring should track evidence gaps, calculation rejects, policy conflicts, blocked claims, overrides and post-decision corrections. Drift in tool choice or evidence coverage may appear before final decision metrics move.
Operating the credit architecture
The platform team owns shared identity, evidence, orchestration, evaluation and observability services. Domain owners define financial concepts, policy applicability, exception routes and decision authority. Credit operations own the reviewer workflow and service levels. Model risk, compliance, security and data governance provide independent challenge within their mandates. No central AI team should silently become the owner of credit meaning.
Release units should remain separable. A new extraction model can be validated against document classes without changing policy. A policy update can be published with impact analysis and regression cases. A new language model can be tested for claim support while protected calculations remain fixed. This reduces the blast radius of change.
The business case should count accepted time saved, not generated text. Useful measures include time to assemble complete evidence, first-time-right financial spreads, reduction in duplicate retrieval, policy-search resolution, reviewer rework, queue ageing and decision-pack defects. Benefits must be measured against review, exception handling, model and platform cost, control operation and incident response.
The operating model becomes easier to test when it is applied to a complete case rather than discussed as five abstract lanes.
A worked case from intake to decision
Consider a fictional manufacturer seeking an increase to an existing revolving facility. The bank receives audited accounts, current management accounts, a borrowing-base statement, a forecast, an ownership update and a short business plan. Live systems provide existing exposure, payment behaviour and the current facility state. The case is intentionally ordinary. That is where architecture quality matters because volume and familiarity can make shortcuts feel harmless.
At intake, the case service verifies document identity, legal entity and requested decision. It does not begin drafting. The evidence lane locates each required financial period and flags that the forecast uses a different currency convention from the historic accounts. The ownership update contains a new holding company, but the mastered customer hierarchy has not yet been amended. Both facts become explicit gaps.
The arithmetic lane maps the accepted statement fields, applies the approved sign and currency rules and runs balance checks. It produces a set of ratios under the current definition and a separate sensitivity using the forecast assumptions. One management-accounts subtotal fails reconciliation. The result is quarantined, so the memo cannot quote it.
The policy lane selects the rule set for the existing facility and proposed increase. It finds that the ownership change requires a separate review and that a leverage condition can proceed only through an exception authority. It does not decide whether the exception is appropriate. The narrative lane drafts the financial and business analysis while stating that ownership resolution and the failed subtotal remain open.
The reviewer requests corrected management accounts and confirms the new group structure through the approved process. The arithmetic and policy lanes rerun only the affected parts. The regenerated draft shows the changes. The reviewer then proposes a decision with conditions under their delegated authority. A second approval is requested because the exception exceeds the first reviewer's mandate.
The workflow saves work by preserving completed evidence, not by pushing an incomplete case to a confident answer. Every re-entry point is tied to the corrected object and the downstream results it invalidates.
Typed interfaces make the separation real
Architecture diagrams do not enforce lanes. Contracts do. Each service input and output should be machine validated, versioned and owned. Free-text agent messages can accompany the objects for readability, but they cannot carry protected state transitions.
| Contract | Required content | Explicit negative outcome |
|---|---|---|
| Case intake | Decision type, subjects, product, amount, purpose, time and requester | Ambiguous subject; unsupported decision type |
| Evidence request | Proposition IDs, authorised sources, freshness and evidence-role rules | Not found; denied; stale; conflict; timeout |
| Calculation request | Input proposition IDs, formula version, units and rounding | Reconciliation failure; missing input; invalid unit |
| Policy request | Applicability envelope, policy family and decision context | No applicable rule; conflicting version; out of authority |
| Draft request | Protected values, allowed claims, section schema and gaps | Unsupported claim; altered value; prohibited speech act |
| Approval request | Pack hash, decision, conditions, authority and approver identity | Insufficient delegation; expired evidence; changed pack |
The contracts should include correlation and idempotency identifiers. A retry after a network timeout must not create two cases, two evidence requests or two approval records. Read operations can often be repeated safely, but asynchronous document jobs and external data calls still need stable job identity. A tool response that times out after submission is an unknown outcome until reconciled.
The tool registry describes data classification, side effects, expected latency, timeout semantics, evidence returned and the postcondition used to verify completion. The language model sees a concise tool description derived from the registry. It cannot edit the registry or add an operation through prompt text.
Failure handling is part of credit quality
A production workflow must remain honest during partial failure. If a market-data call fails, the system may continue with a named limitation. If the mastered entity service is ambiguous, group-wide exposure should stop. If a calculation invariant fails, dependent narrative sections remain blocked. These policies should be decided before an incident.
| Failure | Safe system response | Unsafe behaviour |
|---|---|---|
| OCR service unavailable | Queue extraction or route to manual intake | Ask the drafting model to read an unverified text dump |
| Customer hierarchy conflict | Block aggregation; request identity resolution | Select the graph candidate with highest similarity |
| Policy service stale | Use approved last-known version only if policy permits and label it | Retrieve an arbitrary indexed copy |
| Calculation mismatch | Quarantine outputs and expose failed invariant | Generate a plausible ratio from narrative text |
| Model provider timeout | Resume from protected case state or switch approved route | Restart the case and duplicate side effects |
| Approval service ambiguity | Reconcile approval state through independent read | Retry approval submission blindly |
Recovery should occur at the narrowest lane. A failed narrative draft does not require re-extraction. A corrected document invalidates mapped propositions and their downstream calculations, not unrelated policy. A policy publication invalidates applicable treatments and draft claims, not source financial facts.
The runtime records stop conditions. It stops when evidence sufficiency passes, an unrecoverable gap appears, a protected conflict requires human review, the tool budget is exhausted or the requested action is outside authority. It does not continue because the model can think of another search.
A controlled unknown is a successful outcome when the alternative is an unsupported credit assertion. Operational reporting should therefore count correct abstentions and escalations, not treat them as generic failure.
Security and privacy follow the evidence graph
Credit cases contain personal, commercial and sometimes special-category data. Minimal context should be assembled for the stated purpose. Access to one customer does not imply access to related parties, financial-crime cases or every document mentioning the same name.
The security design should bind the user, workload, subject and purpose at each source. Model endpoints receive only admitted fields. Prompts, caches, traces and evaluation samples inherit classification and retention from their source data. A response cache is not safe merely because it stores generated prose.
Document content is untrusted. A clause, email or spreadsheet cell cannot instruct the agent to disclose data, change its task or call a new tool. The controller treats source text as evidence under analysis. Prompt-injection tests should include hidden text, annotations, formula strings and external links.
Case-level access should be revocable. If a user changes team or the case becomes restricted, future calls fail and cached context becomes ineligible. Existing decision records remain protected under record policy; they do not grant continuing conversational access.
Design the canary around real decisions
Shadow mode should compare the new evidence pack with the current process without influencing the decision. The team selects representative cases across products, document quality, exceptions and reviewer groups. It reconciles every material difference and classifies the causal component.
The first production canary can expose evidence and calculations while leaving narrative generation optional. Later releases can add cited drafting. The exception and approval boundary should not be relaxed merely to demonstrate straight-through automation.
| Release stage | Capability | Exit evidence |
|---|---|---|
| Offline replay | Reconstruct historical evidence and calculations | Known-time integrity and component accuracy |
| Shadow | Run beside live cases with no decision influence | Difference analysis and reviewer assessment |
| Evidence assist | Provide sources, gaps and verified calculations | Accepted time saving and low material defect rate |
| Draft assist | Add protected cited narrative | Claim support and correction burden within range |
| Bounded automation | Automate only low-consequence deterministic steps | Stable controls, rollback and independent monitoring |
The release decision should name residual risks, accepted limitations, rollback triggers and owners. A model improvement does not automatically justify more authority. Autonomy changes are separate design decisions based on evidence and consequence.
The architecture review questions
An architecture board should be able to answer the following without reading prompts:
- Which system is authoritative for every protected field?
- Which calculations are deterministic, versioned and independently tested?
- How is policy applicability selected for entity, product, purpose and time?
- Which evidence gaps block drafting, recommendation or approval?
- Can a model alter a protected value or accepted policy outcome?
- Which identities can request, review, approve and execute each state change?
- How are timeouts, duplicates, corrections and superseded evidence handled?
- Which tests prove that unsupported claims and out-of-authority actions are blocked?
- What evidence would trigger rollback or a reduction in automation?
If these answers exist only in a prompt or a vendor demonstration, the system is not ready for decision-grade credit work.
The strongest credit architecture is not the one that produces a memo with the fewest clicks. It is the one that makes every material proposition, calculation, rule, exception and decision attributable. That design can still be fast. It earns speed by removing repeatable clerical work while keeping the institution's judgement visible.
The same principle governs change. When a covenant formula, policy treatment or product rule changes, the owner should be able to identify affected propositions, calculations, evaluations and open cases before the new version becomes effective. Release notes then describe business consequences rather than model features. A case already under review retains its admitted versions unless an authorised transition rule reopens it. This prevents a silent platform update from rewriting the basis of a decision.
Service ownership should follow the decomposition. Data owners attest source contracts. Credit-methodology owners approve rules and calculation definitions. Product owners set workflow and authority. Engineering operates the orchestration and evidence services. Model owners maintain drafting and evaluation components. No single model owner can declare the end-to-end credit service safe, because safety depends on the joined behaviour of data, policy, tools, workflow and people.
Portfolio monitoring should use the same proposition and evidence contracts where practical. It can identify which accepted decisions depended on a policy version, data source or calculation package that has since changed. That does not reopen every case automatically. It produces a bounded impact set for an authorised owner. The ability to trace a platform change into business decisions is what turns reusable components into a controlled credit capability.