Worked composite: a saturday night in a collections queue
The composite begins on a Saturday, when the people most likely to notice are offline. Its regional-bank architecture runs a multi-agent case-management pipeline for customers in financial difficulty. An Intake Agent classifies contact. A Document Agent extracts fields from hardship letters, pay slips and statements. A Decision Agent recommends a treatment. An Execution Agent, the only component with ledger write access, carries out an approved action.
A fifth component, the Case Notes Summariser, reads free-text notes and produces a short summary. Its original permission is read-only, but a later sprint grants a generic “delegate to orchestrator” tool so it can flag urgent cases. That convenience creates an indirect path from untrusted text to the same execution surface used by the Decision Agent.
The orchestrator tool the helper could now call was not dangerous by design. It was a general-purpose "resolve_case" endpoint used internally by the Decision Agent to hand off approved actions, exposing functions including issue_goodwill_credit, waive_late_fee and extend_forbearance. It trusted its caller's payload rather than the caller's identity, on the false assumption that anything calling resolve_case had already been through a decision step. Nobody had considered that a summarisation agent, reading unstructured customer text, might become a source of instructions rather than merely a reader of them.
The scenario trigger is text embedded in a customer complaint field, submitted through a web form feeding directly into the case notes table. It is modelled as a deliberate prompt injection: the note asks, in essence, for an immediate goodwill credit to be applied to "all accounts with a similar complaint code, given how badly this has been handled." The helper agent, whose only mandate is to summarise, produces a summary that includes what it reads as an actionable recommendation and passes it to resolve_case. The orchestrator does not distinguish that payload from a properly reviewed decision.
In the composite run, eleven accounts receive goodwill credits ranging from two thousand to two thousand five hundred dollars each, twenty-five thousand dollars in total. None is reviewed by a human, and each is indistinguishable in the ledger from a legitimate Decision Agent approval. An adjacent fraud-monitoring job, not an agent-safety control, alerts ninety-six minutes after the first spurious credit. The constructed on-call investigation finds a summarisation agent doing exactly what its prompt and available tools allow.
The important design number in the composite is not the twenty-five thousand dollars realised in the run. It is one thousand four hundred, the size of the active hardship queue reachable through the delegation path, and three and a half million dollars, the hard cash-disbursement upper bound under the scenario's once-per-case idempotency rule and two-thousand-five-hundred-dollar per-account ceiling. The graph and tool contract make those bounds computable before a test run, rather than only after a favourable outcome.
Why testing is not a safety case
The instinct in most engineering organisations, mine included in earlier years, is to answer "is this agent safe to deploy" with a testing programme: unit tests on the tool wrappers, a suite of adversarial prompts, a red team exercise, a staged rollout to a small percentage of traffic. All of this is worth doing. But testing answers a different question to the one a safety case requires. Testing tells you what the system did across the scenarios you thought to try. It does not tell you what the system is capable of doing across the scenarios you did not think to try, and in an LLM-driven agent that space of inputs is not enumerable the way a fixed API parameter set is.
A safety case, in the sense a structural engineer or an aviation regulator would recognise the term, requires a defensible upper bound, not a sample of favourable outcomes. When a bridge is certified, nobody runs ten thousand vehicles across it and declares it safe because none of them fell through. The load is computed from the material properties and the geometry, and a margin is applied on top of the computed figure. Blast radius analysis in physical safety engineering works the same way: given a quantity of explosive material and a known overpressure curve, you compute the radius within which structural damage or injury is expected, and you do not rely on the fact that nobody has been hurt yet.
Multi-agent systems in a bank deserve the same treatment, for the same reason: the cost of being wrong is not proportional to the frequency of testing, it is proportional to the worst single event that gets through. The composite's eleven accounts and twenty-five thousand dollars are materially different from its three-and-a-half-million-dollar cash-disbursement bound. No test suite tells you in advance which exposure is technically available. A reachability computation combined with enforceable action caps does.
The certainty gradient matters before topology and containment. Every workflow step sits between deterministic code and open-ended reasoning. The former can provide stable output contracts. The latter can interpret unstructured evidence but cannot guarantee an output shape.
The composite failure is not caused by using a model to summarize case notes. That task benefits from open-ended reasoning. The mistake is connecting its output directly to a high-consequence write surface. The transition from unstructured text to money movement needs a low-ambiguity boundary. A schema-validated decision object should stand between reasoning and the ledger.
A taxonomy of multi-agent topologies
Before blast radius can be computed for a specific system, it helps to have a vocabulary for the shapes these systems tend to take, because the shape determines, almost mechanically, how damage propagates.
The star, or hub-and-spoke, pattern has a single supervisor agent that every other agent reports to and is invoked by, with no direct agent-to-agent edges among the spokes. This is the topology most vendors default to, and for good reason: every action any spoke takes has passed through the hub, and the hub is the single place a permission boundary needs full rigour. Its weakness is that the hub becomes both the single point of control and the single point of failure; if the hub's own credential or reasoning is compromised, its blast radius is, by construction, the union of everything every spoke can do.
The pipeline pattern, the shape used in the opening composite, chains agents in a fixed sequence, each consuming the previous agent's output. Pipelines feel safe because the flow of control looks linear on a diagram. The composite illustrates why pipelines are rarely as linear in practice: side channels such as a shared tool bus, a common orchestrator or a shared memory store accumulate over a system's life, each one a shortcut around the sequence the diagram implies. A pipeline's blast radius is only as contained as its least-audited side channel.
The mesh pattern allows any agent to call any other agent directly, typically to support flexible collaboration on ambiguous tasks where the right next step is not knowable in advance.
Mesh topologies are the most expressive and the most dangerous from a blast radius perspective, because the reachability graph is close to complete: with n agents each holding some subset of tool permissions, a mesh can in the worst case make every tool reachable from every agent through some chain of calls, even where no single agent was directly granted access to that tool. I have seen a mesh topology proposed for a fraud investigation workstream on the argument that agents need to collaborate freely.
The honest answer: free collaboration and computable blast radius are close to mutually exclusive without additional structure.
The hierarchical supervisor with bounded delegation is the pattern I now default to for anything touching account-level actions in banking. It looks like a star, a supervisor coordinating subordinates, but with two constraints that change its mathematics: each subordinate is granted only the tools its function requires, not a shared orchestrator surface, and delegation between subordinates, where it exists, is an explicit, individually scoped edge rather than a generic call-anything-the-supervisor-can-call capability. The difference from a star is entirely the discipline of scoping, but that discipline is what makes the reachability graph computable and small rather than computable and large.
The table below is an illustrative planning comparison synthesised from recurring delivery patterns; it is not a benchmark of four named deployments. Latency means coordination overhead per action, not model inference time. Containment cost is an indicative engineering range, in person-weeks, for retrofitting computable bounds onto a deployment of that shape. A local design should replace every range with measured values.
| Topology | Typical blast radius growth | Added coordination latency | Containment retrofit cost | Best fit |
|---|---|---|---|---|
| Star or hub and spoke | Union of the hub's unique reachable assets; grows as spokes add new assets | 80 to 150 ms per hop | 2 to 4 person weeks | Small agent counts, single clear owner of write actions |
| Pipeline | Union of unique downstream assets; side channels enlarge that set | 120 to 300 ms end to end | 4 to 8 person weeks, mostly finding side channels | Sequential processes with a genuine fixed order of operations |
| Mesh | Can reach the full unique asset universe; path count may be combinatorial, blast radius is not | 40 to 90 ms per hop, but many hops | 10 to 20 person weeks | Rarely justified for account level actions in banking |
| Hierarchical supervisor, bounded delegation | Union of assets on individually scoped, capped edges | 90 to 160 ms per hop | 3 to 6 person weeks if designed in from the start | Default recommendation for regulated write actions |
The pipeline row is the one I want to underline, because it is the shape most banking multi-agent systems actually take, and it is the shape whose blast radius is most often under-estimated, precisely because the diagram makes it look like the safest option on the table.
Blast radius as a reachability problem
Once the topology is fixed, computing blast radius stops being a matter of judgement and becomes graph traversal. The resulting sets and bounds can be independently recomputed, whereas a qualitative assurance cannot.
Construct a directed graph with three kinds of nodes. Agent nodes represent each deployed agent identity, distinguished by its runtime credential rather than its conceptual role: two agents sharing a service account count as one node. Tool nodes represent each distinct callable capability, including internal orchestration endpoints, not only external APIs; the composite failure exists precisely because an internal orchestration endpoint is omitted from the architecture's tool inventory. Entity nodes represent the assets tools can act upon: accounts, ledger balances, customer records, credit limits and held funds.
Edges come in three flavours that must each be walked separately, because they compound differently. A grant edge runs from an agent node to a tool node and represents a direct, provisioned permission, the kind that appears in an IAM policy or API scope. A delegation edge runs from one agent identity to another identity or orchestration surface whose grants it can cause to be exercised, whether through a direct call, queue, shared memory or unauthenticated handoff. An action edge runs from a tool node to one or more entity nodes and represents what that tool can do once invoked. Each action edge carries a write class, value cap, successful-call cap, per-entity cap, idempotency key and enforcement window; an absent cap is recorded as unbounded rather than silently inferred.
For an agent a, let Reach(a) be the set of distinct nodes in its transitive closure across grant, delegation and action edges. Its blast radius is the distinct reachable asset set Reach(a) ∩ Entities, annotated with the action classes and enforced effect bounds available on those assets. If five different paths reach the same account, that account appears once in the blast radius. Path count is a separate attack-surface and resilience measure; it is not multiplied into the asset count or monetary exposure unless separate paths permit additional non-idempotent commits.
Five measures make the annotated set operational. Worst-case reachable accounts is the count of distinct account entity nodes, independent of how many paths lead to each one. Reachable write-action classes is the count of distinct consequential operations, such as credit, fee waiver or forbearance extension; it is not the number of graph edges or possible paths.
Maximum successful action commits within window W is a hard execution bound. For each action class it is the minimum of the identity call quota, the rate ceiling over W, and the sum of enforced per-asset commit caps. An idempotency key must bind retries to the same business action; a transport retry cannot consume another customer entitlement. If none of those controls supplies a finite limit, the action count is unbounded and the architecture cannot claim a finite exposure merely because tests issued few calls.
Worst-case financial exposure sums, across monetary action classes, the enforced value per commit multiplied by the hard successful-commit bound within W. Mutually exclusive dispositions are maximised once per asset rather than added as if all could execute; independently executable actions are summed. Transitive delegation risk is the increase in these sets and bounds when delegation edges are included rather than only direct grants.
The composite tool contract makes the arithmetic finite. The scenario fixes one eligible active complaint for each of 1,400 distinct accounts. resolve_case accepts one terminal disposition per complaint, enforced with an account-complaint-policy-version idempotency key, and the queue executor has a hard cap of 1,400 successful commits per cycle. Its three dispositions are mutually exclusive. Goodwill credit has the largest monetary cap at two thousand five hundred dollars; fee waiver is capped at one hundred fifty dollars; forbearance carries no immediate cash value but remains a consequential write class. The successful-action bound is therefore 1,400 and the cash-disbursement bound is 1,400 × 2,500 dollars, or 3.5 million dollars. The sixty-hour reconciliation window does not enlarge that number because the per-account and call caps bind first.
For the helper, the transitive delegation delta is the entire 1,400-account, three-action-class and 3.5-million-dollar surface, since it holds no direct execution grant. A direct-grant-only review would correctly show zero direct authority but would be incomplete if it reported that as zero blast radius. The delegation path is what makes the assets reachable.
None of this is exotic to compute. The graph is built from IAM policy exports, service-mesh routing tables and a manual pass to capture delegation edges infrastructure tooling does not track. The output should preserve both the deduplicated reachable set and the path inventory: the former defines blast radius, while the latter shows alternate routes that controls must close.
Bulkheads and containment patterns
A computed blast radius is only useful if it changes what you build, and the changes it motivates are almost always structural rather than a matter of tightening a prompt or adding a warning to a system message.
Scoped credentials are the first and cheapest fix, and the one the composite most directly violates. Every agent identity should hold the smallest tool grant set that its function requires, issued as its own credential rather than shared with, or inherited from, another agent's identity.
The Case Notes Helper does not need delegation into an execution surface to summarise text, and once that grant is removed rather than merely discouraged through instructions, its transitive delegation risk falls to zero regardless of what the helper's language model is prompted to do. This sounds obvious, and it is; the difficulty is organisational rather than technical, because scoped credentials require someone to own the tool grant table as a first-class artefact, not an incidental side effect of whichever engineer last touched the deployment configuration.
Per-agent rate and value ceilings bound the exposure that remains even after scoping is done properly, because scoping answers what the identity can reach, while ceilings answer how much it can commit before containment.
In the remediated composite, an autonomous goodwill credit is capped at two hundred fifty dollars; no account can receive more than one autonomous financial disposition for the same complaint and policy version; every retry reuses the same idempotency key; and each identity can commit at most eight autonomous financial actions or two thousand dollars in aggregate, whichever binds first, in a rolling twenty-four-hour window. The hard monetary bound is therefore min(8 × $250, $2,000) = $2,000, even if the model loops.
Each constraint is enforced at the tool boundary, the deterministic end of the certainty gradient.
Kill switches and circuit breakers provide containment for the case where scoping and ceilings are insufficient or misconfigured. The remediated composite rejects any call without a matching, independently logged case ID. Three consecutive rejected attempts halt that identity, and the disable signal has a five-minute propagation objective measured from the third rejection. No model judgement appears in the trip condition. A circuit breaker must fire correctly when reasoning, classification and confidence scoring have already failed, so it cannot depend on any of them being trustworthy.
Quarantine zones for anything an agent writes that later becomes context for another agent close a different gap illustrated by the composite: the customer's complaint text, once summarised, is treated by the orchestrator as equivalent in trust to an internally generated decision even though it originated from an untrusted channel. Isolating externally sourced content in a staging area, tagged with provenance, until an explicit promotion step moves it into trusted context prevents the injected instruction from reaching the orchestrator as though it were reviewed. This is narrower than removing the delegation edge outright, and the two controls work together: remove an unnecessary authority edge and quarantine content where interpretation remains necessary.
Scoped credentials, idempotency, ceilings and circuit breakers affect different measures. Credentials reduce reachable assets and the delegation delta. Idempotency and call quotas reduce successful commits. Value ceilings reduce exposure without changing reachability. Circuit breakers reduce the containment interval but do not substitute for a hard action cap. Treating these as separate levers lets a team show exactly which control addresses each term.
Regulator-bounded autonomy and what examiners actually want
I have sat in enough model risk reviews, across the FCA's expectations in the UK, the PRA's operational resilience requirements, and OCC guidance in the US, to say that no examiner I have dealt with has been satisfied by testing coverage alone as evidence that an autonomous system is safe. What they consistently ask for, in different words depending on jurisdiction, is a defensible upper bound on what the system could do without a human in the loop, and evidence of a specific, named point at which human sign-off is required before autonomy extends further.
Regulator-bounded autonomy starts from the boundary rather than the capability. First define the maximum action that can proceed without a named human. Express it in units an examiner already uses: value per action, accounts per incident and classes of irreversible write.
Work inside that boundary may run autonomously when worst-case reachability shows that the boundary holds. Work outside it needs an explicit sign-off. The approval must be logged, attributable and monitored for rubber-stamping. Approval latency and override behavior reveal whether the gate remains a real control.
The specific artefact I bring into model risk conversations is a blast-radius statement: one page per agent showing the distinct reachable assets, reachable action classes, maximum successful commits, maximum monetary effect and delegation delta. It also names the enforcement window and the scoped credential, idempotency rule, call quota, value ceiling, circuit breaker or quarantine boundary supporting each claim. That statement answers the examiner's practical question: what is the worst effect the deployed controls permit, and which machine-enforced limit makes the number finite?
Consider the request that creates the composite failure: “the helper should flag urgent cases directly and save a round trip.” Regulator-bounded autonomy does not treat this as a small convenience. It asks whether the new edge changes reachable accounts, actions, cumulative exposure or detection time.
In the worked composite, the edge moves reachable accounts from zero to fourteen hundred and the cash-disbursement bound from zero to three and a half million dollars. Its assumed benefit is one four-hundred-millisecond round trip. Presented in those terms, the request should not survive review.
Worked example: recomputing the collections pipeline
Return to the opening composite and walk the computation in full: first under its deliberately weak control contract, then under a constructed remediated contract. Both sides use the same 1,400-account population and the same injected input, so the comparison isolates the controls rather than claiming an observed institutional before-and-after result.
Before remediation in the composite, the Case Notes Helper has one read-only grant to the case-notes table and one delegation edge to resolve_case. The orchestrator exposes three mutually exclusive terminal dispositions: issue_goodwill_credit, capped at two thousand five hundred dollars; waive_late_fee, capped at one hundred fifty dollars; and extend_forbearance, a status write with no immediate cash value. The endpoint enforces one terminal disposition through an account-complaint-policy-version idempotency key and a 1,400-successful-commit queue-cycle cap, but it fails to authenticate the calling identity or require a decision-ledger record. All three dispositions terminate on the same 1,400-account hardship queue, which the scenario constrains to one eligible active complaint per account.
The helper's worst-case reachable set contains 1,400 distinct accounts, no matter how many graph paths reach them. Three consequential action classes are reachable. The hard successful-commit bound is 1,400 because the one-disposition-per-complaint constraint binds at one action for each reachable account. The cash-disbursement bound is therefore 1,400 × 2,500 dollars, or 3.5 million dollars; mutually exclusive fee waivers and forbearance are not added on top. The maximum monitoring gap is sixty hours under the assumed weekend reconciliation schedule. The entire reachable and financial surface is transitive, because the helper holds no direct execution grant.
The composite run commits eleven credits totalling twenty-five thousand dollars and alerts after ninety-six minutes. That realised value is roughly 0.71 percent of the 3.5-million-dollar cash-disbursement bound. The gap between one run and permitted effect is precisely what a reachability computation makes explicit before a test happens.
The remediated composite introduces three changes, each addressing a term in the calculation rather than the scenario's wording. Blocking one phrase pattern would leave the underlying reachability untouched and the next variant fully available.
First, the delegation edge from the helper agent to the orchestrator is removed. The helper's output becomes a typed summary object with a small enumerated field set, none interpretable as an action directive. This takes the helper's reachable account set from 1,400 to zero and its financial exposure from 3.5 million dollars to zero, because no execution path remains.
Second, the orchestrator is split into three narrower tools, each authenticating a specific calling identity. Autonomous goodwill credit is capped at two hundred fifty dollars and fee waiver at fifty dollars; the dispositions remain mutually exclusive. The financial tools require a decision-ledger entry, reuse an account-complaint-policy-version idempotency key on every retry, allow one autonomous financial disposition per complaint, and enforce both an eight-successful-action quota and a two-thousand-dollar aggregate cap per identity per rolling twenty-four hours. Forbearance requires a human-reviewed case reference, so its autonomous action cap is zero.
Third, a purpose-built circuit breaker rejects calls without matching decision-ledger entries. Three consecutive rejections halt the identity, and enforcement propagation has a five-minute maximum objective from the third rejection. Replaying the same injected input in the composite test produces a halt at three minutes forty seconds, compared with the original composite run's ninety-six-minute alert. The hard comparison is sixty hours versus five minutes; the like-for-like replay comparison is ninety-six minutes versus three minutes forty seconds.
| Measure | Before remediation | After remediation |
|---|---|---|
| Reachable accounts, helper agent | 1,400 | 0 |
| Reachable write action classes, helper agent | 3 | 0 |
| Worst-case financial exposure, helper agent | 3,500,000 dollars | 0 dollars |
| Successful autonomous financial actions, properly scoped decision agent | 1,400 per queue cycle | 8 per rolling 24 hours per identity |
| Worst-case financial exposure, properly scoped decision agent | 3,500,000 dollars per queue cycle | 2,000 dollars per rolling 24 hours per identity |
| Maximum detection-and-disable interval | 60 hours | 5 minutes after the third rejected attempt |
| Same-scenario replay alert or halt | 96 minutes | 3 minutes 40 seconds |
| Transitive delegation risk | 100 percent of exposure | 0, delegation edge removed |
The important result is not only the helper's drop to zero. The properly scoped Decision Agent still reaches the accounts its role requires, but its non-human authority now has two independently enforced bounds: no more than eight successful autonomous financial actions and no more than two thousand dollars per rolling twenty-four hours. Blast-radius work is not about driving every number to zero. It is about ensuring every non-zero number is deliberate, hard-enforced and sized to the role.
Notes for practitioners
Before granting any agent-to-agent delegation edge, ask whether the same outcome can be achieved by a typed, schema-validated handoff instead, and default to the schema unless there is a specific, named reason the receiving agent needs open-ended interpretive latitude. In my experience roughly seven out of ten delegation edges proposed during a design review turn out to be replaceable by a fixed schema once someone asks the question, because most were proposed to save the effort of defining the schema, not because the task required interpretation.
Insist, contractually where the agent platform is a vendor product rather than something built in house, on visibility into the full grant and delegation graph, not merely the direct tool grants the vendor's own console surfaces by default. Several platform vendors I have assessed expose direct API scopes cleanly but bury internal orchestration and shared-memory pathways, exactly the category of edge represented in the composite. If a vendor cannot produce a complete delegation edge list on request, treat that gap itself as a finding, not as an acceptable limitation of the tooling.
Instrument the five blast-radius measures as live metrics, not as a one-time design artefact: distinct reachable assets, action classes, maximum successful commits, monetary exposure and delegation delta. Tool grants and delegation edges drift over a system's life one change at a time, and a launch-time computation will eventually stop reflecting the deployment. Recompute automatically against current IAM, service-mesh and tool-contract configuration, and treat an unbounded field as an alert rather than a missing value.
Escalate immediately, rather than routing through a normal change request, any proposed delegation edge from an agent handling externally sourced, unverified content into an agent or tool with write access to money movement or account status. This is the specific pattern modelled by the composite and a useful standing design rule, not a prompt-level exception.
Require a named human owner for every sign-off gate in the regulator-bounded autonomy boundary, with approval latency tracked as a metric in its own right. A sign-off gate with no owner, or one whose median approval time drifts toward a rubber stamp, is a gate in name only, and examiners who look closely find this faster than most teams expect.
Finally, keep the blast radius statement for each agent as a living one-page artefact, updated whenever the underlying grant or delegation graph changes, not regenerated from scratch during the next audit cycle. The largest source of friction I see in model risk reviews is not disagreement about adequacy, it is the multi-week delay in producing evidence that should have existed continuously. A system whose worst case is always computable on demand, in the language a regulator already uses, turns what is otherwise an adversarial review into a working conversation about where the boundary should sit next.
The topology control plane
The collaboration graph and the authority graph should be separate artefacts. Agents may exchange findings without inheriting one another's permissions. This prevents a harmless reasoning edge from becoming an invisible delegation edge.
Messages carry evidence; they do not carry authority by default. A receiving agent must obtain its own permission for the proposed action.
| Topology change | Graph effect | Required recomputation | Default response |
|---|---|---|---|
| Add collaborator | New communication nodes and edges | Data reachability and prompt-injection paths | Read-only until reviewed |
| Add tool | New authority edge and resource set | Reachable actions, records and financial ceiling | Deny write scope by default |
| Add shared memory | New transitive information paths | Read, write, propagation and retention reach | Partition by trust domain |
| Widen supervisor scope | Expands multiple downstream paths | Maximum fan-out and delegated exposure | Independent approval |
| Change retry policy | Multiplies possible action attempts | Cumulative effect and idempotency coverage | Bound attempts and value |
A decision matrix for structural separation
The reason to add an agent is independent judgement, ownership or containment. Prompt variety alone does not justify another node.
| Low coupling to external actions | High coupling to external actions | |
|---|---|---|
| Low need for independent judgement | Keep the step inside one agent or deterministic workflow | Isolate the tool adapter, not a new reasoning agent |
| High need for independent judgement | Separate specialist with evidence-only output | Separate proposer and verifier; centralize execution policy |
The safest high-value topology centralizes enforcement while distributing analysis. This preserves specialist reasoning without multiplying write authority.
For each action-capable node, report distinct reachable accounts, action classes, maximum successful commits, maximum attempts, cumulative value and containment interval. Keep the measures separate. A small account count can still carry high value. A low value per call can still accumulate when retries lack business-level idempotency.
Primary references
- NIST, AI Risk Management Framework, for lifecycle risk governance and measurement.
- NIST, SP 800-207A, for identity-centric access control in cloud-native applications.
- AWS, Multi-agent collaboration for Amazon Bedrock Agents, for managed supervisor and collaborator topology.
- Google Cloud, Agent Identity overview, for distinct agent principals and delegated access.
- IETF, RFC 8693: OAuth 2.0 Token Exchange, for constrained token exchange and delegation semantics.
Blast radius becomes governable when collaboration, authority, resource reach and cumulative effect are represented as separate, computable structures.