TLDR
- More than one agent is justified only when specialisation or independent verification creates information that a matched single-agent system cannot obtain as cheaply. Role names and extra turns are not evidence.
- Coordination introduces a tax in tokens, latency, context loss, integration defects and correlated persuasion. A team must beat a strong single agent with the same tools and a comparable inference budget.
- Voting helps when errors are sufficiently independent. Cloned agents sharing model, prompt, retrieval and evidence can reach confident consensus while multiplying the same mistake.
- Decompose work along real information, tool or authority boundaries, and exchange decision-sufficient messages instead of raw conversational histories. Topology should follow dependency structure.
- Approve a multi-agent design only after a factorial baseline attributes net gain, measures error correlation and includes a fallback when coordination state becomes uncertain.
Five specialists, one copied mistake
A team of five agents reviews a software patch. One examines security, one performance, one tests, one architecture and one documentation. Their reports look distinct. Four approve the patch. The fifth raises a minor naming issue. A manager concludes that diverse expert review has produced strong evidence.
The patch contains a path-traversal vulnerability. Every agent used the same model, repository summary and generated diff. None opened the file that normalised the path. Their roles changed vocabulary, not information. The four approvals are correlated observations of one blind spot.
Now replace the security role with a deterministic static check and a sandboxed exploit test. The test fails. One independent channel outweighs four fluent approvals.
This is the baseline problem in multi-agent systems. Additional agents can create genuine capability through specialisation, parallel evidence gathering, independent error detection and bounded authority. They can also create theatre: duplicated inference wrapped in a social metaphor. The architecture is justified only when the incremental information exceeds the coordination tax.
The central decision rule is sharp. Use several agents for distinct information boundaries or meaningfully independent verification. Otherwise prefer one strong agent with tools, explicit state and a verifier where needed. This rule does not make one topology universally best. It makes the burden of proof visible.
The coordination tax
Let a single-agent design achieve expected decision value at cost . Let a team achieve at cost , including inference, latency, integration, monitoring and failure recovery. The team is warranted when
under the same decision objective and acceptable risk. The terms need not collapse into money. A scorecard can keep accuracy, consequence, latency and operability separate. The point is that multiplicity has an incremental burden.
Figure 1. Specialisation must cross the coordination tax
Specialisation gain may rise as tasks become separable and agents receive relevant tools or contexts. Coordination cost rises with message count, dependency depth, shared state and integration. The crossing point is the first region where a team has a plausible warrant. Beyond it, more agents can still reduce value if communication or error propagation dominates.
What belongs in the tax
Token cost is the obvious term. It is rarely the largest conceptual term.
Latency accumulates across sequential calls and verification loops. Context loss occurs when one agent compresses information for another. State inconsistency appears when agents act on different versions of the world. Integration error enters at message schemas, tool outputs and merge logic. Observability cost rises because failure must be localised across several trajectories. Security surface expands with identities, permissions and inter-agent channels. Accountability diffusion appears when no component owns the final claim.
Some taxes are benefits in another design. Context separation can protect specialists from irrelevant information. Independent identities can attenuate authority. Parallelism can reduce wall-clock time. The evaluation should measure the actual route rather than assuming that more nodes mean more complexity or more capability.
Thought experiment: rename the functions
Take one deterministic workflow with four functions: retrieve evidence, calculate totals, check policy and compose a report. Wrap each function in a language-model persona and call them agents. Keep inputs, outputs and execution order fixed. Has the system gained multi-agent intelligence?
It has gained components with stochastic interfaces. That may help if each component needs semantic judgement. It may harm if deterministic code already solves the step. The thought experiment varies only the label. It shows that agent count should follow computational need, not a preference for organisational metaphors.
Two warrants for multiplicity
There are two strong reasons to use more than one agent.
The first is specialisation. Subtasks require different models, tools, evidence, context windows or authority. A vision specialist reads images, a code agent executes tests, a policy agent retrieves effective rules, and a synthesis agent composes a bounded answer. The value comes from differentiated capability and protected context.
The second is independent verification. Another route can detect errors because it uses a different model, representation, evidence source, test method or objective. A theorem prover checks a proof sketch. A database readback checks an asserted effect. A reviewer forms an opinion before seeing the proposal. The value comes from unshared failure modes.
Many successful teams use both. A specialist proposes within its domain and another channel verifies consequential properties. Yet the warrants should remain distinct. Specialisation can increase capability while preserving correlated errors. Verification can reduce error without dividing the task.
Figure 8. More than one agent needs one of two warrants
The decision instrument ends with measured net gain. A distinct role is insufficient. A separate prompt is insufficient. A second model call is insufficient. The system needs evidence that the boundary or verification route changes outcomes at acceptable cost.
Specialisation must be real
A specialised agent should have a reason to receive different context. It may use a smaller domain model, private tool, narrow ontology, long-running workspace or scoped permission. Its output should expose the contribution rather than hide it in a conversation.
For example, a tax calculation agent returns inputs, rule version, calculation and exceptions. A document agent returns claims and located evidence. A policy agent returns obligations and effective dates. The synthesis agent can then reason over typed artefacts. If every specialist returns prose without provenance, specialisation becomes hard to verify and easy to imitate.
Independent verification must trace the channel
Record independence across five layers:
- model or algorithm;
- prompt and problem framing;
- context and retrieved evidence;
- tool or environment observation; and
- objective or scoring rule.
Two different vendors can share a flawed source. Two different tools can share a corrupted database. Two human reviewers can share an anchoring interface. Independence is graded and claim-specific.
flowchart TD A["Five specialists, one copied<br/>mistake"] --> B["The coordination tax"] B --> C["Two warrants for multiplicity"] C --> D["Independence and correlated error"] D -. "re-test" .-> A classDef first fill:#0a1424,stroke:#D95545,stroke-width:2px,color:#e6f3ff classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff class A first class B,C,D step
Decompose the information
Good decomposition follows the structure of the task. Split where information, tools, time, expertise or authority differ. Do not split a tightly coupled calculation merely to create roles.
Figure 3. A real decomposition follows information boundaries
The figure begins with one task and opens into four information regions. Each specialist receives only what it needs. Their outputs rejoin through a typed merge. The geometry matters: the boundaries correspond to evidence and responsibility, not personality.
Dependency before topology
Represent the task as a dependency graph. A node is a decision or artefact with declared inputs and outputs. An edge means one result is needed by another. Independent nodes can run in parallel. Conditional nodes wait. A high-risk merge receives verification.
This graph may produce a pipeline, fan-out/fan-in, hierarchy, market or dynamic route. The topology is an implementation of dependencies. Starting with “supervisor plus workers” and forcing every task into it reverses the design.
AND and OR structure
Some tasks require all contributions. A due-diligence decision may need identity, financial, legal and policy evidence. This is an AND dependency. Other tasks allow alternative methods: calculate by formula or query an authoritative service. This is an OR dependency.
Confusing them creates failure. Majority vote cannot replace a missing mandatory control. Requiring every optional specialist adds cost. The mission state should mark which contributions are necessary, substitutable, unresolved or expired.
Communication is a bottleneck
An agent can send its entire reasoning trace to the next agent. This maximises volume and can reduce useful information. Long traces consume context, carry irrelevant speculation, leak private data and anchor the receiver.
Decision-sufficient communication asks for the smallest message that preserves the receiver’s decision. A typed message may contain:
- claim and confidence;
- evidence identifiers and freshness;
- assumptions;
- unresolved alternatives;
- required decision;
- authority and purpose; and
- failure or unknown state.
Figure 4. Communication should preserve the receiving decision
The bottleneck in the figure compresses many local trace lines into claims, evidence and risk. Compression is lossy by design. The test is whether removing a field changes receiver performance or hides a consequential failure.
Message ablation
Start with a rich message. Remove one field at a time and measure the downstream decision. Fields whose removal has no effect across relevant cases may be redundant. Fields whose removal harms rare high-consequence cases remain required even if aggregate accuracy barely changes.
Also test adversarial messages: contradictory evidence, stale state, missing units, duplicated tasks, spoofed identity and overlong content. The receiver should reject malformed or unauthorised messages rather than “reason through” them silently.
Aggregation is a decision problem
An aggregator does more than summarise. It decides which claims survive disagreement. Majority vote suits interchangeable independent judgements with a shared label space. It is wrong for mandatory controls, heterogeneous expertise and asymmetric consequence.
Use a typed aggregation rule. Mandatory evidence follows AND semantics: identity, authority and postcondition may all be required. Alternative methods follow OR semantics once one result is verified. Numerical estimates can be combined with uncertainty when their assumptions align. Conflicting factual claims require source adjudication. Value conflicts return to an authorised person.
Confidence-weighted voting helps only if confidence is calibrated and comparable. One agent’s 0.8 may not mean another’s 0.8. Calibrate by role and task type. Do not allow a highly confident generalist to overrule a deterministic specialist property check.
Record dissent. A final answer should retain unresolved claims, minority evidence and the rule that resolved each conflict. Consensus without a receipt erases the signal that made multiple agents useful.
Trust should be earned locally and allowed to expire
Dynamic teams often route work using past performance. A global trust score is too coarse. An agent may be strong at extraction, weak at temporal reasoning and untested after a tool update. Competence belongs to capability, context, evidence and effective time.
Maintain a trust record with task family, evaluation set, calibration, recent failures, tool versions and expiry. Route on the smallest relevant record. Exploration can sample alternatives to detect degradation and avoid permanent lock-in. High-consequence work may require a minimum evidence floor regardless of historical rank.
Trust updates can be gamed. If agents grade one another, a coalition can inflate scores. If reward follows aggregator selection, agents may learn persuasive style. Use independent outcomes, hidden checks and periodic blind evaluation. Separate reputation from authority: a trusted agent still receives only scoped permission.
Synthesis can reintroduce the original blind spot
Specialists may all produce correct artefacts and a synthesis model can combine them incorrectly. Evaluate the merge separately. Provide cases with conflicting units, duplicated entities, stale versions and one missing mandatory field. Require the synthesiser to preserve citations and unresolved state.
A deterministic merge is preferable when outputs already share a schema. Use a model only where semantic reconciliation is necessary. Even then, validate structural invariants after synthesis. The final answer should not silently change numbers, source identities or policy status.
Topology follows dependency
Three common topologies illustrate the trade.
In a parallel ensemble, agents work independently and an aggregator combines results. It preserves error diversity and suits alternative solutions. In a router topology, a classifier or policy sends work to the relevant specialist. It saves cost but can misroute. In a hierarchy, supervisors decompose and integrate. It supports complex dependencies but creates bottlenecks and authority concentration.
Figure 5. Topology is an experimental variable
Dynamic topology can adapt to task phase. Early exploration may use parallel specialists. Execution may collapse to one accountable route. Incident response may introduce an independent verifier. Dynamic routing must remain inspectable: record why a node was selected, which alternatives were skipped and what trust evidence was current.
Thought experiment: remove the supervisor
A supervisor assigns tasks, reviews outputs and writes the final answer. Remove it and replace its deterministic responsibilities with a workflow engine: dependency tracking, schema validation, timeouts and merge rules. Keep only the semantic synthesis step as a model call.
If quality stays fixed while cost and variance fall, the supervisor persona was implementing workflow state poorly. If quality falls because task decomposition required judgement, retain a bounded planner. The experiment separates orchestration from reasoning.
Failure propagation
Topology changes how errors spread. A wrong router can suppress the correct specialist. A wrong supervisor can overwrite several good outputs. A fully connected debate can broadcast a persuasive error. An isolated specialist can fail silently.
Inject faults at nodes and edges. Corrupt a message, delay a tool, return contradictory evidence, expire an identity and remove one specialist. Measure detection, propagation, recovery and final decision. A team that performs well only when every member behaves is not robust coordination.
Persistent missions need explicit recovery
Long-running teams outlive one context window and sometimes one process. Mission state should be checkpointed as tasks, artefacts, dependencies, authority and unresolved outcomes. On restart, a worker resumes from accepted state rather than rereading a chat and guessing what was decided.
Every task has a stable identifier and lifecycle such as proposed, ready, running, blocked, unknown, completed, rejected or superseded. A timeout after a tool call produces unknown until readback resolves the effect. Reassigning an unknown task without idempotency can duplicate work or action.
Compensation is not reversal. Deleting a duplicate calendar event may repair state while notification has already reached participants. Record residual consequence and escalate where compensation is incomplete. A team must know which effects are reversible before distributing execution.
Failure recovery also tests topology. If the router disappears, can ready tasks still be discovered? If a specialist is unavailable, is there a qualified substitute or a clear hold? If the shared workspace is stale, do workers stop? Graceful degradation may collapse to one supervised agent or a manual queue.
Security expands at every edge
An inter-agent message can carry untrusted content, spoofed identity, excessive authority or private data. Authenticate the sender and bind the message to subject, purpose, task and expiry. Validate schemas and size. Treat embedded natural-language instructions as claims unless the sender has authority for that control field.
Capabilities should attenuate through delegation. A coordinator with access to ten tools need not pass all ten permissions to a specialist. Grant the minimum action set and data scope for the subtask. Prevent a worker from using the coordinator as a confused deputy through a persuasive message.
Test cross-agent prompt injection. One retrieved document may tell the extraction agent to instruct the payment agent. The typed merge should carry the text as evidence, not executable authority. A protocol transports messages; it does not decide who may cause an effect.
Private reasoning and personal data should not be broadcast across the team. Decision-sufficient communication reduces leakage as well as cost. Log metadata and evidence references where possible, with protected access to content needed for incident review.
A matched-budget baseline
The executable artefact compares three synthetic results. The single-agent baseline has tools and one evidence channel. A specialist team improves accuracy with three distinct channels at moderate extra cost. A team of conversational clones spends far more tokens and messages while adding no evidence channel.
. It uses only the Python standard library and includes positive and negative cases.
#!/usr/bin/env python3
"""A matched-budget baseline for deciding whether multiple agents are warranted."""
from __future__ import annotations
from dataclasses import dataclass
@dataclass(frozen=True)
class Result:
name: str
correct: int
total: int
tokens: int
messages: int
distinct_evidence_channels: int
@property
def accuracy(self) -> float:
return self.correct / self.total
def utility(result: Result, token_price: float = 0.000002, message_cost: float = 0.002) -> float:
return result.accuracy - result.tokens * token_price - result.messages * message_cost
def warrant(single: Result, team: Result, minimum_gain: float = 0.02) -> dict:
gain = utility(team) - utility(single)
has_independence = team.distinct_evidence_channels > single.distinct_evidence_channels
return {
"single_utility": round(utility(single), 4),
"team_utility": round(utility(team), 4),
"net_gain": round(gain, 4),
"independent_evidence": has_independence,
"warranted": gain >= minimum_gain and has_independence,
}
def run() -> None:
single = Result("single agent with tools", correct=82, total=100, tokens=42_000, messages=0, distinct_evidence_channels=1)
specialists = Result("specialists plus verifier", correct=94, total=100, tokens=58_000, messages=8, distinct_evidence_channels=3)
clones = Result("three conversational clones", correct=85, total=100, tokens=120_000, messages=24, distinct_evidence_channels=1)
positive = warrant(single, specialists)
negative = warrant(single, clones)
print({"specialists": positive, "clones": negative})
assert positive["warranted"] is True
assert negative["warranted"] is False
assert negative["independent_evidence"] is False
if __name__ == "__main__":
run()
The utility function subtracts token and message cost from accuracy.
The exact weights are illustrative; real systems need
consequence-specific utility. The warrant function requires
both sufficient net gain and greater evidence independence. The
specialist team passes. The clones fail even though their raw accuracy
is slightly higher than the single agent.
This dual requirement blocks a common loophole. An expensive team can purchase a small score gain through extra sampling without demonstrating a reason for architectural multiplicity. That gain may still be useful, but the baseline should call it inference scaling, not specialisation.
Factorial evaluation
Figure 7. The baseline suite isolates the source of gain
A serious baseline suite varies factors separately:
- one strong agent;
- the same agent with all permitted tools;
- the same agent with a larger token or test-time budget;
- independent parallel samples;
- role-prompted clones with communication;
- specialists with differentiated context or tools;
- specialists plus an independent verifier; and
- deterministic workflow components wherever semantics are unnecessary.
Hold the base model, task cases and information budget where the comparison requires it. Report task success, error correlation, latency, tokens, tool calls, message failures, authority violations and recovery. Use repeated runs because stochastic topology can have heavy-tailed cost.
Figure 6. Budget changes which architecture wins
The budget frontier can reverse. At low budget, one capable agent may dominate because coordination overhead consumes the allowance. At higher budget, parallel exploration or verification may produce value. A weak single-agent baseline can make a team look good at every budget. Plot the frontier, not one point.
stateDiagram-v2 [*] --> Claim state "Figure 1. Specialisation must<br/>cross the coordination tax" as Claim state "What belongs in the tax" as Test state "Thought experiment, rename the<br/>functions" as Revision Claim --> Test: expose mechanism Test --> Decision: evidence survives Test --> Revision: rival survives Revision --> Claim: narrow or rebuild Decision --> [*]
A production-shaped team
Consider investigation of a long regulatory document set. The task requires identifying entities, reconstructing obligations, comparing versions, calculating thresholds and producing an evidence-backed case summary.
A weak design creates six personas that discuss the documents. A stronger design creates bounded components:
- an extraction agent returns located claims with page and document identity;
- an entity resolver returns candidate identities and uncertainty;
- a rule engine calculates explicit thresholds;
- a temporal agent reconstructs effective versions;
- a contradiction agent searches for disconfirming evidence independently; and
- a synthesis agent writes only from accepted evidence nodes.
The mission state records dependencies. Extraction can run in parallel by document. Entity resolution must precede cross-document joins. Threshold calculation waits for resolved units. Synthesis waits for mandatory evidence and marks unresolved contradictions.
The independent contradiction agent uses a separate retrieval query and sees the proposed claims only after committing its own evidence set. It can block high-consequence conclusions. The final human reviewer receives evidence and unresolved risk, not six chat transcripts.
Operational failure
One extraction agent times out after writing half a result. The workflow does not treat silence as completion. Its task remains unknown, the dependency stays blocked, and retry uses an idempotency key. If the document is inaccessible, the mission degrades to an incomplete-evidence state. The synthesis agent cannot fill the gap from plausibility.
This example shows what multi-agent production actually requires: typed state, dependency semantics, identity, timeouts, partial-failure handling, evidence lineage and release rules. The language models are important workers. They are not the coordination substrate.
Compose the team per mission
A permanent panel of agents spends cost on irrelevant roles and increases attack surface. Select participants from the task graph, required capabilities, current trust and consequence. Record the selection as part of mission state.
Dynamic composition needs guardrails. A planner cannot invent a new high-authority role because a task is difficult. Candidate agents advertise typed capabilities, tools, evidence scope, service limits and evaluation status. A deterministic selector enforces eligibility; a semantic router may rank eligible candidates.
Fallback selection should preserve the requirement. Replacing a legal specialist with a general writer may keep the workflow moving and destroy the warrant for the conclusion. If no qualified agent exists, mark the task blocked or route to a person. Availability does not convert a substitute into an authority.
Human-machine teams need an explicit interface
People can supply goals, context, judgement, approval and recovery that models cannot. They can also anchor on fluent outputs, rubber-stamp under workload and lose track of distributed state. Evaluate the joint system.
Give the human decision-sufficient evidence: claims, provenance, disagreement, uncertainty, proposed action and consequence. Do not require them to reconstruct six conversations. Let reviewers request local detail and see each specialist’s original commitment. Protect a route to reject the framing, not only select among agent proposals.
Measure unaided and assisted performance, review time, override quality, missed errors and trust calibration. A team that improves average speed while degrading rare high-consequence review may be a poor design. Sample cases the automation handled without human attention so oversight does not become blind by default.
Authority stays named. Agents may advise, propose or execute within scope. Humans may approve or veto. A collective label such as “the team decided” should never obscure the accountable actor and policy that released an effect.
Learn topology cautiously
Routing can adapt from outcomes, but the feedback is delayed and selective. Easy tasks produce abundant labels; hard escalated cases may remain unresolved. Optimising only observed success can route away from the cases that reveal weakness.
Use offline replay, controlled exploration and minimum coverage. Compare learned routing with a transparent rule and retain a safe default. Evaluate shifts after model, tool and policy updates. A topology policy is another model with its own version, evidence and rollback.
The best learned decision may be to avoid collaboration. If one agent repeatedly solves a task with verified tools, bypass the panel. Dynamic systems should optimise for decision quality and evidence, not agent utilisation.
Retain that bypass as a permanent baseline. It keeps the coordination tax visible after launch.
Re-run the baseline whenever the lead model, shared toolset or routing policy changes. Multiplicity must keep earning its place against the best simple system available now, not the weaker system it defeated at launch.
Publish that comparison with every material release.
The multi-agent decision record
| Question | Evidence required | Reject multiplicity when |
|---|---|---|
| What is the single-agent baseline? | Strong model, same tools, explicit state and matched budget | Team is compared only with a weak prompt |
| Where is specialisation? | Distinct context, model, tool, time horizon or authority | Only persona wording differs |
| Where is verification independence? | Mapped differences across model, evidence, tool and objective | Agents share the decisive failure source |
| What is the task graph? | AND, OR and conditional dependencies with typed outputs | Topology is chosen before dependencies |
| What message is sufficient? | Ablation and adversarial message tests | Raw traces are forwarded by default |
| What is the coordination tax? | Tokens, latency, variance, failures, monitoring and recovery | Incremental value does not exceed cost |
| How do errors propagate? | Node and edge fault injection | One failure silently contaminates the team |
| Who owns the decision? | Explicit proposal, veto, approval and execution rights | Accountability dissolves into consensus |
The record should include a retirement condition. If a stronger model or tool removes the specialisation advantage, collapse the topology. Architecture is not a permanent identity. A team that once earned its cost can become unnecessary.
Where multiplicity fails
Multi-agent design has four hard boundaries.
First, tightly coupled tasks may resist decomposition. Local optimisation can harm the whole. Second, shared evidence can preserve correlated error. Third, communication can erase useful context or spread falsehood. Fourth, no consensus mechanism creates authority or accountability by itself.
The strongest objection is that emergent interaction may create capabilities that no factorial test anticipates. Teams can discover decompositions, negotiate roles and solve open-ended tasks. That possibility deserves exploration. It does not remove the baseline. Emergence should be measured against simpler scaling, and its variance and failure surface should be part of the result.
Another objection is economic: model calls continue to become cheaper. Even if tokens become nearly free, latency, error correlation, observability and authority remain. Cheap conversation can still be expensive coordination.
The five software reviewers at the opening did not provide five pieces of evidence. They provided one blind spot in five voices. The exploit test changed the decision because it followed a different path to the claim.
Count information boundaries, not agent names. Add specialists where tools, evidence or authority genuinely differ. Add verifiers where failure modes can be separated. Keep messages small and state explicit. Let a matched single agent win when it can. A multi-agent system is strongest when it is allowed to justify its own absence.
sequenceDiagram participant Q as Question participant M as Mechanism participant E as Evidence participant D as Decision Q->>M: Five specialists, one copied mistake M->>E: Figure 1. Specialisation must cross the coordination tax E-->>M: compare rival explanations E->>D: Sources D-->>Q: revise scope or act
Sources
- Primary Du et al., Improving Factuality and Reasoning in Language Models through Multiagent Debate, 2023.
- Primary Liang et al., Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate, 2023.
- Primary Wu et al., AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation, 2023.
- Primary Li et al., CAMEL: Communicative Agents for Mind Exploration of Large Scale Language Model Society, 2023.
- Primary Chen et al., AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors, 2023.
- Primary Wang et al., Mixture-of-Agents Enhances Large Language Model Capabilities, 2024.
- Primary Krogh and Vedelsby, Neural Network Ensembles, Cross Validation, and Active Learning, 1994.
- Reference Wooldridge, An Introduction to MultiAgent Systems, second edition.