The hidden shape of the job

Imagine three people moving a piano upstairs. One person cannot simply “do the left side” and another “do the right side” independently. Their actions are coupled by weight, timing, geometry and a shared object. Now imagine three people translating three unrelated pages. The work can be separated almost completely. A third task asks three scientists to decide which of two explanations fits a result. Their dependency is neither physical nor sequential. It is epistemic: the value comes from preserving rival hypotheses long enough to test them.

Calling all three situations “collaboration” discards the variable that matters. Coordination theory offers a better starting point: coordination is the management of dependencies among activities.¹ Classic multi-agent work reached a similar conclusion from computation. Decker and Lesser argued that there is no single best coordination mechanism for every environment; mechanisms should respond to features of the task environment.² The task contains a shape before any agent is assigned to it.

Draw the task graph before the chat graph

Represent each unit of work as a node. Draw an edge only when one unit changes what another unit can validly do. The edge must say why. A predecessor edge means one result is required before another begins. A producer-consumer edge means an output becomes another task’s input. A shared-resource edge means two activities compete for the same tool, budget or record. A mutual-constraint edge means the activities must be reconciled together. An epistemic edge means two claims must remain in tension until evidence discriminates between them.

This is the task dependency graph, written here as G_D. It is not a list of roles. “Researcher”, “critic” and “manager” describe labels attached to actors. “Claim B cannot be assessed until evidence A arrives” describes a causal dependency. The latter can tell us what communication or sequencing is required.

A second graph is equally important. Its nodes are decision roles, services or people. Its edges express rights such as propose, inspect, approve, execute, verify and compensate. This is the authority graph, G_A. The authority graph answers questions that the task graph cannot: Who may release a payment? Who may merely recommend it? Who can revoke a capability? Who verifies that the effect occurred?

Only after both graphs exist should we draw the communication graph, G_C. Its message edges should be justified by a dependency in G_D or an obligation in G_A. A framework may make full-mesh conversation easy, but ease is not a causal reason for every node to hear every message.

flowchart TD
  A["Opening case, five analysts<br/>and one payment button"] --> B["The hidden shape of the job"]
  B --> C["What each topology is actually for"]
  C --> D["Compile from two graphs"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#D95545,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

What each topology is actually for

Topology names are useful only when they denote a coordination contract. A contract states what dependency the pattern manages, what information it assumes, what authority it grants and what causes it to fail. Without those clauses, “hierarchy”, “swarm” and “debate” become decorative language.

Pipeline: preserve precedence and typed hand-offs

A pipeline fits when the work has a stable partial order. One stage transforms an artefact and hands a typed result to the next. It is strongest when intermediate outputs can be validated locally, replayed safely and versioned. The critical property is not that agents speak one after another. It is that downstream work is invalid until a named upstream condition holds.

The break condition is frequent backflow or hidden shared state. If stage four regularly discovers that stage one must reinterpret the original evidence, the supposed line is actually a loop. If several stages mutate the same operational record, a message chain does not provide transaction semantics. Pipelines also concentrate error in early stages, so each hand-off needs provenance and acceptance criteria.

Hierarchy: decompose work while retaining accountable control

A hierarchy fits a decomposable goal with nested scope. A parent assigns bounded tasks, integrates results and holds responsibility for unresolved conflicts. This reduces the number of relationships each worker must track. Horling and Lesser’s survey describes hierarchies among several organisational paradigms and notes that organisational design can materially affect system performance.⁴

The break condition is a manager whose context, judgement or throughput becomes the system’s limiting resource. A hierarchy can also erase local information when every observation is compressed upward. The coordinator must receive decision-sufficient evidence, not every token, and its authority must be narrower than its conversational visibility.

Market: allocate substitutable work under private differences

A market fits when several agents could perform a task, their capabilities or costs differ, assignments can change, and delivered work is verifiable. Smith’s Contract Net Protocol was an early high-level protocol for distributing problem-solving tasks through announcements, bids and awards.³ The mechanism is useful because bids can reveal local information that a central scheduler does not possess.

The break condition is an outcome that cannot be verified, or an assignment with large externalities on other tasks. If every bidder has the same model, tool access and cost, bidding may reveal only sampling noise. Markets allocate; they do not automatically create truth, shared intent or legitimate authority.

Debate: preserve contest until an adjudicator can discriminate

Debate fits a decision with rival explanations, evidence that can be partitioned or independently inspected, and an adjudication rule that can recognise a better argument. Multi-agent debate has improved reasoning and factuality on some evaluated tasks.⁹ That evidence supports debate as a conditional mechanism, not a universal upgrade.

The break condition is correlated evidence, persuasive conformity or an evaluator weaker than the dispute. A later study of five popular multi-agent frameworks across more than 150 tasks identified failures in system design, inter-agent alignment, verification and termination, with gains often limited relative to simpler baselines.¹⁰ A debate protocol must protect independence before communication, reveal sources and allow “no decision” when the judge cannot discriminate.

Swarm: aggregate local rules without global deliberation

A swarm fits many similar units acting from local observations under a simple aggregate objective. Reynolds showed how local behavioural rules could generate flock-like global motion, while later control work analysed conditions under which nearest-neighbour interactions converge.⁶⁷ The engineering lesson is narrow: global coordination can emerge without a central planner when the local rule and connectivity assumptions are right.

The break condition is a need for globally consistent mutable state, semantic adjudication or irreversible local actions. A swarm of language agents sharing free-form messages is not automatically analogous to a flock. The agents are richer, the messages can carry adversarial instructions, and the global objective may be underspecified.

Compile from two graphs

A useful compiler does not pretend to discover a mathematically optimal organisation from a few sliders. Its job is narrower. It converts explicit task and authority assumptions into a ranked design, exposes contradictions and generates tests. The output is a hypothesis for a configured system, not a verdict.

Step 1: type the dependencies

For each edge in G_D, record its kind, strength and timing. A hard predecessor blocks execution. A soft predecessor improves quality but can be skipped under a deadline. A shared resource needs arbitration. A mutual constraint needs iterative reconciliation. An epistemic conflict needs protected independence followed by adjudication.

This is where many agent designs fail. They decompose a goal into nouns such as “planner”, “researcher” and “reviewer”, then connect the nouns conversationally. The compiler instead asks what makes one work unit valid or invalid. Roles are assigned after dependencies, because roles do not create the dependency they are meant to manage.

Step 2: type authority separately

For each role or service in G_A, record the subject, purpose, resource, action type, limits, expiry and revocation path. Distinguish proposal from permission. Distinguish a decision receipt from an effect receipt. Distinguish an agent’s belief that an action succeeded from a readback by an authoritative system.

Irreversibility changes the design. A research summary can be regenerated. A public filing, payment or access revocation may not be cheaply reversed. As irreversibility rises, execution should pass through fewer, more deterministic authority edges, with idempotency, explicit confirmation and independent outcome verification.

Step 3: score candidate contracts by mismatch

Let a candidate topology be T. The compiler estimates five losses. D_dep is dependency distortion: required dependencies that the topology omits or artificial dependencies it invents. D_auth is authority leakage: rights available outside the declared authority graph. C_msg is communication cost. R_corr is correlated-failure exposure. R_rec is recovery difficulty after interruption.

Lower is better. The weights make authority leakage more expensive than message cost. That is a policy choice, not a universal constant. The equation is valid only when assessors use a common ordinal scale and agree on the relative weights. Remove that assumption and the arithmetic should be treated as a discussion aid, not a ranking.

The hybrid uses parallel evidence workers, a short pipeline for typed transformations, a bounded debate at the rule conflict, and a separate authority gate for payment. Its score is not evidence that it will outperform every implementation. It reveals why no single topology manages all the dependencies without distortion.

Step 4: compile overlays rather than a single label

A practical compiler emits layers. The work layer may be a pipeline or hierarchy. The allocation layer may be a market. The epistemic layer may invoke debate only for a contested node. The execution layer may be a deterministic authority kernel. The recovery layer specifies checkpoints, receipts and compensation.

This decomposition prevents a common category error. A market can select a specialist without granting the specialist payment authority. A debate can challenge an interpretation without becoming the execution path. A hierarchy can own escalation without seeing unrestricted customer data. A hybrid is disciplined when each overlay manages a named dependency; it is not a licence to combine every pattern.

Run the extended compiler as code

The Python version below uses only the standard library. It adds shared-state coupling and output verifiability to the browser instrument, so its scores are deliberately more discriminating. Save it as topology_compiler.py and run python topology_compiler.py. Edit the sample profile or import compile_topology into a design notebook.

from __future__ import annotations

from dataclasses import dataclass
from enum import Enum
from typing import Dict, List, Tuple


class Topology(str, Enum):
    PIPELINE = "pipeline"
    HIERARCHY = "hierarchy"
    MARKET = "market"
    DEBATE = "debate"
    SWARM = "swarm"


@dataclass(frozen=True)
class TaskProfile:
    """All fields use an ordinal 0..4 scale."""

    serial_dependency: int
    decomposability: int
    allocation_uncertainty: int
    contestability: int
    locality_and_scale: int
    capability_heterogeneity: int
    independent_evidence: int
    irreversibility: int
    shared_state_coupling: int
    output_verifiability: int

    def validate(self) -> None:
        for name, value in self.__dict__.items():
            if not 0 <= value <= 4:
                raise ValueError(f"{name} must be between 0 and 4")


def clamp(value: float) -> float:
    return max(0.0, min(100.0, value))


def compile_topology(profile: TaskProfile) -> Tuple[List[Tuple[Topology, float]], List[str]]:
    """Rank topology fit and emit invariant or break-condition warnings.

    The score is a design prior. It is not a forecast of task accuracy.
    """
    profile.validate()
    p = profile

    scores: Dict[Topology, float] = {
        Topology.PIPELINE: clamp(
            18 * p.serial_dependency
            + 6 * p.output_verifiability
            - 9 * p.contestability
            - 8 * p.shared_state_coupling
        ),
        Topology.HIERARCHY: clamp(
            15 * p.decomposability
            + 7 * p.irreversibility
            + 5 * p.shared_state_coupling
            - 4 * p.locality_and_scale
        ),
        Topology.MARKET: clamp(
            14 * p.allocation_uncertainty
            + 11 * p.capability_heterogeneity
            + 8 * p.output_verifiability
            - 9 * p.shared_state_coupling
            - 6 * p.irreversibility
        ),
        Topology.DEBATE: clamp(
            17 * p.contestability
            + 13 * p.independent_evidence
            + 4 * p.output_verifiability
            - 8 * p.serial_dependency
        ),
        Topology.SWARM: clamp(
            17 * p.locality_and_scale
            + 6 * (4 - p.shared_state_coupling)
            - 12 * p.irreversibility
            - 5 * p.serial_dependency
        ),
    }

    warnings: List[str] = []
    if p.irreversibility >= 3:
        warnings.append(
            "Invariant: keep irreversible execution behind one typed authority gate "
            "with idempotency and independent readback."
        )
    if p.contestability >= 3 and p.independent_evidence < 2:
        warnings.append(
            "Debate break condition: agents lack sufficiently independent evidence."
        )
    if p.allocation_uncertainty >= 3 and (
        p.capability_heterogeneity < 2 or p.output_verifiability < 2
    ):
        warnings.append(
            "Market break condition: bids are not informative or deliverables are hard to verify."
        )
    if p.locality_and_scale >= 3 and (
        p.irreversibility >= 2 or p.shared_state_coupling >= 2
    ):
        warnings.append(
            "Swarm break condition: local action is too irreversible or globally coupled."
        )
    if p.serial_dependency >= 3 and p.contestability >= 3:
        warnings.append(
            "Hybrid indicated: use a pipeline for evidence flow and debate only at contested nodes."
        )

    ranked = sorted(scores.items(), key=lambda item: item[1], reverse=True)
    return ranked, warnings


if __name__ == "__main__":
    reimbursement = TaskProfile(
        serial_dependency=2,
        decomposability=3,
        allocation_uncertainty=1,
        contestability=3,
        locality_and_scale=1,
        capability_heterogeneity=2,
        independent_evidence=3,
        irreversibility=4,
        shared_state_coupling=3,
        output_verifiability=4,
    )
    ranking, warnings = compile_topology(reimbursement)
    print("Ranked coordination layers:")
    for topology, score in ranking:
        print(f"  {topology.value:10s} {score:5.1f}")
    print("\nDesign warnings:")
    for warning in warnings:
        print(f"  - {warning}")

A positive result means the stated task properties make a topology plausible enough to prototype and compare. A negative result means the topology lacks an enabling condition or violates an invariant. It does not mean multi-agent work is impossible; it may mean the task should remain with one agent and tools, be decomposed differently, or use a hybrid.

The compiler cannot establish model competence, empirical superiority, regulatory compliance or safe operation. It also cannot infer the true task graph from a vague prompt. Its value is to make the designer state assumptions that a benchmark can later challenge.

{
  "task_id": "reimbursement_assessment",
  "dependency_edges": [
    {"from": "policy_fact", "to": "eligibility", "kind": "hard_predecessor"},
    {"from": "claim_fact", "to": "amount", "kind": "producer_consumer"},
    {"from": "interpretation_a", "to": "adjudication", "kind": "epistemic_conflict"}
  ],
  "authority_edges": [
    {"subject": "case_coordinator", "action": "propose_payment", "limit": 100000},
    {"subject": "policy_kernel", "action": "execute_payment", "requires": ["approval", "freshness", "idempotency_key"]}
  ],
  "task_profile": {
    "serial_dependency": 2,
    "decomposability": 3,
    "contestability": 3,
    "independent_evidence": 3,
    "irreversibility": 4
  }
}

stateDiagram-v2
  [*] --> Claim
  state "Draw the task graph before the<br/>chat graph" as Claim
  state "Pipeline: preserve precedence<br/>and typed hand-offs" as Test
  state "Hierarchy: decompose work<br/>while retaining accountable<br/>control" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Test the seams under pressure

A topology is not validated because the happy path completes. The revealing tests change the property that justified the topology. Remove independent evidence from debate. Equalise bidder capabilities in a market. Increase the number of peers in a mesh. Interrupt a pipeline after an external effect but before its receipt. Compromise the most central node in a hierarchy.

Failure is transmitted along edges

Recent work on multi-agent systems has made the cost of unnecessary edges visible. AgentPrune reported large reductions in token use while retaining comparable performance in its evaluated benchmarks by pruning redundant communication.¹¹ G-Designer found that different tasks benefited from different communication graphs and that simple tasks could use much smaller structures.¹² These results are benchmark-specific, but they support a general design caution: a communication edge should earn its place.

Edges also carry attacks and social influence. Research on the security of interacting agents identifies risks such as collusion, privacy propagation and coordinated attacks across networks.¹⁴ A compromised agent in a mesh can expose every peer. A compromised leaf behind typed interfaces may expose one bounded task. The security difference is structural.

Interrupt the system where certainty is weakest

Suppose the authority gate submits a payment request. The external system commits it, but the network times out before the effect receipt returns. The agent sees an unknown outcome. A naive pipeline retries and pays twice. A chat transcript saying “I think the tool failed” is not world state.

The recovery topology needs an idempotency key, a durable intent record, a decision receipt, an effect receipt and an independent readback. On timeout, the system enters an unknown-outcome state and queries the authoritative ledger before retrying. This pattern belongs to topology because it determines how control and evidence reconnect after interruption.

The topology congruence experiment

Test five synthetic task families. Use a serial transformation, a divisible allocation problem, a contested evidence judgement, a local search problem and a decomposable hierarchical plan. For every task instance, randomise pipeline, hierarchy, market, debate, swarm and a compiler-produced hybrid. Keep base prompts, models, tools and total token budget fixed. Separate initial private work from communication so independence can be measured.

Inject one fault per run: stale evidence, a compromised message, a missing predecessor result, an agent timeout or a revoked capability. Measure correct completion, calibrated abstention, token and tool cost, invalid effect attempts, time to recovery, fault reach and the fraction of initially distinct hypotheses that survive until adjudication.

The hypothesis strengthens if congruent topologies produce a better Pareto frontier and if specific mismatches predict specific failures. It weakens if one simple topology dominates after budget control, or if task features do not predict outcomes. A rival explanation is prompt or role quality rather than topology. Control it with identical role-neutral workers, typed interfaces and topology-only rewiring. Another rival is that learned graph search discovers latent task features better than the hand-coded compiler.

The strongest alternative: learn the graph

Hand-designed rules may miss subtle interactions. AutoGen makes conversation patterns programmable,⁸ while more recent systems explicitly search or learn prompts and topologies. MASS jointly optimises prompts and workflow structure, and G-Designer generates task-adaptive communication graphs.¹³¹² It is plausible that empirical search will beat a simple compiler on a stable benchmark.

That is not a reason to abandon causal structure. It changes the compiler’s role. Use the task graph as a prior, the authority graph as a hard constraint and learned search inside the remaining safe design space. The optimiser may add or remove communication edges. It may not grant a new subject the right to execute a payment because validation accuracy rose on a benchmark.

Dynamic tasks create another boundary. The dependency graph may change as evidence arrives. A topology can recompile its work and communication layers, but authority changes require a fresh policy decision. An agent that discovers a new subtask does not thereby acquire a new capability.