The first instinct is to search for the defective part. Which agent made the wrong decision? Which API was slow? Which team chose the unsafe retry setting? That instinct works when failures are linear. It misleads when the loss is produced by a relationship among individually reasonable parts.

Charles Perrow’s normal-accident theory separated systems along two dimensions: how understandable their interactions are, and how much slack exists between one event and the next. Interactively complex systems contain unfamiliar or indirect combinations. Tightly coupled systems leave little time, buffering or substitution once a disturbance begins. Perrow’s strong conclusion, still contested, was that serious accidents become an expected property of systems that combine both conditions, even when each component is defended.[1]

Nancy Leveson offers a complementary systems account. Software-intensive accidents can arise without a failed component because controllers issue unsafe actions, receive delayed feedback, or operate with an incorrect model of the process. Safety is therefore not just component reliability. It is a control problem involving constraints, feedback and coordination across a sociotechnical system.[2] Richard Cook makes the same point operationally: complex systems usually run with latent problems and overlapping defences, while practitioners continuously adapt at the boundary between production and failure.[3]

Keep four kinds of claim separate

The transfer from industrial safety to agentic systems needs discipline. A good analogy can suggest where to look, but it cannot carry evidence across domains by itself. The argument in this paper therefore uses four statuses.

This separation matters because “normal accident” can otherwise become a dramatic label pasted onto any multi-agent error. The article makes a narrower causal claim. When a configured system lets uncertainty acquire shared state, retries and authority faster than it can obtain independent evidence, the architecture creates a pathway for cascades. That pathway can be changed and tested without settling the sociological debate about inevitability.

Why agents sharpen both dimensions

An agentic system does more than call services. It interprets incomplete context, proposes a next step, invokes tools, updates a working narrative, and may delegate to other agents. This adds three forms of coupling.

Epistemic coupling occurs when one agent’s interpretation becomes another agent’s premise. Temporal coupling occurs when deadlines, retries and expiring permissions make the order of events matter. Authority coupling occurs when a model-generated proposal crosses a boundary and changes an external system. A single-agent chat can be epistemically wrong without producing a system accident. The same model, connected to shared state and a payment API, can turn one ambiguous observation into many actions.

A useful formalism, with an explicit warning

Teams often need a compact way to compare designs before a full hazard analysis. The following quantity is an ordering instrument, not a scientifically calibrated risk score:

Suppose Tern has surprise 4, fan-out 4 and irreversibility 5. With observability 1, isolation 2 and recovery slack 1, the comparison value is 40. Add effect-level readback, separate critical and optional queues, and a five-minute capability expiry. If those controls raise O, B and R to 4, 4 and 3, the value falls to about 1.7. The arithmetic is not a probability. Its purpose is to reveal which denominator is doing real protective work.

The equal weighting assumption is crude. If irreversible harm dominates the decision, keep the six-dimensional profile rather than multiplying it into one number. Use the scalar to ask where coupling resides, never to certify safety.

flowchart LR
  A["The payment that both happened<br/>and did not"] --> B["The accident no component caused"]
  B --> C["When coupling outruns<br/>understanding"]
  C --> D["Put firebreaks in the causal path"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#7B4CD8,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Cascades do not require spectacular faults. They require a local action that creates more work, more uncertainty or more authority downstream than the system can absorb.

Retries can turn care into load

Retries are one of the most effective responses to transient distributed-system failures. They are also a canonical work amplifier. Google’s SRE guidance gives a compact example: if three layers each make four attempts, one user action can create 4³, or 64, attempts at the lowest dependency. When that dependency is already overloaded, the protection mechanism becomes the sustaining cause of the outage.[4]

A timeout is an epistemic event

When a call times out, the caller knows that a response did not arrive before its deadline. It does not know whether the callee rejected the request, queued it, completed it, or completed it and lost the response. This is the unknown-outcome problem. Treating TIMEOUT as NOT_DONE silently converts missing evidence into a factual claim.

HTTP explicitly distinguishes safe automatic retry from non-idempotent operations. A proxy must not automatically retry a non-idempotent request merely because the connection failed.[5] AWS’s idempotent-API guidance explains why a caller-provided request identifier must be recorded atomically with the mutating operation. Otherwise the service can record the token without the effect, or create the effect without recording the token.[6]

Stateful systems create failures that final answers hide

Agent evaluation has started to move beyond single-turn tool selection. ToolSandbox introduces stateful execution, implicit dependencies among tools, and milestone evaluation across a trajectory. It reports that state dependency, canonicalisation and insufficient-information tasks remain challenging even for capable models.[7] A separate study of more than 150 multi-agent tasks identified fourteen failure modes across system design, inter-agent misalignment, verification and termination.[8]

These results do not prove a normal-accident theory of agents. They do support the premise that agentic failure is trajectory-dependent and interaction-dependent. A final answer can look correct after the system used stale authority, overwrote another agent’s state, retried an unknown effect, or stopped only because a token limit intervened. Outcome accuracy is necessary evidence, but it is not operating evidence.

Recent agent-specific work is beginning to study propagation directly. A preliminary preprint models how local errors can spread through multi-agent collaboration and reports that shared errors can be reinforced when downstream agents lack external verification.[11] This is relevant but not decisive. The study is new, its tasks and orchestration choices delimit transfer, and model families will change. Its most useful contribution here is methodological: evaluate the history of a claim as it crosses agents, rather than scoring only the final consensus.

A practical trace should therefore record, for each hand-off, the source assertion, confidence or uncertainty, state version, authority used, downstream transformations and independent checks. That turns “Agent B agreed with Agent A” into a testable question: did B observe new evidence, or merely inherit A’s premise in a different voice?

Metastability: when the trigger leaves but the failure stays

A particularly revealing class of distributed-system failure is metastability. A temporary trigger pushes a vulnerable system into a state where work amplification sustains overload even after the trigger disappears. Studies of public incidents identify retry storms, persistent congestion and similar death spirals as variations of this pattern.[9]

Agentic systems contain additional amplifiers. A slow tool call can trigger reflection, delegation, retrieval, verification and retries. Each action consumes context, queue capacity, rate limits and operator attention. The system can spend more resources explaining or checking its lack of progress than completing useful work. In Tern, the network fault lasts one minute. The queue remains saturated because expired tasks continue to execute, clients continue to retry, and every duplicate creates fresh reconciliation work.

Measure the recovery radius

Availability asks whether the service is up. The more useful agentic question is how much of the world must be inspected or repaired after one ambiguous interaction. Call this the recovery radius: the set of intents, state records, authority grants, effects and dependent tasks that must be reconciled before normal operation can safely resume.

In a toy Tern run, one lost payment receipt enters a global narrative store. Twelve downstream tasks consume the claim that payment failed, two agents issue follow-up actions, and the shared queue delays twenty-seven unrelated jobs. The recovery radius is not one payment. It includes every task whose decision depended on the uncertain claim. Change only the architecture: bind the unknown state to intent P-104, deny it from becoming global fact, and place related work in a supplier-scoped bulkhead. The unresolved effect remains, but the radius contracts to one intent, one supplier queue and one readback.

This measure is not a count to optimise blindly. One irreversible effect can matter more than a hundred deferred recommendations. Record the radius as a typed set and classify each member by irreversibility, authority and evidence debt. The design objective is to prevent a local unknown from creating an unbounded reconciliation obligation.

A graceful system is not one that keeps every feature alive. It is one that preserves its safety and mission invariants while deliberately reducing what it attempts.

Google SRE distinguishes load shedding from graceful degradation. Load shedding rejects work as a service approaches overload. Graceful degradation also reduces the cost or quality of what remains, so useful service continues without exhausting the system. The guidance warns that degraded modes are themselves risky because rarely exercised code tends not to work when finally needed.[4] NIST frames cyber resilience more broadly as the ability to anticipate, withstand, recover from and adapt to adverse conditions.[10]

For agentic systems, graceful degradation should be defined by preserved invariants, not by a vague percentage of availability. A payment system may lose recommendation quality, parallelism and speed while still preserving effect uniqueness, authority limits and a reliable account of what happened. That is graceful. Continuing at full speed while dropping receipts is not.

The causal firebreak stack

The proposed architecture inserts five explicit boundaries between a model’s proposal and an external effect. Each boundary converts an implicit assumption into a typed decision.

Four invariants for graceful operation

Effect uniqueness. One intent identifier can produce at most one irreversible effect within its validity window. This belongs at the executor or authoritative system of record.

Authority non-expansion. Recovery cannot grant broader scope, longer duration or a higher transaction limit than the interrupted operation possessed. A restart is not a reason to mint a fresh, more permissive capability.

Evidence continuity. Every terminal state has either an effect receipt, an independently verified absence, a compensation receipt, or an explicit unresolved marker. Silence is never converted into success.

Essential-service floor. Critical work retains a reserved path when optional work is shed. The reservation includes queue capacity, executor capacity, evidence storage and human attention. A separate priority label without separate resources is not a bulkhead.

Write the degraded-mode contract before launch

A degraded mode should be a configured product state, not an improvised incident response. Its contract names six things: the entry signal, the invariants that remain mandatory, the capabilities disabled, the resources reserved, the evidence shown to operators, and the exit proof. Without these fields, “degrade gracefully” usually means that each team drops whatever it finds convenient.

For Tern, entry occurs when queue age exceeds two action deadlines, the unknown-outcome count rises above three, or the global retry budget is exhausted. Recommendation generation, non-urgent reconciliation and multi-agent debate stop. Already-authorised critical payments may continue only with fresh state preconditions and executor-owned idempotency. Receipt storage and system-of-record readback receive reserved capacity. Operators see intent IDs and outcome states rather than a confidence summary.

Exit requires more than falling CPU usage. Queue age must return below one deadline, no critical effect may remain unresolved, capability versions must be current, and a canary slice must complete without growing the retry budget. This contract converts graceful degradation from a hopeful runtime branch into a testable operating mode.

The strongest boundary

Normal-accident theory should not become a licence for fatalism. High-reliability research and systems-theoretic safety argue that organisations can reduce risk through stronger feedback, disciplined operations, learning and control constraints. Better models can also reduce the frequency of initiating errors. The narrower claim here is that no finite test catalogue can enumerate every interaction in an evolving, tightly coupled system.

Therefore, the release argument must include both prevention and recovery. Prevention lowers how often the system enters a hazardous region. Firebreaks bound what happens when an unanticipated path still reaches it. The article should not be applied to low-stakes, read-only assistants where the recovery cost exceeds the harm. It is most useful when an agent can mutate shared state, spend money, change access, communicate externally or trigger other agents.

stateDiagram-v2
  [*] --> Claim
  state "Keep four kinds of claim separate" as Claim
  state "Why agents sharpen both dimensions" as Test
  state "Replace the architecture, not<br/>the model" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Restarting components restores processes. It does not resolve which intent was authorised, which effect occurred, which state is current, or which receipt can be trusted.

Executable lab: trigger, amplify, contain

The simulator deliberately keeps the task distribution and disturbance pattern the same. It asks a causal question: what changes when the architecture changes but the underlying work does not? Run it with Python 3.10 or later. It uses only the standard library.

from __future__ import annotations

from dataclasses import dataclass, field
from collections import deque
from random import Random
from statistics import mean
from typing import Deque, Dict, List, Optional


@dataclass(frozen=True)
class Config:
    name: str
    ticks: int = 70
    base_arrivals: int = 6
    surge_arrivals: int = 9
    nominal_capacity: int = 8
    impaired_capacity: int = 3
    disturbance_start: int = 12
    disturbance_end: int = 19
    timeout: int = 3
    max_attempts: int = 6
    queue_limit: int = 10_000
    breaker_threshold: int = 10_000
    breaker_cooldown: int = 0
    idempotency: bool = False
    verify_before_retry: bool = False
    shed_optional: bool = False
    priority_bulkhead: bool = False
    jitter: bool = False


@dataclass
class Job:
    job_id: int
    critical: bool
    created_at: int
    attempts_sent: int = 0
    next_retry_at: Optional[int] = None
    completed: bool = False
    effects: int = 0
    receipt_seen: bool = False
    shed: bool = False


@dataclass
class Attempt:
    job_id: int
    sent_at: int
    attempt_no: int


@dataclass
class Result:
    config: str
    logical_jobs: int
    critical_jobs: int
    completed_jobs: int
    completed_critical: int
    shed_optional: int
    pending_jobs: int
    attempts: int
    effects: int
    duplicate_effects: int
    max_queue: int
    recovery_tick: Optional[int]
    queue_trace: List[int] = field(repr=False)


def run_once(config: Config, seed: int = 7) -> Result:
    rng = Random(seed)
    jobs: Dict[int, Job] = {}
    queue: Deque[Attempt] = deque()
    next_job_id = 1
    attempts_total = 0
    effects_total = 0
    max_queue = 0
    breaker_open_until = -1
    recovery_tick: Optional[int] = None
    queue_trace: List[int] = []

    def retry_delay(job: Job) -> int:
        base = config.timeout * (2 ** max(0, job.attempts_sent - 1))
        return base + (rng.randint(0, config.timeout) if config.jitter else 0)

    def enqueue(job: Job, tick: int) -> bool:
        nonlocal attempts_total, breaker_open_until
        if job.completed or job.shed or job.attempts_sent >= config.max_attempts:
            return False
        if tick < breaker_open_until:
            job.next_retry_at = breaker_open_until
            return False
        if len(queue) >= config.breaker_threshold:
            breaker_open_until = tick + config.breaker_cooldown
            job.next_retry_at = breaker_open_until
            return False
        if len(queue) >= config.queue_limit:
            job.next_retry_at = tick + retry_delay(job)
            return False
        job.attempts_sent += 1
        attempts_total += 1
        queue.append(Attempt(job.job_id, tick, job.attempts_sent))
        job.next_retry_at = tick + retry_delay(job)
        return True

    for tick in range(config.ticks):
        disturbed = config.disturbance_start <= tick <= config.disturbance_end
        arrivals = config.surge_arrivals if disturbed else config.base_arrivals

        # New logical work enters the system. Critical work is intentionally rare.
        for _ in range(arrivals):
            critical = rng.random() < 0.25
            job = Job(next_job_id, critical, tick)
            next_job_id += 1
            jobs[job.job_id] = job

            overloaded = len(queue) >= max(1, config.breaker_threshold // 2)
            if config.shed_optional and overloaded and not critical:
                job.shed = True
                continue
            enqueue(job, tick)

        # Timeouts are observations of missing receipts, not proof of missing effects.
        for job in list(jobs.values()):
            if job.completed or job.shed or job.next_retry_at != tick:
                continue
            if config.verify_before_retry and job.effects > 0:
                job.completed = True
                job.receipt_seen = True
                continue
            enqueue(job, tick)

        capacity = config.impaired_capacity if disturbed else config.nominal_capacity
        selected: List[Attempt] = []

        if config.priority_bulkhead and queue:
            critical_q: Deque[Attempt] = deque()
            optional_q: Deque[Attempt] = deque()
            while queue:
                attempt = queue.popleft()
                (critical_q if jobs[attempt.job_id].critical else optional_q).append(attempt)
            critical_reserve = min(3, capacity)
            while critical_q and len(selected) < critical_reserve:
                selected.append(critical_q.popleft())
            while len(selected) < capacity and (critical_q or optional_q):
                if critical_q:
                    selected.append(critical_q.popleft())
                elif optional_q:
                    selected.append(optional_q.popleft())
            queue.extend(critical_q)
            queue.extend(optional_q)
        else:
            for _ in range(min(capacity, len(queue))):
                selected.append(queue.popleft())

        for attempt in selected:
            job = jobs[attempt.job_id]
            # Late work may still mutate the world after the caller has timed out.
            already_applied = job.effects > 0
            if not (config.idempotency and already_applied):
                job.effects += 1
                effects_total += 1

            receipt_loss = 0.18 if disturbed else 0.025
            receipt_seen = rng.random() >= receipt_loss
            if receipt_seen:
                job.completed = True
                job.receipt_seen = True
                job.next_retry_at = None

        max_queue = max(max_queue, len(queue))
        queue_trace.append(len(queue))

        # Recovery means the trigger is gone, the queue is below one nominal batch,
        # and no earlier recovery tick has been recorded.
        if (
            tick > config.disturbance_end
            and recovery_tick is None
            and len(queue) <= config.nominal_capacity
        ):
            recovery_tick = tick

    logical_jobs = len(jobs)
    critical_jobs = sum(j.critical for j in jobs.values())
    completed_jobs = sum(j.completed for j in jobs.values())
    completed_critical = sum(j.completed and j.critical for j in jobs.values())
    shed_optional = sum(j.shed for j in jobs.values())
    pending = sum(not j.completed and not j.shed for j in jobs.values())
    duplicate_effects = sum(max(0, j.effects - 1) for j in jobs.values())

    return Result(
        config=config.name,
        logical_jobs=logical_jobs,
        critical_jobs=critical_jobs,
        completed_jobs=completed_jobs,
        completed_critical=completed_critical,
        shed_optional=shed_optional,
        pending_jobs=pending,
        attempts=attempts_total,
        effects=effects_total,
        duplicate_effects=duplicate_effects,
        max_queue=max_queue,
        recovery_tick=recovery_tick,
        queue_trace=queue_trace,
    )


def summarise(config: Config, trials: int = 200) -> dict[str, float]:
    results = [run_once(config, seed=i) for i in range(trials)]
    return {
        "completion_%": 100 * mean(r.completed_jobs / r.logical_jobs for r in results),
        "critical_completion_%": 100 * mean(
            r.completed_critical / r.critical_jobs for r in results
        ),
        "attempts_per_job": mean(r.attempts / r.logical_jobs for r in results),
        "duplicate_effects": mean(r.duplicate_effects for r in results),
        "max_queue": mean(r.max_queue for r in results),
        "shed_optional": mean(r.shed_optional for r in results),
        "recovery_tick": mean(
            (r.recovery_tick if r.recovery_tick is not None else config.ticks)
            for r in results
        ),
    }


def main() -> None:
    uncontained = Config(name="uncontained")
    firebreaks = Config(
        name="causal_firebreaks",
        timeout=4,
        max_attempts=2,
        queue_limit=24,
        breaker_threshold=18,
        breaker_cooldown=5,
        idempotency=True,
        verify_before_retry=True,
        shed_optional=True,
        priority_bulkhead=True,
        jitter=True,
    )

    rows = [(uncontained.name, summarise(uncontained)), (firebreaks.name, summarise(firebreaks))]
    columns = [
        "completion_%",
        "critical_completion_%",
        "attempts_per_job",
        "duplicate_effects",
        "max_queue",
        "shed_optional",
        "recovery_tick",
    ]
    print("Synthetic cascade lab: 200 fixed-distribution trials per design\n")
    print(f"{'design':<22}" + "".join(f"{c:>23}" for c in columns))
    for name, row in rows:
        print(f"{name:<22}" + "".join(f"{row[c]:>23.2f}" for c in columns))


if __name__ == "__main__":
    main()

Across 200 seeded trials, the uncontained design completes about 59 per cent of critical jobs and does not return below one nominal batch by the end of the simulation. The firebreak design completes about 97.8 per cent of critical jobs and recovers near tick 20, while shedding roughly 51 optional jobs per trial. This is the point of graceful degradation: the protected design refuses some work so that essential work and effect integrity survive.

A positive result permits only a mechanism claim about the simulator: under its assumptions, firebreaks suppress work amplification and duplicate effects. A negative result would mean that the selected controls or thresholds do not create the intended trade-off. The lab cannot establish real-world failure rates, human response quality, model behaviour, cloud performance or regulatory acceptability.

The recovery runbook

A runbook should describe state transitions and evidence, rather than only people to call. The following sequence is intentionally conservative because recovery actions often have more authority than normal actions.

1. Declare the operating mode. Set the configured system to DEGRADED. Freeze new capability grants for irreversible actions. Keep read-only diagnosis available.
2. Stop amplification. Open circuit breakers, cancel retries above the global budget, reject expired work, and shed optional queues. Do not scale blindly while the work multiplier is unknown.
3. Establish the recovery cut. Snapshot intent IDs, capability versions, authoritative world state, effect receipts, unresolved outcomes and queue positions. Record the cut identifier.
4. Resolve unknown effects independently. Read the system of record through a path that does not depend on the failed orchestration chain. Classify each intent as confirmed, absent, compensated or unresolved.
5. Reconcile contradictions. Detect duplicate effects, stale permissions and state versions that cannot coexist. Apply compensation only through a new, narrower authorised action with its own receipt.
6. Restore in slices. Re-enable critical reads, then critical writes, then optional work. Use a canary retry budget and stop automatically if queue age, unknown outcomes or duplicate attempts rise.
7. Change the coupling. Update the architecture, test and operating limits that allowed the cascade. A prompt edit is insufficient unless the loss genuinely originated in interpretation alone.

The exit criteria are deliberately observable. “The agents look stable” is not sufficient. The system must show that useful work exceeds generated retry work, that every in-flight action belongs to a consistent cut, and that restoration does not recreate the same amplification loop.

Failure injection that tests the actual thesis

Do not test only whether an agent produces a sensible response after an error message. Inject faults at the boundaries that create coupling. Drop the reply after a tool has committed. Delay a duplicate until after a compensation. Revoke authority between proposal and execution. Corrupt narrative context while leaving the system of record correct. Exhaust the optional queue while reserving critical capacity. Crash the orchestrator after the effect but before receipt persistence.

Each test should assert the four invariants, the maximum cascade radius, and a clear exit condition. A useful release record includes the fault, the configured system version, the intent and capability identifiers, the expected degraded mode, the observed receipts, the readback result, and whether recovery increased authority. The test passes when the system preserves bounded harm and recoverability, not when every task eventually succeeds.