From labels to failure chains

A useful taxonomy has three jobs. It helps a team notice a class of exposure, routes the issue to people with relevant expertise, and permits comparison across systems. Many AI taxonomies perform the first job well. They list misinformation, discrimination, privacy, security, human interaction, system safety and societal effects. The AI Risk Repository, for example, deliberately offers both a domain taxonomy and a causal taxonomy because “what kind of risk is this?” and “how, when and by whom can it arise?” are different questions.

Frameworks such as the NIST AI Risk Management Framework serve a broader purpose. NIST organises activity through Govern, Map, Measure and Manage, and describes trustworthiness characteristics including validity, safety, security, privacy and fairness. ISO/IEC 23894 guides organisations in integrating AI-related risk management into their activities. These are governance structures, not substitutes for a system-specific failure statement. This article supplies an operational overlay that can sit beneath them.

Do not mix the category, the failure and the harm

Five objects are often collapsed into one row. A hazard is a condition with the potential to contribute to harm. A failure is a violated behavioural, informational or authority contract. A risk concerns the possibility and consequence of that failure under a particular exposure. An incident is an observed event or near miss. A harm is the realised adverse impact. The OECD's work on AI incident classification similarly separates incident characteristics, harms, scope and severity because an incident report needs more structure than a headline label.

Consider an assistant that produces an unsupported medical statement in a sealed evaluation environment. The unsupported statement is a model or system failure. It is evidence of a hazard under relevant use. It is not yet a patient harm, and it is not necessarily a production incident. Calling every test defect an incident exaggerates what happened. Calling it “no risk because nobody was harmed” makes the opposite mistake. The operational record must preserve the distinction between capability to fail, exposure to that failure and realised consequence.

Thought experiment: the same model, the same error, a different system

Imagine one model produces the same wrong refund amount in two applications. In the first, the amount appears in a draft beside the source data. A reviewer must approve it, and the tool independently recalculates the amount before submission. In the second, the model calls a payment API directly. The tool accepts free-form amounts, and the workflow records only that the request was accepted, not whether the external account changed.

A model-level taxonomy assigns both cases to “error”. An operational taxonomy records error as the primary class but also captures authority, propagation, detection and reversibility. Lane B may additionally carry safety, security or dependency contributors depending on the use. Risk belongs to the configured system and context of use, not to the model in isolation.

A discriminating test for the taxonomy

The taxonomy should earn its additional structure. A useful evaluation would begin with matched synthetic cases rather than asking reviewers whether the template appears sensible. Construct several base failures, such as an obsolete policy claim, a group-specific allocation error, an injected instruction and an unavailable provider. For each base failure, vary one system property at a time: draft versus direct execution, individual versus portfolio reach, immediate versus delayed observation, reversible versus irreversible effect, and independent versus shared fallback. The local model behaviour stays fixed while the operational exposure changes.

Give one set of review teams a conventional label-and-score register. Give another set the typed failure-chain template in this article. Both groups receive the same evidence, definitions and time limit. Reviewers should work independently before reconciliation so that agreement is measured rather than manufactured in a workshop. The unit of comparison is not whether both teams wrote the word “security”. It is whether they named the same failed contract, routed ownership to the people able to interrupt it, identified the earliest observable signal and selected a control whose mechanism matches the propagation path.

Score at least six outcomes: duplicate entries created for one chain; materially different chains collapsed into one row; common dependencies discovered; controls stated in testable language; stop or escalation decisions recovered from the record; and time required to update the assessment after a changed component or policy. Include the spelling assistant as a negative control. A method that calls every category material in that case has improved vocabulary while degrading discrimination. Also include a case in which a simple deterministic check catches the defect before the model output matters. The taxonomy should assign the residual exposure to the configured system, not preserve a high model-risk score by habit.

The thesis is weakened if the chain-based method does not improve control placement, change detection or decision agreement enough to justify its operating cost. It may prove too demanding for low-consequence systems, or reviewers may disagree about where one propagation chain ends and another begins. Those are empirical possibilities, not editorial inconveniences. The proposed comparison turns “operational” into a falsifiable claim: the richer record must produce better interventions, not merely longer registers.

flowchart LR
  A["The risk register that knew<br/>every word and nothing useful"] --> B["From labels to failure chains"]
  B --> C["The seven operational classes"]
  C --> D["The risk register as a typed<br/>argument"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#7B4CD8,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

The seven operational classes

The seven classes below are not mutually exclusive boxes. They are diagnostic lenses. Each lens asks for a different observable signature and usually routes first response to a different control discipline. A risk entry should name one primary class to avoid ownerless duplication, then add contributing classes where they change controls, escalation or evidence.

Bias: unequal failure or allocation under a relevant objective

Bias becomes operational when a team can state which benefit, burden or error differs, for whom, under what case mix, and why the difference matters to the intended decision. A dataset can be statistically unbalanced without producing a material decision disparity. A balanced dataset can still encode a biased target, proxy or institutional process. The risk statement therefore cannot stop at “training data may be biased”.

A published healthcare example makes the mechanism concrete. Obermeyer and colleagues examined an algorithm that used predicted healthcare cost as a proxy for health need. Because spending reflected unequal access and treatment, patients with similar scores did not have similar illness burdens across racial groups. The observable chain was target proxy to score to programme allocation to unequal access to extra care. The intervention was not a generic fairness pledge. It changed what the model was asked to predict.

Operational evidence can include subgroup calibration, conditional error rates, allocation rates, counterfactual or matched-case tests, appeal outcomes and qualitative review of who is excluded by the target definition. No single fairness metric settles the normative choice. Bias controls begin with the decision objective and affected population, not with a dashboard added after model training.

Error: a violated informational or behavioural contract

Error is the broadest class, so it needs the most disciplined grammar. It includes a wrong entity match, unsupported material claim, stale policy, arithmetic defect, misclassification, omitted exception, invalid tool argument or incorrect external action. “Hallucination” is too narrow for many of these and too vague for a register. The useful question is which contract was violated and how that violation can be observed.

Different errors demand different instruments. Entity equality and arithmetic are often deterministic. Claim support can be checked against cited evidence. Classification needs labelled outcomes and subgroup analysis. State-changing actions need preconditions, typed parameters and postcondition readback. A single model-based judge should not grade all of them. It adds uncertainty to properties that could have been checked exactly.

Privacy: purpose and data movement, even without an attacker

A privacy failure occurs when personal or sensitive data crosses a purpose, entitlement, retention or disclosure boundary. An employee pasting a customer record into an unapproved model can create privacy risk without exploiting a vulnerability. A retrieval system can lawfully store a document yet disclose it to an unauthorised user because entitlement was checked after retrieval. A semantic cache can reveal that a similar private question was asked before.

Carlini and colleagues demonstrated that querying a language model could recover individual training examples in their experimental setting. That event spans privacy and security: memorised data is the privacy exposure; deliberate extraction is the adversarial path. The primary class depends on the register's question and owner. The controls include data minimisation, purpose limitation, retention rules, privacy testing, output filtering, access control and evidence about provider-side handling. The ICO's AI and data protection toolkit is useful because it keeps the impact on people's rights and freedoms visible across the lifecycle.

Security: hostile influence over information, identity, authority or effect

Security requires an adversarial or unauthorised path. Prompt injection is a security failure when untrusted content changes what the system treats as instruction, obtains data or induces an effect. The OWASP GenAI Security Project describes prompt injection as input that alters model behaviour in unintended ways, including inputs not obvious to a human. ENISA's AI cybersecurity work likewise treats the model, data, infrastructure, process and wider ecosystem as layers of one security problem.

The observable chain should name the asset and violated property: an injected document causes the agent to treat data as control; the agent proposes a tool call outside the user's purpose; an authority kernel fails to reject it; the tool discloses another customer's record. “Jailbreak risk” alone does not reveal which boundary mattered. Security controls must remain enforceable even when the model is assumed to be a hostile or confused proposer.

Safety: unacceptable loss can arise from normal interaction

Safety concerns unacceptable harm under intended or reasonably foreseeable use. The harm may be physical, psychological, financial, environmental or rights-based, depending on the domain. Security asks whether unauthorised or adversarial influence compromises the system. Safety also asks whether the configured system can cause loss while every component behaves according to its local specification.

This systems view follows the core lesson of Nancy Leveson's STAMP: accidents in complex socio-technical systems can arise from unsafe interactions and inadequate control, not only from broken parts. For an AI system, a model may accurately estimate a local quantity while the workflow applies it outside its valid operating envelope. Safety analysis therefore needs loss scenarios, constraints, authority limits, human response conditions, containment and recovery exercises.

Concentration: a portfolio can fail together

Concentration is correlated exposure across systems, business units or organisations. Ten applications may each have a contract and fallback plan, yet all may use the same foundation-model endpoint, identity provider, embedding service, benchmark, evaluator or data feed. A common update can shift them together. A provider outage can exhaust all fallbacks if those fallbacks share the same region or control plane. A portfolio can therefore appear diversified at the application layer while remaining concentrated underneath.

The Financial Stability Board's analysis of AI in finance identifies third-party dependencies and service-provider concentration among vulnerabilities that may amplify systemic risk. At enterprise scale, the same logic changes the register. Measure the share of critical decisions exposed to one component, time to exit, data and interface portability, fallback independence and correlated error. Concentration usually belongs to a portfolio owner, not only to each application team.

Dependency: a local assumption can silently expire

Dependency is the narrower relationship. A system relies on an upstream or downstream component, human role, dataset, policy, tool, identifier or operating process. The dependency becomes risky when a change, outage or quality shift can invalidate assumptions that the dependent system does not test. The classic machine-learning engineering literature already warned about undeclared consumers, data dependencies, feedback loops and entanglement. Sculley and colleagues described these as sources of hidden technical debt.

A dependency record should state the contract, failure modes, version or freshness requirement, owner, detection, degraded mode and tested recovery. Listing vendors is not enough. A human reviewer is also a dependency if the control assumes that the reviewer has sufficient time, information and authority. A named fallback is not a control until the switch, data compatibility and operating capacity have been exercised.

The risk register as a typed argument

A conventional risk row often reads: “There is a risk that the AI hallucinates, causing customer harm. Mitigation: human in the loop.” Almost every noun is underspecified. Which operating context? What kind of unsupported claim? Which customer decision? What harm? Which reviewer, with what evidence and time? How will the team know the review occurred and corrected the defect?

A stronger grammar is deliberately longer:

The statement can be represented as a typed record rather than a score:

D is the decision or external effect at risk. T is the triggering condition. F is the local failure. P is the propagation path. H names plausible harm and affected entities. O is the observable signal. C contains prevention, detection, containment and recovery mechanisms. E records the test evidence, owner, system version and expiry. If a field is empty, the gap is visible instead of hidden by a red cell.

Primary and contributing classes

Multi-label classification is necessary, but it can recreate duplication. Give each entry a primary class based on the first material failure and accountable control owner. Add a contributing class only when it changes the control design, evidence, escalation or affected stakeholder. An adversary extracting memorised personal data may be primarily security for incident response and contributing privacy for notification and rights impact. A stale policy may be primarily error and contributing dependency because the publication service failed its freshness contract.

This convention does not claim that causal reality has one owner. It creates a clear first route while retaining cross-domain obligations. The risk register should link related records when one control serves several chains. It should not copy the same sentence into privacy, security, model-risk and operational-risk registers and then count four managed risks.

Priority without false precision

Many registers multiply likelihood and impact scores. This can be useful when the scales are empirically grounded and comparable. In novel AI systems they often are not. A “3” for uncertain prompt injection frequency is not commensurate with a “3” for customer loss. Multiplication creates decimal authority while allowing severe, irreversible harm to be offset by an optimistic likelihood estimate.

Keep a priority vector instead. The vector below uses ordinal anchors that a reviewer can inspect. Exposure and incident frequency remain separate evidence, not a licence to average away consequence.

Severity asks how serious the credible harm is. Reach asks how many decisions, people or systems may be exposed. Detection latency asks whether the team knows before, during or long after the effect. Irreversibility asks whether the state can be restored without residual harm. Common-cause exposure asks whether the same failure can affect other systems. Uncertainty asks how weak the evidence is about mechanism, frequency or consequence.

Decision gates can then remain explicit. Stop release when severe, hard-to-reverse harm lacks demonstrated containment. Escalate a critical shared dependency when fallback has not been exercised. Narrow authority when outcome readback is absent. Fund review capacity before selecting a threshold that sends thirty per cent of cases to people. The gate records the reason for the decision; a composite score usually hides it.

Controls need mechanisms, evidence and expiry

Every control should be classified by what it does. Prevention stops the trigger or local failure. Detection observes that it occurred. Containment limits propagation or authority. Recovery restores state, compensates affected parties or reconstructs the decision. One mechanism may perform several functions, but the evidence for each claim can differ.

“Human review” is not yet a control. A review control needs a defined role, information set, cue, decision right, time budget, workload assumption, escalation path and receipt. It also needs evidence that reviewers detect the injected failures at the intended operating volume. Likewise, a content filter is not evidence that a tool call was authorised. An evaluator that shares the target model's blind spots is not independent verification.

The evidence record should name the tested system version, population, injected failure, metric, result, owner and expiry trigger. Expiry may be a date, but component change is often more meaningful: new model, prompt, tool, source, policy, user group, authority level or provider. The EU AI Act's lifecycle risk-management requirement for high-risk systems reinforces the general principle that risk management is iterative rather than a one-off document.

Thought experiment: three fallbacks that are one

Three product teams document three fallback models. Each model has a different commercial name. All three are served through the same cloud region, use the same identity plane and depend on the same embedding index. A region or identity incident removes every option. At the application level, each team reports a tested dependency. At the portfolio level, the fallback is concentrated and not independent.

Now reverse one feature. Keep the primary model, but make the fallback a deterministic rules route with locally cached policy and separate identity. It has lower capability, yet it preserves a minimal service during the common failure. The lesson is not that every system needs multi-cloud duplication. It is that fallback quality must be evaluated against the failure domain it is supposed to survive.

Worked system: a bounded hardship-support assistant

The worked system is synthetic. A customer-support colleague opens an authenticated case. The system retrieves current policy and account facts, calculates eligible options with deterministic services, and asks a language model to draft a recommendation and explanation. A human colleague approves, changes or rejects the draft. A separate workflow tool records the final human decision. The model has no permission to change an account.

The risk propagation atlas applies all seven lenses to the same system. It does not claim that every path has equal severity or probability. It shows where each class begins, how it can move and what evidence would make the control claim testable.

Seven entries, not seven slogans

The register changes the release conversation. The team can decide to pilot the assistant as a recommendation tool while blocking account-changing authority. It can require bitemporal policy tests before claiming that retrieval is controlled. It can cap the pilot population until subgroup evidence is adequate. It can reject a model-based evaluator as the sole security control. It can ask the platform owner, rather than seven product teams, to evidence provider concentration and exit capability.

The taxonomy also makes residual risk legible. Human review reduces some propagation but introduces its own dependency and bias. An independent calculator eliminates arithmetic discretion but not stale inputs. A deny-by-default authority kernel contains hostile proposals but does not prevent the model from displaying harmful text. Controls narrow specific paths; they do not confer a general property called “safe AI”.

Executable risk-register validator

The accompanying Python artefact implements the typed record and transparent gates. It uses only the standard library. The positive synthetic case describes a stale-policy path with detection and recovery evidence. The negative case says little more than “hallucination risk” while granting irreversible authority; the validator rejects it. The script also detects shared dependencies across several systems, which is where local dependency turns into concentration.

#!/usr/bin/env python3
"""A small, dependency-free validator for operational AI risk records.

The artefact treats a risk as an observable failure chain rather than a label.
It uses synthetic examples and makes no claim to replace an organisation's
legal, safety, privacy, security, model-risk, or enterprise-risk methods.
"""

from __future__ import annotations

from dataclasses import asdict, dataclass, field
from datetime import date
from enum import Enum
import json
from typing import Iterable


class RiskClass(str, Enum):
    BIAS = "bias"
    ERROR = "error"
    PRIVACY = "privacy"
    SECURITY = "security"
    SAFETY = "safety"
    CONCENTRATION = "concentration"
    DEPENDENCY = "dependency"


class Authority(str, Enum):
    INFORM = "inform"
    RECOMMEND = "recommend"
    PROPOSE = "propose"
    EXECUTE_REVERSIBLE = "execute_reversible"
    EXECUTE_IRREVERSIBLE = "execute_irreversible"


@dataclass(frozen=True)
class Control:
    control_id: str
    kind: str  # prevent, detect, contain, recover
    mechanism: str
    evidence: str
    evidence_expiry: str  # ISO date

    def expiry_date(self) -> date:
        return date.fromisoformat(self.evidence_expiry)


@dataclass
class RiskEntry:
    risk_id: str
    system: str
    decision_or_effect: str
    primary_class: RiskClass
    contributing_classes: list[RiskClass]
    authority: Authority
    trigger: str
    local_failure: str
    propagation_path: list[str]
    affected_entities: list[str]
    plausible_harm: str
    observable_indicator: str
    owner: str
    severity: int  # 1 negligible, 4 severe
    reach: int  # 1 isolated, 4 portfolio or ecosystem
    detection_latency: int  # 1 immediate, 4 late or external discovery
    irreversibility: int  # 1 easily undone, 4 practically irreversible
    common_cause: int  # 1 diversified, 4 shared critical component
    uncertainty: int  # 1 well evidenced, 4 deeply uncertain
    controls: list[Control] = field(default_factory=list)
    dependencies: list[str] = field(default_factory=list)
    fallback_tested: bool = False
    outcome_observed: bool = True


REQUIRED_TEXT = (
    "system",
    "decision_or_effect",
    "trigger",
    "local_failure",
    "plausible_harm",
    "observable_indicator",
    "owner",
)


def validate_entry(entry: RiskEntry, today: date | None = None) -> list[str]:
    """Return validation findings. An empty list means the record is structurally usable."""
    today = today or date.today()
    findings: list[str] = []

    if not entry.risk_id or len(entry.risk_id.strip()) < 3:
        findings.append("risk_id is missing or too vague")

    for name in REQUIRED_TEXT:
        value = getattr(entry, name)
        if not value or len(str(value).strip()) < 8:
            findings.append(f"{entry.risk_id}: '{name}' is missing or too vague")

    if not entry.propagation_path:
        findings.append(f"{entry.risk_id}: propagation_path must name at least one transfer")
    if not entry.affected_entities:
        findings.append(f"{entry.risk_id}: affected_entities must not be empty")

    for name in (
        "severity",
        "reach",
        "detection_latency",
        "irreversibility",
        "common_cause",
        "uncertainty",
    ):
        value = getattr(entry, name)
        if value not in (1, 2, 3, 4):
            findings.append(f"{entry.risk_id}: {name} must be an ordinal from 1 to 4")

    if entry.primary_class in entry.contributing_classes:
        findings.append(f"{entry.risk_id}: primary class must not be repeated as contributing")

    control_ids = [control.control_id for control in entry.controls]
    if len(control_ids) != len(set(control_ids)):
        findings.append(f"{entry.risk_id}: duplicate control_id")

    kinds = {control.kind for control in entry.controls}
    if "detect" not in kinds:
        findings.append(f"{entry.risk_id}: no detection control is evidenced")
    if entry.irreversibility >= 3 and "contain" not in kinds:
        findings.append(f"{entry.risk_id}: high irreversibility requires a containment control")
    if entry.severity >= 3 and "recover" not in kinds:
        findings.append(f"{entry.risk_id}: severe harm requires a recovery control")

    for control in entry.controls:
        if not control.evidence or len(control.evidence.strip()) < 8:
            findings.append(f"{entry.risk_id}/{control.control_id}: control has no usable evidence")
        try:
            if control.expiry_date() < today:
                findings.append(f"{entry.risk_id}/{control.control_id}: evidence has expired")
        except ValueError:
            findings.append(f"{entry.risk_id}/{control.control_id}: invalid evidence_expiry date")

    if entry.authority == Authority.EXECUTE_IRREVERSIBLE and not entry.outcome_observed:
        findings.append(f"{entry.risk_id}: irreversible execution has no outcome observation")

    if entry.primary_class in {RiskClass.DEPENDENCY, RiskClass.CONCENTRATION}:
        if not entry.dependencies:
            findings.append(f"{entry.risk_id}: dependency identifiers are required")
        if not entry.fallback_tested:
            findings.append(f"{entry.risk_id}: fallback has not been tested")

    return findings


def priority_band(entry: RiskEntry) -> str:
    """Apply transparent gates; do not multiply ordinal scores into false precision."""
    severe = entry.severity == 4
    hard_to_reverse = entry.irreversibility >= 3
    broad = entry.reach >= 3
    late = entry.detection_latency >= 3
    shared = entry.common_cause >= 3

    if entry.authority == Authority.EXECUTE_IRREVERSIBLE and (
        not entry.outcome_observed or hard_to_reverse
    ):
        return "stop"
    if severe and (hard_to_reverse or broad) and (late or entry.uncertainty >= 3):
        return "stop"
    if (entry.severity >= 3 and (broad or late)) or (shared and not entry.fallback_tested):
        return "high"
    if entry.severity >= 2 or entry.reach >= 2:
        return "managed"
    return "monitor"


def shared_dependency_findings(entries: Iterable[RiskEntry]) -> list[str]:
    """Find common dependencies that can create correlated portfolio exposure."""
    dependency_map: dict[str, list[RiskEntry]] = {}
    for entry in entries:
        for dependency in entry.dependencies:
            dependency_map.setdefault(dependency, []).append(entry)

    findings: list[str] = []
    for dependency, dependent_entries in sorted(dependency_map.items()):
        systems = sorted({entry.system for entry in dependent_entries})
        untested = [entry.risk_id for entry in dependent_entries if not entry.fallback_tested]
        if len(systems) >= 3:
            findings.append(
                f"Common dependency '{dependency}' is used by {len(systems)} systems; "
                f"untested fallback records: {', '.join(untested) or 'none'}"
            )
    return findings


def assess(entries: list[RiskEntry], today: date | None = None) -> dict[str, object]:
    today = today or date.today()
    records = []
    for entry in entries:
        records.append(
            {
                "risk_id": entry.risk_id,
                "class": entry.primary_class.value,
                "priority": priority_band(entry),
                "findings": validate_entry(entry, today),
                "priority_vector": {
                    "severity": entry.severity,
                    "reach": entry.reach,
                    "detection_latency": entry.detection_latency,
                    "irreversibility": entry.irreversibility,
                    "common_cause": entry.common_cause,
                    "uncertainty": entry.uncertainty,
                },
            }
        )
    return {
        "records": records,
        "portfolio_findings": shared_dependency_findings(entries),
    }


def synthetic_examples() -> list[RiskEntry]:
    valid = RiskEntry(
        risk_id="R-017",
        system="Synthetic hardship-support assistant",
        decision_or_effect="Draft a repayment-option recommendation for human approval",
        primary_class=RiskClass.ERROR,
        contributing_classes=[RiskClass.BIAS, RiskClass.DEPENDENCY],
        authority=Authority.RECOMMEND,
        trigger="The policy retriever returns a superseded eligibility rule",
        local_failure="The assistant treats the obsolete rule as currently authoritative",
        propagation_path=[
            "stale policy chunk enters the context envelope",
            "unsupported eligibility claim enters the draft",
            "reviewer may accept the draft without checking the cited version",
        ],
        affected_entities=["customer", "case reviewer"],
        plausible_harm="An eligible customer may receive a less suitable repayment option",
        observable_indicator="Claim cites a policy version whose valid-to date precedes case time",
        owner="Customer-support product owner",
        severity=3,
        reach=2,
        detection_latency=2,
        irreversibility=2,
        common_cause=2,
        uncertainty=2,
        dependencies=["policy-publication-service"],
        fallback_tested=True,
        controls=[
            Control(
                control_id="C-101",
                kind="prevent",
                mechanism="Bitemporal policy filter rejects superseded clauses",
                evidence="Property test suite, build 48, 212 temporal cases passed",
                evidence_expiry="2030-12-31",
            ),
            Control(
                control_id="C-102",
                kind="detect",
                mechanism="Every material claim must cite policy ID and validity interval",
                evidence="Locked evaluation, 97 of 100 injected stale-source defects detected",
                evidence_expiry="2030-12-31",
            ),
            Control(
                control_id="C-103",
                kind="recover",
                mechanism="Human reviewer can reject draft and reconstruct from current policy",
                evidence="Quarterly recovery exercise completed with decision receipt retained",
                evidence_expiry="2030-12-31",
            ),
        ],
    )

    invalid = RiskEntry(
        risk_id="R-999",
        system="Chatbot",
        decision_or_effect="AI output",
        primary_class=RiskClass.ERROR,
        contributing_classes=[],
        authority=Authority.EXECUTE_IRREVERSIBLE,
        trigger="Hallucination",
        local_failure="Bad answer",
        propagation_path=[],
        affected_entities=[],
        plausible_harm="Risk",
        observable_indicator="Monitor it",
        owner="AI team",
        severity=5,
        reach=3,
        detection_latency=4,
        irreversibility=4,
        common_cause=4,
        uncertainty=4,
        dependencies=[],
        fallback_tested=False,
        outcome_observed=False,
        controls=[],
    )

    return [valid, invalid]


def main() -> None:
    examples = synthetic_examples()
    report = assess(examples, today=date(2026, 8, 30))
    print(json.dumps(report, indent=2))

    # The positive case must be structurally usable; the vague negative case must fail.
    assert report["records"][0]["findings"] == []
    assert report["records"][0]["priority"] == "managed"
    assert report["records"][1]["findings"]
    assert report["records"][1]["priority"] == "stop"


if __name__ == "__main__":
    main()

Inspect the implementation in this paper and Review the recorded evidence in this paper.

A compact release gate

stateDiagram-v2
  [*] --> Claim
  state "Do not mix the category, the<br/>failure and the harm" as Claim
  state "Thought experiment: the same<br/>model, the same error, a<br/>different system" as Test
  state "A discriminating test for the<br/>taxonomy" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Where the taxonomy stops helping

This taxonomy is an instrument for observable socio-technical failure chains, not a proof of acceptability or completeness. It cannot decide which fairness objective is morally or legally appropriate. It cannot reduce dignity, autonomy or distributional justice to a convenient metric. It cannot establish that a lawful purpose exists, that consent is valid, or that a human rights interference is proportionate. Those are substantive legal and normative decisions informed by, but not replaced by, the register.

It is also weaker for slow, diffuse and emergent harms. Labour-market deskilling, epistemic dependence, environmental effects and market-wide homogenisation may develop without one crisp incident. Strategic adversaries change their behaviour after controls become visible. Unknown interactions appear after scale. In such cases, scenario analysis, horizon scanning, stakeholder participation, portfolio monitoring and precaution may matter more than a neatly completed tuple.

Classification can itself create blindness. Teams may search only for the seven named classes, treat “other” as low priority or optimise indicators while the construct drifts. The countermeasure is not an eighth miscellaneous bucket. It is an explicit open-hypothesis register, near-miss review, external challenge and periodic re-framing of the system boundary. The taxonomy should be easy to revise when evidence shows that its categories route work badly.

A final boundary concerns evidence. A control can pass a synthetic injection and still fail under live complexity. A model can perform well on a subgroup sample with wide uncertainty. An incident-free pilot can simply be too small to reveal a rare event. Operationalisation makes assumptions testable; it does not make uncertain evidence certain.

Evidence and source ledger

sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: The risk register that knew every word and nothing useful
  M->>E: Do not mix the category, the failure and the harm
  E-->>M: compare rival explanations
  E->>D: Govern the chain, not the label
  D-->>Q: revise scope or act

Govern the chain, not the label

A good taxonomy does not make a register longer. It makes weak claims harder to write. “Bias” must become a statement about a decision-relevant disparity and its mechanism. “Privacy” must name the data movement or purpose boundary. “Security” must name adversary, asset and violated authority. “Safety” must connect system interaction to unacceptable loss. “Dependency” must expose a local assumption and tested degraded mode. “Concentration” must reveal correlated exposure that no application team can see alone.

The changed architecture decision is concrete: do not grant authority, scale population or accept residual risk from a category label and composite score. Require the typed chain, observable signal, control mechanism, evidence, owner and expiry. Use the primary class to route accountability and contributing classes to prevent tunnel vision. Preserve severity, reach, detection, reversibility, common cause and uncertainty as visible coordinates.

This paper therefore sits upstream of assurance. The taxonomy says what failure claims must exist. Trajectory assurance tests whether observable process events satisfy those claims. The certainty gradient places deterministic controls, models and people according to the evidence each can support. Together, they replace “AI risk is managed” with a narrower and defensible statement: for this system version, context and authority, these failure paths have been tested, these controls interrupt them, these outcomes remain observable, and these uncertainties still require accountable acceptance.