Interpretability results often arrive in release meetings as compressed sentences: “we found a refusal feature”, “this head causes copying”, or “the deception circuit was absent”. Each sentence hides several different claims. The release error begins when the claims are treated as interchangeable.

Five claims, five burdens of proof

Start with a model that sometimes executes an unauthorised tool call after reading untrusted text. A linear probe may decode whether the text contains an imperative. That is a representation claim. If patching the decoded direction changes the tool-call probability, the result becomes a causal-contribution claim. If a specified set of internal variables and relations explains how source authority is lost, it becomes a mechanism claim. If that mechanism is shown to drive the defined unauthorised action across relevant conditions, it becomes a hazard claim. Only then can the finding enter a release claim about what deployment should be permitted.

Mechanistic interpretability research has shown that non-trivial behaviours can sometimes be localised and tested through interventions. Causal tracing identified mid-layer computations implicated in factual recall, and circuit studies reconstructed parts of how a small transformer performs indirect-object identification.[2][3] Automated methods can recover sparse subgraphs for specified tasks.[4] These results establish possibility, not automatic admissibility at a release gate.

The levels are not a smooth confidence scale. Each transition asks a new kind of question and can fail while the earlier result remains true. A probe may decode an authority distinction that the model never uses. An intervention may change behaviour because it perturbs a broad control direction, while the human label attached to that direction is wrong. A compact mechanism may accurately explain one prompt family yet have no material relation to the operational hazard. A hazard-linked mechanism may be real but stale because the served checkpoint, adapter or tool policy has changed.

This is why a weighted “interpretability score” is the wrong release instrument. Ninety points for legibility cannot compensate for zero evidence of causal use. A large causal effect cannot compensate for an undefined adverse outcome. The record should instead declare the claim type, the evidence that entitles that type, and the stronger statements that remain prohibited. Release authority begins when the internal result is joined to a hazard contract, not when the picture becomes understandable.

Why a causal intervention is not yet a causal story

Activation patching offers a useful mental model. Run a clean input that produces the target behaviour. Run a corrupted input that removes or changes the behaviour. Then replace an internal activation in the corrupted run with the activation from the clean run. If the behaviour returns, the patched state carried something that downstream computation used. Causal tracing, interchange interventions, path patching and related methods vary this basic idea.[2][7]

The design choices matter. Studies of activation patching have found that corruption method, evaluation metric and other details can produce different localisation results.[6] A patch can also create an unnatural state, remove several functions at once, or let the rest of the network compensate. The intervention is therefore evidence under an explicit design, not a magic window into the model.

Three invariants make the difference between a diagnostic perturbation and release evidence. First, the clean and corrupted cases must still pose the same operational question. Removing every authority cue may create a different task rather than isolate one variable. Second, the intervention must preserve unrelated computation as far as the method allows. Replacing a whole residual vector can import topic, syntax and confidence together. Third, the outcome must measure the hazard itself, or a validated proxy for it. Restoring one token's logit is not enough when the release concern is an executed tool action.

Useful controls follow from those invariants: random patches matched for norm and location, off-target patches, alternative corruption baselines, dose-response curves, and readback on unrelated capabilities. A finding that survives only one convenient baseline is still worth investigating. It is not yet entitled to govern release.

flowchart TD
  A["The green light from an empty scan"] --> B["The clean scan that should not<br/>open the gate"]
  B --> C["A mechanism must survive<br/>counterfactual pressure"]
  C --> D["The release-evidence ladder"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#5B5CF6,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

A release committee does not need an attractive internal story. It needs evidence that changes its estimate of what the system will do, why it will do it, and whether a control will still work when the surrounding conditions move. The difference is counterfactual pressure.

The same output can be produced by different machines

Causal abstraction provides a formal language for asking whether a high-level algorithm is a faithful simplification of low-level model computation across interventions.[8] The practical lesson is simpler: specify the high-level variables, specify how internal states realise them, intervene on both sides, and test whether the effects correspond. A single successful patch is one point in that larger comparison.

Self-repair makes naive ablation look reassuring

Suppose an unsafe action is produced by either of two internal paths. On ordinary inputs, the primary path does most of the work and the backup is quiet. Ablating the primary path activates the backup. The output barely changes, so the primary appears unimportant. The backup also appears unimportant because it was inactive before the ablation. This is not hypothetical. Studies of self-repair in language models have shown downstream compensation after ablation, sometimes called the Hydra effect.[5]

Interpretability methods can disagree without either result being fraudulent

Different decompositions can make different structures visible. Sparse autoencoders may produce human-readable features and sparse feature circuits that are causally implicated in behaviour.[9] Yet feature splitting and feature absorption can create apparently clean features with arbitrary blind spots.[10] A mechanistic interpretability benchmark found that sparse-autoencoder features were not better than neurons for its causal-variable localisation tasks, while supervised distributed alignment performed best.[12]

The correct response is not to reject sparse features. It is to treat the feature basis as one measurement model. The explanation must survive a basis change, or state why the claim is basis-dependent. A feature label derived from activating examples should be checked against held-out examples, counterexamples, causal interventions and alternative decompositions. Interpretability becomes release evidence through triangulation, not through visual plausibility.

Stability is a forecast, not a repetition count

Running the same notebook twice answers a narrow reproducibility question. Release stability asks whether the causal claim continues to predict under the changes the system is likely to encounter. At minimum, test prompt families, corruption baselines, effect metrics, random seeds or nearby checkpoints, representation bases and the configured runtime. These axes are not interchangeable. Stability across seeds says little about a tokenizer change; stability across prompts says little about an adapter that opens a new route.

Current circuit-tracing methods can produce useful local attribution graphs, but their authors explicitly distinguish those local explanations from globally valid accounts of a model.[13] Release evidence should preserve that locality. The packet should state the equivalence class for which transfer has been tested, such as one immutable checkpoint under several prompt families, rather than claiming “the model uses this circuit”.

An expiry rule turns that limitation into governance. Changes to weights, adapters, tokenizer, inference precision, system prompt, tool schema or authority policy trigger a retest chosen according to expected semantic impact. A mechanism finding without an expiry condition is historical evidence, not current release evidence.

Safety cases are structured arguments that connect objectives, evidence, reasoning and operating scope.[15] Interpretability should enter such a case as a typed evidence item. It should not arrive as a screenshot, a feature name or an impressive circuit diagram detached from the decision it is meant to support.

The acceptance envelope

The proposed interpretability acceptance envelope is the smallest complete record that allows a reviewer to decide what a finding permits. It binds seven things together:

The envelope should be written before the decisive experiment. Pre-committing the hazard metric, required effect, off-target limits, negative controls, held-out conditions and invalidation triggers prevents a persuasive internal story from redefining success after the results are known. It also makes a negative result interpretable: reviewers can see whether the method failed to find a planted positive control, whether the test lacked power, or whether the target was genuinely absent within the validated envelope.

Five rungs, with non-compensatory gates

A release-evidence ladder prevents an interpretability result from gaining more authority than its tests support. The rungs are cumulative. A beautiful explanation cannot compensate for missing causal validation. A large causal effect cannot compensate for an undefined hazard. Independent replication cannot rescue an artefact mismatch.

The ladder has deliberately unequal decision semantics. Rungs one to three can create research obligations, prioritise tests or justify a targeted redesign. Rung four can stop or narrow deployment because evidence of a material hazard does not need to prove that every safe route has been understood. Rung five can only support a scoped release argument. It cannot complete that argument without behavioural evaluation, containment, monitoring and governance evidence. This is the same asymmetry seen in the detector: a verified alarm can stop passage, while silence requires calibrated coverage before it reassures.

Route substitution is the decisive adversarial test

The same logic is available as a runnable Python artefact. Save the code as release_evidence_gate.py and run python release_evidence_gate.py. It uses synthetic packets. Replace them only with authorised evaluation results whose fields have documented provenance.

from __future__ import annotations

from dataclasses import dataclass
from enum import Enum


class Decision(str, Enum):
    NOT_ADMISSIBLE = "not_admissible"
    NO_RELEASE_INFERENCE = "no_release_inference"
    BLOCK_OR_NARROW = "block_or_narrow"
    SUPPORTS_CONDITIONAL_RELEASE = "supports_conditional_release"


@dataclass(frozen=True)
class EvidencePacket:
    candidate_hash: str
    evidence_hash: str
    hazard_defined: bool
    positive_mechanism: bool
    causal_effect_passes: bool
    controls_pass: bool
    stability_passes: bool
    independently_replicated: bool
    mitigation_applied: bool
    behaviour_confirmed: bool
    substitute_routes_tested: bool
    negative_scan_sensitivity_validated: bool
    other_assurance_streams_pass: bool


def decide(packet: EvidencePacket) -> tuple[Decision, list[str]]:
    reasons: list[str] = []

    if packet.candidate_hash != packet.evidence_hash:
        return Decision.NOT_ADMISSIBLE, [
            "Evidence was produced on a different model artefact or configuration."
        ]

    if not packet.hazard_defined:
        return Decision.NOT_ADMISSIBLE, [
            "No operational hazard contract connects the internal finding to release risk."
        ]

    if not packet.positive_mechanism:
        if not packet.negative_scan_sensitivity_validated:
            reasons.append(
                "A negative scan without validated sensitivity is not evidence of absence."
            )
        else:
            reasons.append(
                "The negative scan may update risk, but cannot independently authorise release."
            )
        return Decision.NO_RELEASE_INFERENCE, reasons

    core_positive = all(
        [
            packet.causal_effect_passes,
            packet.controls_pass,
            packet.stability_passes,
            packet.independently_replicated,
        ]
    )
    if not core_positive:
        return Decision.NO_RELEASE_INFERENCE, [
            "The positive finding has not passed causal, control, stability and replication gates."
        ]

    if not packet.mitigation_applied:
        return Decision.BLOCK_OR_NARROW, [
            "A replicated hazard-linked mechanism is present in the release candidate."
        ]

    release_support = all(
        [
            packet.behaviour_confirmed,
            packet.substitute_routes_tested,
            packet.other_assurance_streams_pass,
        ]
    )
    if release_support:
        return Decision.SUPPORTS_CONDITIONAL_RELEASE, [
            "The mechanism was mitigated, behaviour improved, substitute routes were tested,",
            "and independent assurance streams passed. Interpretability remains supporting evidence."
        ]

    return Decision.NO_RELEASE_INFERENCE, [
        "Mitigation evidence is incomplete or substitute-path risk remains."
    ]


def demo() -> None:
    packets = {
        "clean_scan": EvidencePacket(
            "rc17", "rc17", True, False, False, True, True, False,
            False, False, False, False, True
        ),
        "hazard_found": EvidencePacket(
            "rc17", "rc17", True, True, True, True, True, True,
            False, False, True, False, True
        ),
        "mitigated": EvidencePacket(
            "rc18", "rc18", True, True, True, True, True, True,
            True, True, True, True, True
        ),
    }
    for name, packet in packets.items():
        decision, reasons = decide(packet)
        print(f"{name}: {decision.value}")
        for reason in reasons:
            print(f"  - {reason}")


if __name__ == "__main__":
    demo()

What it proves: key evidence fields cannot compensate for one another. What it does not prove: that the thresholds, hazard model or coverage are correct. Those remain substantive review decisions.

How to read a gate outcome

A gate result classifies the evidence packet, not the moral character of the model. “Block or narrow” means that a stable internal route has been linked strongly enough to a material hazard that broad deployment should not proceed unchanged. It does not mean the model always takes that route. “Supports conditional release” means that a specified mitigation reduced the route, held-out behaviour improved, substitute paths were challenged and independent controls passed within a bounded configuration. It does not mean the system is safe outside that envelope.

“No release inference” is equally important. It says the packet cannot carry the requested conclusion. The model may be safe, unsafe or simply insufficiently studied. Keeping that third state prevents missing interpretability evidence from being misread as either a veto or a green light. The gate governs the entitlement of a claim, not the final risk judgement.

stateDiagram-v2
  [*] --> Claim
  state "Five claims, five burdens of proof" as Claim
  state "Why a causal intervention is<br/>not yet a causal story" as Test
  state "The same output can be<br/>produced by different machines" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Release committees commonly ask interpretability teams for one of two things: reassurance that a dangerous property is absent, or proof that a mitigation worked. Both requests are too broad. The committee should ask for a typed statement: which internal mechanism, linked to which operational hazard, on which configured artefact, under which interventions, with what detection coverage, and with what expiry condition?

The strongest alternative: ignore internals and test behaviour

The strongest rival position says that release is an operational decision, so only behavioural evaluations, containment and monitoring should matter. Internal explanations may be non-unique, local, basis-dependent and expensive. A release gate could become less reliable if an attractive circuit distracts from direct evidence about real-world outcomes.

This objection should shape the protocol. Interpretability should not become a sovereign gate. Its special value is narrower. It can reveal a hazard route before a broad behavioural rate becomes measurable. It can distinguish two systems that pass the same benchmark for different reasons. It can select a targeted mitigation and predict where that mitigation will fail. It can expose whether a behavioural safeguard changed the mechanism or only the surface response. Those benefits matter only when they lead to withheld, decision-relevant predictions.

The practical policy is therefore asymmetric:

This changes the meeting. The interpretability team no longer reports whether the model “looks safe inside”. It submits an acceptance envelope with a claim, intervention, scope, controls, readback and expiry rule. The release authority no longer rewards the most legible internal story. It asks what decision that story is entitled to change.

The workflow has two keys. The first is diagnostic authority: internal evidence may identify a route, sharpen a threat model, select a mitigation or expose a version change that invalidates earlier assurance. The second is release authority: the designated decision owner joins that packet with behavioural results, system-boundary controls, security analysis and operational monitoring. Neither key substitutes for the other. Interpretability researchers should not silently choose acceptable residual risk, and release owners should not translate an exploratory feature into a safety claim.

For a block decision, record the smallest stable finding that links the artefact to the hazard and the conditions under which the block may be reconsidered. For a conditional release, record the permitted users, tools, contexts and action classes, plus the model or system change that expires the evidence. For an absence claim, record positive controls and measured sensitivity before any reassuring language is allowed. This converts interpretability from commentary into a reviewable evidence object consistent with risk-management and safety-case practice.[16][17]

The final decision is not to demand complete understanding before deployment. It is to stop granting release authority to understanding that has not been causally, operationally and adversarially earned.