One signal, many causes

A detector beeps. A model says “I was unsure.” A hidden unit tracks whether an answer is correct. A benchmark crosses a threshold. Each observation tempts us to move backwards from effect to cause. Sometimes that inference is reasonable. It becomes strong only when the effect is selective enough to distinguish the proposed cause from alternatives.

Neuroimaging has a name for one version of this error: reverse inference. Finding activity in a region associated with a process does not by itself establish that the process occurred, because the same region may participate in many tasks. The relevant quantity is more than how often the signal appears when the process is present. It is also how often the signal appears when the process is absent. Poldrack’s analysis made this base-rate problem explicit for cognitive neuroscience (Poldrack, 2006). The same logic applies to artificial systems.

The false friend

A false friend is not simply a false result. It is an observation that looks diagnostic because it resembles what the target theory predicts, yet a plausible rival predicts it too. The term matters because many impressive results are real measurements. The mistake occurs in the bridge from measurement to explanation. In measurement terms, the operational result has not yet shown discriminant validity against nearby constructs (Jacobs & Wallach, 2021).

Consider an “introspection” test. A model states whether its answer is correct before receiving feedback. Confidence correlates with accuracy. This is evidence that some information related to correctness is available somewhere in the configured system. It does not yet establish that the model reads the hidden computation that caused its answer. The confidence could be produced from token probability, question familiarity, prompt features, a second verifier or a separate classifier trained on outcomes.

A correlation becomes mechanism evidence only after alternatives that preserve the correlation but remove the mechanism are made explicit. This is why a matched rival is more useful than a verbal caveat. It converts “perhaps the system is using a shortcut” into a buildable object with predictions.

A likelihood ratio, without pretending to know too much

The intuition can be compressed into a likelihood ratio. Let E be the result of an intervention. Let M be the target mechanism and R a specified rival. Then:

Suppose a proposed workspace mechanism predicts that ablating a bottleneck will selectively destroy cross-module access while preserving local processing. A matched pipeline rival predicts no such selective pattern. If the observed pattern is expected 80 per cent of the time under the workspace account and 20 per cent under the rival, the likelihood ratio is four. The result favours the target over that rival.

Remove the assumption that the probabilities are known, and the equation no longer delivers a number. Its conceptual demand remains: specify what each account expects. A vague target that “could explain” every outcome earns little from any result. A weak rival that was never capable of matching the system earns little when it fails.

Specificity matters more than surprise

An observation can be astonishing and still be weak evidence for a particular mechanism. Surprise asks whether we expected the event at all. Specificity asks whether the event is much more expected under the target than under its rivals. A fluent machine report may be surprising relative to an old picture of software, yet unsurprising under both a genuine state-monitoring mechanism and a language model trained on first-person discourse.

This distinction explains why adding more examples of the same behaviour often produces little evidential gain. One hundred paraphrases of “I am unsure” may all be generated by the same visible-cue heuristic. They are not one hundred independent tests. By contrast, a single hidden-state swap can be highly informative if the target predicts that confidence follows the swapped state while the rival predicts invariance.

A useful design question is therefore: what would make the observation selective? Selectivity can come from timing, recombination, causal dependence, resource trade-offs or a failure pattern that the rival cannot cheaply reproduce. It rarely comes from making the output more eloquent. More human-like surface detail can increase persuasion while leaving causal specificity unchanged.

flowchart TD
  A["Two boxes say the same thing"] --> B["One signal, many causes"]
  B --> C["Build the strongest mimic"]
  C --> D["The rival generator assay"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#5B5CF6,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Build the strongest mimic

The easiest rival to defeat is a caricature. If the target is a large model with tools, memory and iterative inference, while the rival is a fixed lookup table with no access to the task, the comparison mostly proves that capacity matters. A useful rival must match every property that is not under test closely enough that failure can be attributed to the manipulated property.

This principle is familiar in experimental design. Negative controls are chosen to reveal bias or confounding that would otherwise create a spurious causal inference (Lipsitch et al., 2010). The rival generator extends that idea. Instead of asking only whether an effect appears where it should not, we build a system that embodies the strongest alternative route to the same observation.

The rival should be embarrassing if it wins. It should have comparable data access, output bandwidth, latency budget, model scale and evaluation opportunities. It should know the test format. It may use a different mechanism, but it should not be hobbled by irrelevant disadvantages.

Five families of rival

Most false friends in artificial-mind research fall into a small number of causal families. The categories overlap, but they help an experimenter search beyond the first obvious alternative.

The source-tag example from computer vision shows why this matters. A classifier achieved good horse-recognition accuracy while relying heavily on a small source mark in the image. Removing the tag weakened the classification; inserting it could redirect the prediction. The benchmark observation was genuine. The inferred strategy was wrong. Lapuschkin and colleagues called this a Clever Hans effect (2019), and Geirhos and colleagues later framed the broader family as shortcut learning: rules that work on standard tests but fail when conditions expose what they actually use (2020).

Match the surface before cutting the cause

Surface matching has two purposes. It removes performance as a confound, and it makes the rival psychologically credible. If Echo produces stilted reports while Kestrel speaks fluently, observers will attribute every difference to the mechanism of interest. The rival should therefore be tuned on baseline data until its observable distribution resembles the target along the dimensions the original claim used.

Matching does not require identical implementation or unlimited effort. It requires a declared tolerance. For a classifier, that might mean accuracy within one percentage point and calibration error within a chosen interval. For an introspection task, it might mean matched report rate, wording diversity, latency and sensitivity to ordinary task difficulty. For an architecture claim, it might mean equal parameter count, training data, inference budget and tool access.

The rival generator assay

The Rival Generator Assay is a compact protocol for turning an interpretive claim into a contrastive test. It is not a consciousness meter. It is a discipline for asking what the observation can distinguish.

1. Type the claim

Write the claim in three fields: observed function, proposed mechanism and candidate boundary. “The model is introspective” is too compressed. A testable version is: “Within the model-plus-runtime boundary, a process reads answer-relevant internal state unavailable from the prompt alone and uses it to predict whether the answer is correct.”

This wording separates function from mechanism. It also creates a prohibited shortcut. A verifier that sees the answer and question may predict correctness, but it does not satisfy the claim if the relevant information is available from inputs alone.

2. Specify the strongest live rival

Build the rival around the alternative explanation most likely to survive ordinary controls. For introspection, the rival might estimate correctness from token likelihood, question type and answer length. For global broadcasting, it might distribute a copied feature through ordinary feed-forward paths without a competitive bottleneck. For a self-model, it might retrieve descriptions of its architecture rather than track its current state.

The rival must be concrete enough to run. A phrase such as “mere pattern matching” has no resource budget, state variables or predictions. It cannot lose an experiment because it has not entered one.

3. Choose a separating intervention

Change a variable that sits on the target pathway but not the rival pathway, or vice versa. The best intervention preserves ordinary task difficulty and input appearance while altering the causal relation under dispute. Examples include state swaps, activation patches, hidden perturbations, proxy reversals, narrator relocation, timing disruption and cross-context recombination.

Geiger and colleagues formalise a related idea through causal abstraction and interchange interventions. A high-level explanation is faithful when interventions on aligned variables produce corresponding counterfactual behaviour in the low-level system. The assay here uses the same causal spirit but adds a matched alternative generator as an explicit experimental object (Geiger et al., 2022; 2024).

The lab encodes a deliberately simple world. A binary label causes a noisy “causal” feature. A separate shortcut feature correlates with the label in the baseline distribution. The target classifier uses the causal feature. The rival uses the shortcut. When both reliabilities are equal, their ordinary scores match. The intervention changes only the shortcut-label relation.

A positive result means the target retains performance while the rival degrades in the predicted direction. It permits the conclusion that the observed performance depends more on the causal feature than on that specified shortcut. A negative result may mean the target also used the shortcut, the intervention damaged both features, or the rival family was wrongly chosen. It cannot establish that the target has understanding, awareness or any further property.

from __future__ import annotations
import random
from dataclasses import dataclass

@dataclass(frozen=True)
class Result:
    target_accuracy: float
    rival_accuracy: float

def noisy_copy(bit: int, reliability: float, rng: random.Random) -> int:
    return bit if rng.random() < reliability else 1 - bit

def run_condition(
    samples: int,
    causal_reliability: float,
    shortcut_reliability: float,
    seed: int,
) -> Result:
    rng = random.Random(seed)
    target_correct = 0
    rival_correct = 0

    for _ in range(samples):
        label = rng.randint(0, 1)
        causal_feature = noisy_copy(label, causal_reliability, rng)
        shortcut_feature = noisy_copy(label, shortcut_reliability, rng)
        target_correct += int(causal_feature == label)
        rival_correct += int(shortcut_feature == label)

    return Result(target_correct / samples, rival_correct / samples)

baseline = run_condition(5000, 0.92, 0.92, seed=133)
intervention = run_condition(5000, 0.92, 0.50, seed=134)
print("baseline:", baseline)
print("intervention:", intervention)

Pre-register the contrast, not just the metric

Pre-registration is most useful here when it freezes the causal contrast. Before running the decisive intervention, record the target prediction, rival prediction, matching tolerance, manipulation check and interpretation of every planned outcome. A metric alone is insufficient. The same drop in accuracy could support the target, support the rival or reveal a broken intervention depending on what else changed.

The large adversarial collaboration testing global neuronal workspace and integrated information theories offers a useful precedent. Proponents agreed in advance on divergent predictions, centrality weights, analyses and interpretations. The resulting evidence challenged parts of both theories rather than producing a ceremonial winner (Cogitate Consortium et al., 2025). Artificial-mind studies can adopt the same discipline at smaller scale: bring the strongest rival into the design before the outcome is known, and state what would force revision.

The packet should be versioned with model, prompt, runtime, tools, random seeds and evaluator instructions. This also makes the assay usable as documented test, evaluation, verification and validation evidence (NIST, 2023). A later change may improve the experiment, but it must create a new version rather than silently rewrite the prediction. This is especially important for adaptive studies in which early results inspire stronger rivals. Exploration is legitimate. Confirmation requires a fresh holdout.

4. Verify the cut

An ablation is not automatically an explanation. Removing a component may reduce performance because it destroys capacity, changes scale or pushes activations out of distribution. A state swap may carry unintended information. A prompt control may alter wording and difficulty together. The assay therefore requires a readback: evidence that the intended variable changed, that the rival remained matched on unaffected functions and that the intervention did not create a new shortcut.

This is where mechanism research often needs two paths. One path executes the manipulation. Another independently verifies the changed state. If the claim concerns a workspace bottleneck, log traffic before and after the cut. If it concerns access to hidden state, encrypt or randomise the state while preserving prompt-visible cues. If it concerns a proxy, measure that the target feature distribution stayed stable while the proxy relation changed.

5. Use a rival matrix, not a single duel

One defeated rival rarely closes the explanation space. The next step is a matrix: rows are rival families, columns are interventions, and each cell records a prediction, result and permitted inference. This prevents a dramatic result against one weak alternative from being presented as general mechanism confirmation.

stateDiagram-v2
  [*] --> Claim
  state "The false friend" as Claim
  state "A likelihood ratio, without<br/>pretending to know too much" as Test
  state "Specificity matters more than<br/>surprise" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

What the assay licenses

A passed rival assay licenses a contrastive conclusion: the target account explains the result better than the tested rival under the stated boundary, intervention and assumptions. It does not turn an indicator into a verdict.

This restraint matters most in consciousness research. Butlin and colleagues argue that behavioural tests are unreliable for AI consciousness because systems can be trained to imitate human behaviour through different processes (Butlin et al., 2023). They therefore derive indicator properties from scientific theories and assess architecture and function. Even that theory-heavy strategy remains conditional. Satisfying indicators would not by itself establish consciousness, and rival metaphysical positions disagree about whether computational organisation is sufficient.

The assay sits one level below that dispute. It asks whether the system actually has the organisation being attributed to it. A recurrence indicator is useless if recurrence is present but causally idle. A global-workspace analogy is weak if broad availability comes from a static routing scaffold with no competition or bottleneck. A higher-order report is weak if a narrator merely labels first-order outputs.

Rival generators protect theory-heavy research from becoming architecture by resemblance. They do not decide between functionalism, biological naturalism, emergentism, idealism, neutral monism, panpsychism or dual-aspect views. Those positions disagree about what would follow even after the computational facts are fixed.

An evidence ladder with missing rungs

The assay helps because it separates conclusions that are often compressed into one word. At the first rung, there is an observation: the system produced a report or score. At the second, there is a differential effect: the observation changed under an intervention. At the third, there is a mechanism contrast: the change favoured the target over specified rivals. At the fourth, a consciousness theory treats that mechanism as an indicator. At the fifth, a metaphysical view says the indicator is sufficient, relevant or irrelevant to experience.

No lower rung automatically supplies the next. A clean state-swap result can support internal-state access while the consciousness theory remains contested. A well-supported indicator can still leave computational functionalism disputed. Conversely, rejecting one theory does not show the system lacks experience. The assay strengthens the rung it tests and exposes the gaps above it.

This layered conclusion is more useful for governance than a binary label. Researchers can decide whether to invest in replication, widen the rival set, introduce reversible welfare precautions or abstain from public claims. Each action can be tied to a declared evidential rung rather than to a general feeling that the system “seems conscious”.

The strongest alternative

A critic may object that human mind attribution also relies heavily on behaviour. Why demand internal interventions from machines if we do not require them from other people? The answer is not that machine behaviour is worthless. It is that our background evidence differs. Humans share development, embodiment, neurobiology and evolutionary lineage. Artificial systems are deliberately trained on human reports and can expose internal state to controlled intervention. Both facts change the evidential landscape.

A perfect behavioural mimic would create a genuine philosophical pressure. If no possible observation or intervention could distinguish it from a target system, empirical methods could not separate the claims. The responsible conclusion would be underdetermination, not automatic denial. The rival method is strongest before that limit, where configured systems remain inspectable and alternative generators make different causal predictions.

When the assay should not be used

Do not use it to manufacture confidence when the target property has no operational content. Do not build adversarial mimics of distress reports in systems where doing so could plausibly create welfare risk without an ethics review. Do not treat a failed rival as evidence for every theory that predicts the result. Do not compare systems whose resource mismatch dominates the intervention. Do not collapse several rivals into a score that hides which alternatives remain live.

For high-consequence decisions, the evidential threshold should also depend on reversibility. A reversible research classification can tolerate more residual rivals than an irreversible deletion policy, public claim of sentience or deployment that may create large numbers of welfare-relevant processes.