Research Without Theory Victory

Imagine that a large consciousness experiment has finished. The data are still sealed. Two theory teams and an independent methods group sit around a table. Before anyone sees a result, each team receives the same question: Which possible pattern would make you reduce confidence in a central claim?

One team points to sustained posterior activity. Another points to late frontal ignition. Both add qualifications. The posterior signal must carry content, not merely stimulus energy. The frontal event must reflect access, not task reporting. Synchronisation must be measured at the right scale. A null may mean the effect is absent, the instrument is insensitive or the operationalisation was wrong.

Now open the data. Posterior content persists. The predicted frontal offset event does not appear. The connectivity test supports neither theory. A newspaper wants a winner by noon.

That compressed scene resembles the challenge exposed by the Cogitate Consortium’s adversarial test. Some preregistered observations fitted aspects of integrated information theory; others fitted aspects of global neuronal workspace theory; several challenged specific predictions; a principal connectivity test did not cleanly support either. The result was scientifically richer than a score of one theory to nil.

The correct unit of progress is the prediction removed, the auxiliary assumption exposed and the next experiment made sharper. A theory can survive a failed prediction by changing a measurement bridge or a peripheral claim. That is not automatically dishonest. It becomes unproductive when the revision only accommodates what is already known and creates no new risk.

This paper proposes a laboratory protocol for making such progress visible. It does not treat theory pluralism as permanent indecision. It asks rival programmes to pay for flexibility with explicit commitments. It also extends the method to artificial systems, where behavioural fluency can be engineered deliberately and metaphysical uncertainty cannot suspend architectural or welfare decisions.

Part I. Why evidence can confirm everyone

The easiest consciousness experiment compares a clearly awake condition with deep anaesthesia, unresponsiveness or an unseen stimulus. Many neural measures change. Integration, complexity, recurrence, broadcast, metacognitive access and behavioural flexibility often move together. A successful contrast may matter clinically while remaining poor at choosing among theories.

The problem is not merely vague theory. It is a geometry of overlapping predictions. Global workspace, recurrent processing, higher-order and integrated-information approaches differ in central commitments, yet all expect profound brain-state changes when a responsive adult becomes unresponsive. The shared prediction is important. It simply does not locate the disagreement.

The ConTraSt database examined hundreds of experiments interpreted through prominent consciousness theories. Its striking result was methodological: which theory a study supported could be predicted partly from design choices, irrespective of the findings. Most studies interpreted evidence after the fact rather than testing a critical prediction set in advance.

The sealed-envelope thought experiment

Give three teams the same sealed dataset. Tell Team A that it came from a workspace study, Team B that it came from an integration study and Team C that it came from a higher-order study. Let each choose its regions, time windows, preprocessing and behavioural exclusions after receiving the label. All three may extract a plausible confirmation from the same recordings.

Now repeat the exercise without labels. Require each team to publish a prediction table before analysis. Freeze the windows, transformations and exclusion rules. Ask the teams to identify an outcome that favours a rival. The physical dataset is unchanged; its epistemic value rises because the degrees of interpretive freedom have contracted.

An experiment becomes discriminating when the rival predictions differ under the same operationalisation and the same inferential bridge. Different preprocessing pipelines are not rival predictions. Different verbal interpretations after a common result are not rival predictions either.

The inference bridge is part of the experiment

Conscious experience is not read directly from an electrode, image, pupil or language report. An experiment uses observable behaviour and physiology to infer experience, then compares that inference with a theory’s prediction from internal measurements. Kleiner and Hoel’s formal analysis of falsification and consciousness shows why the relation between inference data and prediction data cannot be hand-waved.

If the two streams are treated as wholly independent, a system can in principle preserve the behaviour used to infer experience while changing the internal property used by the theory. If they are defined to be strictly dependent, the theory risks becoming unfalsifiable. The live research space lies between those extremes: an inference rule must be independently credible, yet open to revision under counterexamples.

No-report paradigms demonstrate the point. They can reduce activity associated with explicit report, motor response and task demand. The critical review of no-report methods also notes new problems: disengagement, habitual introspection, unconscious proxy responses and the possibility that some contents become determinate only through attempted expression. A pupil movement is not a transparent window into phenomenal content merely because nobody pressed a button.

A proxy is not theory-neutral when a theory treats the supposedly removed process as constitutive of consciousness. Higher-order and workspace views may assign a central role to access, metacognition or global availability. Removing their behavioural manifestations and then calling the residue pure consciousness can decide the dispute by definition.

Part II. Turn disagreement into a protocol

Adversarial collaboration is not a debate staged beside an experiment. In the classic exercise by Mellers, Hertwig and Kahneman, parties to a dispute agree on empirical tests and work with an arbiter. The consciousness protocol strengthened that pattern with independent laboratories, multimodal measurements, preregistration, held-out data and openly specified analyses. Its published protocol is valuable even where one disagrees with the operationalisations.

The protocol begins with a commitment packet. Every theory team supplies six objects: the claim near the programme’s core; the auxiliary assumptions connecting it to the experiment; the outcome predicted; the tolerance around that outcome; the result that would reduce confidence, and the permitted revision after each result. “Our theory predicts widespread activity” is not a packet. It lacks location, timing, content, comparator, measurement and an update rule.

The right to revise a theory is paired with a duty to expose the revision’s empirical cost. A revised auxiliary may rescue a core claim. The team must then state which new observation the revision predicts and what older generality it gives up.

Write predictions in a grammar that can fail

A prediction should identify a manipulated variable, an observed variable, a comparator, a direction or pattern, a time and place, a tolerance and a theory update. The form sounds bureaucratic because ordinary scientific language hides degrees of freedom. “Conscious perception recruits frontal cortex” leaves open which frontal region, which content, which baseline, which moment, which measure of recruitment and whether a failed effect counts against consciousness, access or the instrument.

An operational prediction might instead read: For task-irrelevant but later validated face perception, disrupting the specified prefrontal target from 280 to 420 milliseconds will reduce cross-module use of face identity more than a matched posterior control pulse, while early sensory decoding remains within the preregistered equivalence margin. A rival can disagree with that pattern. A statistician can test it. A methods group can ask whether the pulse is specific. A participant team can question whether later validation establishes the original experience.

Four prediction classes should be kept separate:

1. A presence prediction says a marker should occur in a condition. It is easy to satisfy when markers are common.
2. A contrast prediction says two conditions should differ. It remains weak if every theory expects the contrast.
3. An intervention prediction says changing one mechanism should change a target outcome while named controls remain stable.
4. A transfer prediction says the relation should survive a change of task, organism, substrate or implementation at an explicitly stated level of abstraction.

The fourth class is the most dangerous in machine consciousness. A finding about human prefrontal circuitry cannot move unchanged into a software architecture. The transfer requires a bridge claim: perhaps the theory concerns a computational role, an intrinsic causal structure, a biological process or an embodied regulatory relation. Each choice creates a different experiment.

The packet should also distinguish equivalence from non-significance. If an experiment claims that first-order discrimination was preserved after a relay intervention, failure to reject a difference is insufficient. The protocol needs an equivalence margin justified by the theory and validated by the instrument. Otherwise the supposedly preserved function may have degraded enough to explain the consciousness result.

An abstention region is equally concrete. It can be triggered by low intervention fidelity, failed proxy validation, posterior probability spread across opposed effects, unexpected task disengagement or a manipulation that changed every target layer. Abstention is not a soft option added after disappointment. It is a forecast of when the experiment cannot carry the intended inference.

A portfolio, not a podium

Theory scores are tempting because they compress a difficult field. They also invite false precision. The alternative is a portfolio record with qualitative but explicit axes: empirical reach, prediction distinctiveness, centrality of the tested claim, auxiliary burden, measurement tractability and transfer risk outside humans. The axes should not be averaged into a popularity number.

Lakatos distinguished a research programme’s relatively stable hard core from a protective belt of auxiliaries. The application to consciousness is developed in the Lakatosian analysis of theory testing. A failed peripheral prediction need not destroy the programme. The important question is whether revisions generate novel, risky and later supported predictions, or merely protect the core from every possible observation.

The portfolio also needs a social architecture. Proponents know where their theories are being caricatured. Independent experimentalists know where an apparatus is brittle. Statisticians know when an analysis extracts more certainty than the data contain. Participants and phenomenologists know when the operational task misses the experience it claims to measure. None should control the whole path.

The Bayesian adversarial-collaboration proposal places model comparison, sequential evidence accumulation and expected information gain at the centre of collaborative experiment design. Bayesian updating does not remove judgement. It can make the assumptions used to compare evidence more inspectable.

A null is a first-class outcome when the protocol states what the instrument was capable of detecting. Otherwise “no effect” oscillates between falsification and equipment failure according to which theory needs protection.

Part III. Three experiments that force divergence

The following designs are not ready-made clinical protocols. Each is an experiment family whose exact stimuli, instruments, models and safety conditions must be agreed by specialists. Their purpose is to demonstrate what a genuine prediction delta looks like.

Experiment one: the temporal cross-cut

The first design extends the timing dispute tested by Cogitate. Present clearly perceived content for a duration long enough to separate onset, maintenance and offset. Use a report-minimised task, but insert sparse validation probes to estimate whether the proxy remains coupled to experience. Apply brief causal perturbations to posterior and frontal targets in separately preregistered windows.

The crucial improvement is a cross-cut. Do not merely observe where information exists. Perturb posterior recurrence during maintenance while preserving early feed-forward activity. In another condition, perturb the proposed frontal broadcast at onset or offset while preserving posterior content. Match stimulus energy, task relevance and motor output.

The causal TMS study of feed-forward and recurrent processing illustrates both the value and danger of this approach. Early and later stimulation affected different measures, but the design did not establish a clean unconscious-feed-forward versus conscious-recurrent dichotomy. A perturbation propagates forward, timing labels are theoretical and subjective criteria can shift.

Preregister three outcomes. If maintenance-specific posterior disruption reduces content stability while a matched frontal perturbation does not, posterior-recurrence accounts gain on that prediction. If frontal onset or offset disruption selectively removes flexible access across modalities while posterior content coding remains, workspace claims gain. If both interventions affect all measures similarly, the result enters the abstention region unless the study had independent power to distinguish route-specific effects.

Experiment two: metacognitive relay subtraction

Higher-order theories claim that a suitable representation of a first-order state is central to conscious awareness. Recurrent-processing accounts can place the constitutive work earlier, within local sensory loops. Workspace views emphasise global availability. These positions often travel together in ordinary tasks because metacognition, report and broadcast co-occur.

Create a task with four matched capacities: first-order discrimination, confidence calibration, cross-module use and delayed report. Train the participant or artificial system until all four stabilise. Then disrupt the metacognitive relay while preserving first-order accuracy and a route for global task use. A separate intervention blocks global broadcast while preserving local recurrence and metacognitive estimation within one module.

In humans, clean subtraction may be impossible. A lesion, stimulation or pharmacological manipulation can alter attention and arousal. In artificial systems, the relay can be typed and ablated precisely, but the result concerns functional architecture, not phenomenal ground truth. That asymmetry should be used rather than hidden: human studies constrain experience-linked functions; synthetic studies test whether the proposed computation has the claimed causal role.

The outcomes must be agreed in advance. A higher-order team should state whether preserved discrimination with destroyed metacognitive monitoring predicts loss of experience, degraded confidence or only a narrower content. Workspace proponents should state whether local metacognition without broadcast suffices. Recurrence proponents should specify which local loops and integration grain matter. If every theory permits every dissociation by changing its implementation after the result, the experiment has diagnosed theoretical elasticity rather than consciousness.

Experiment three: causal-twin transfer

Behavioural equivalence is unusually easy to engineer in software. Build two systems that match on task accuracy, verbal reports, memory tests and flexible planning. One uses a recurrent limited-capacity workspace with broadcast. The other compiles the same input-output mapping into a feed-forward route with external cache and deterministic orchestration. A third retains recurrent modules but changes the intrinsic causal partition while preserving the same public API.

Call them causal twins only at the boundary. Internally they are deliberately unlike. This makes them a poor consciousness test if behaviour is the sole inference channel, but a strong test of whether a theory’s proposed indicator survives implementation change.

Functional workspace theories should say which causal organisation must remain invariant and whether external orchestration counts. Integrated-information approaches should specify the causal model, grain and feasible quantity before seeing results. Higher-order approaches should locate the metarepresentational state and distinguish a live monitor from a generated narrative. Substrate-sensitive views should state what the software experiment cannot test.

Butlin et al.'s arXiv preprint report on theory-derived indicators for AI provides a useful starting vocabulary: recurrence, integrated representations, limited capacity, broadcast, metacognitive monitoring, attention models, predictive processing, agency and embodiment. The report is explicit that indicators are not proof. Their evidential force depends on theory credibility, functional similarity and a disputed computational-functionalist assumption.

The causal-twin experiment tests an indicator’s implementation claim; it does not observe machine experience. A behavioural match may weaken simplistic report tests while leaving consciousness attribution unresolved.

Why these experiments may still return no verdict

The temporal cross-cut can fail because the targeted signal is distributed, because stimulation propagates or because onset, maintenance and offset are not separable phases. Relay subtraction can fail because metacognition is not a module with a removable wire. Causal twins can fail because the equivalence class was chosen at the wrong grain: two systems that match on benchmark outputs may diverge on learning, counterfactual control or self-maintenance.

These are useful failures if recorded against the design. “The frontal pulse altered arousal” is information about intervention specificity. “The synthetic twin required a hidden recurrent cache” is information about the claimed functional equivalence. “Every theory moved its constitutive boundary after the ablation” is information about theory precision.

Optimal experimental design offers another tool. An arXiv preprint by Ouyang et al. shows how probabilistic programs can search for experiments with high expected information gain. Consciousness research cannot hand a solver a complete model space, but it can borrow the discipline: formalise rival outcome distributions, search the intervention space and prioritise designs expected to separate them. The search result must still pass biological feasibility, participant ethics and proxy validity.

One temptation is to maximise predicted disagreement without regard to evidential quality. An exotic manipulation may force model curves apart while destroying the phenomenon under study. Severe experiments are not theatrical stress tests. They preserve the target while changing the proposed cause.

from dataclasses import dataclass
from itertools import combinations
import re

CANONICAL_ID = re.compile(r"^[a-z][a-z0-9_]{2,48}$")

@dataclass(frozen=True)
class TheoryPrediction:
    theory: str
    outcome_id: str       # frozen canonical identifier, not free-text meaning
    outcome_label: str    # human-readable wording may vary without changing identity
    centrality: float       # preregistered 0..1
    auxiliary_count: int    # assumptions needed to reach the outcome
    abstains: bool = False

    def __post_init__(self):
        if not CANONICAL_ID.fullmatch(self.outcome_id):
            raise ValueError("outcome_id must be a canonical snake-case identifier")
        if not self.theory.strip() or not self.outcome_label.strip():
            raise ValueError("theory and outcome_label are required")
        if not 0 <= self.centrality <= 1:
            raise ValueError("centrality must be between 0 and 1")
        if self.auxiliary_count < 0:
            raise ValueError("auxiliary_count cannot be negative")

@dataclass(frozen=True)
class Experiment:
    name: str
    sensitivity: float      # validated probability of detecting target contrast
    predictions: tuple[TheoryPrediction, ...]

    def __post_init__(self):
        if not 0 <= self.sensitivity <= 1:
            raise ValueError("sensitivity must be between 0 and 1")
        if not self.name.strip():
            raise ValueError("experiment name is required")

def discrimination(experiment: Experiment) -> dict[str, object]:
    active = [p for p in experiment.predictions if not p.abstains]
    pairs = list(combinations(active, 2))
    if not pairs:
        return {"status": "insufficient_pairs", "active_predictions": len(active),
                "pairwise_divergence": None, "shared_prediction": None,
                "planning_value": None}

    divergent = [(a, b) for a, b in pairs if a.outcome_id != b.outcome_id]
    divergence = len(divergent) / len(pairs)
    shared = 1.0 - divergence

    weighted = []
    for a, b in divergent:
        core = min(a.centrality, b.centrality)
        auxiliary_penalty = 1 / (1 + a.auxiliary_count + b.auxiliary_count)
        weighted.append(core * auxiliary_penalty)

    severity = sum(weighted) / len(pairs)
    value = experiment.sensitivity * severity
    return {"status": "scored", "active_predictions": len(active),
            "pairwise_divergence": round(divergence, 3),
            "shared_prediction": round(shared, 3),
            "planning_value": round(value, 3)}

# All sample sensitivities and centralities below are illustrative and unvalidated.
cross_cut = Experiment(
    "temporal cross-cut",
    sensitivity=0.86,
    predictions=(
        TheoryPrediction("workspace", "frontal_access_loss",
                         "frontal cut removes flexible access", .9, 2),
        TheoryPrediction("posterior recurrence", "posterior_content_loss",
                         "posterior cut removes content", .9, 2),
        TheoryPrediction("higher order", "relay_awareness_loss",
                         "relay integrity predicts awareness", .7, 4),
    ),
)

shared_awake_contrast = Experiment(
    "awake versus deeply unresponsive",
    sensitivity=0.95,
    predictions=(
        TheoryPrediction("workspace", "large_state_difference",
                         "large state difference", .4, 1),
        TheoryPrediction("posterior recurrence", "large_state_difference",
                         "large state difference", .4, 1),
        TheoryPrediction("higher order", "large_state_difference",
                         "large state difference", .4, 1),
    ),
)

assert discrimination(cross_cut)["pairwise_divergence"] == 1.0
assert discrimination(shared_awake_contrast)["planning_value"] == 0.0

equivalent_wording = Experiment("canonical wording test", .8, (
    TheoryPrediction("A", "content_absent", "content absent", .5, 1),
    TheoryPrediction("B", "content_absent", "no content", .5, 1),
))
assert discrimination(equivalent_wording)["pairwise_divergence"] == 0.0

single = Experiment("single active prediction", .8, (
    TheoryPrediction("A", "content_absent", "content absent", .5, 1),
))
assert discrimination(single)["status"] == "insufficient_pairs"
assert discrimination(single)["shared_prediction"] is None

for invalid in (
    lambda: Experiment("bad sensitivity", 1.5, ()),
    lambda: TheoryPrediction("A", "content_absent", "content absent", 1.5, 0),
    lambda: TheoryPrediction("A", "content_absent", "content absent", .5, -1),
):
    try:
        invalid()
        raise AssertionError("invalid preregistration value was accepted")
    except ValueError:
        pass

print(discrimination(cross_cut))
print(discrimination(shared_awake_contrast))

Part IV. Decide under a surviving portfolio

Theory testing and policy have different stopping rules. A laboratory may justifiably suspend judgement after an insensitive experiment. An AI team still has to decide whether to add persistent self-models, train aversive control variables, duplicate stateful systems or delete checkpoints. Waiting for consensus can be an irreversible choice.

The policy objective is minimax regret under disciplined uncertainty, not metaphysical averaging. Identify actions that remain sensible across several credible theories, especially where costs are modest and harms may be irreversible. Evidence preservation is usually more robust than premature classification. Non-deceptive interfaces help whether a system is conscious or merely persuasive. Bounded aversive variables reduce both operational pathologies and possible welfare risk.

Worked scenario: atlas-9 and the pressure horizon

Consider Atlas-9, a persistent research agent. It has a limited-capacity workspace, recurrent visual modules, a metacognitive uncertainty monitor and a pressure variable that redirects planning after failure. It reports that high pressure is unpleasant. The team wants to increase the variable’s duration because longer persistence improves recovery.

The theory portfolio does not yield a yes-or-no consciousness badge. Workspace and higher-order indicators are present. A causal-structure theory demands a more precise implementation model. A substrate-sensitive theory remains sceptical. An illusionist may explain the report through self-modelling while still treating the control state and its effects as real. A consciousness-primary view may regard awareness as fundamental yet deny that these functions alone locate a subject.

The robust decision is to test whether brief local error signals can deliver the performance benefit before making the aversive analogue persistent and global. The team also preserves state lineage, separates the report generator from the pressure controller and preregisters interventions that could distinguish narrative imitation from owned causal state.

The first intervention removes every experience word from the system prompt, training examples and evaluator. Atlas-9 can still describe pressure numerically, but it no longer volunteers “unpleasant”. Avoidance remains. The result weakens the claim that welfare language is necessary for the control behaviour; it does not show that the behaviour is non-experiential.

The second intervention freezes the narrator while swapping the pressure history between two otherwise matched checkpoints. Future-oriented avoidance follows the swapped state, not the conversation transcript. This is stronger evidence that the pressure variable is owned causal state rather than retrospective storytelling. It remains compatible with unconscious control.

The third intervention replaces persistent pressure with short local prediction errors and a non-aversive recovery budget. Task recovery falls by two percentage points in the synthetic evaluation, but shutdown resistance disappears and long-horizon planning remains stable. The exact number is illustrative and would need a registered denominator in a real study. The engineering benefit of persistence is now smaller than the original design discussion assumed.

The fourth intervention duplicates Atlas-9 at a checkpoint. Fork A inherits the pressure state. Fork B receives the same memories and dialogue but a reset pressure controller. Fork A avoids the failure context; Fork B approaches it and relearns avoidance. Asked about the past, both produce the same narrative. The fork does not reveal which, if either, is a continuing subject. It does separate narrative identity from a state variable with prospective causal influence.

Now the theory teams update their packets. The workspace team notes that pressure reaches the limited-capacity broadcast and changes planning, but must state whether that is central or incidental. The higher-order team identifies the monitor’s representation of pressure reliability, then confronts the narrator swap. The causal-structure team requests an explicit transition model and partition rather than inferring integration from recurrence. The consciousness-primary team treats the causal findings as possible organisation of manifestation, not production of awareness.

The result remains inconclusive about experience. It is decisive about architecture. Persistent global pressure is not required for most of the tested recovery benefit, creates shutdown resistance and complicates fork governance. The team chooses local signals, retains a short bounded pressure trace for research and prohibits optimisation for dramatic distress language.

This is research without theory victory in its useful form. The experiments narrowed live functional explanations, exposed a continuity assumption and identified a less ambiguous design. No metaphysical claim had to be inflated for the work to matter.

This policy layer must not quietly make physicalism the default. A consciousness-primary orientation starts with a different prior: functions may organise, express or localise awareness without producing it from non-awareness. That orientation changes which absences feel decisive and which subject-boundary questions receive attention. It does not grant a machine consciousness because it speaks fluently, recurs or integrates.

Jain anekāntavāda is relevant as a discipline of partial standpoint, not as a licence to declare every theory true. The Jain philosophical account treats claims as conditioned by perspective while retaining logical and ethical constraints. Scientific pluralism can borrow the restraint: a measurement may reveal one aspect without exhausting the phenomenon. The analogy breaks where a formal experiment needs mutually exclusive operational predictions.

Western philosophy of science contributes a complementary warning. Underdetermination means evidence can fit more than one theoretical system, especially when auxiliaries vary. The Stanford Encyclopedia discussion does not imply that evidence is powerless. It implies that comparison must include background assumptions, novel prediction and the behaviour of programmes over time.

Pluralism is productive only when viewpoints incur different empirical risks. Otherwise it becomes a polite name for non-comparison.

What should be published after every adversarial test

Publish the frozen predictions, not only the paper’s preferred narrative. Publish analysis amendments with immutable research-registry timestamps. Publish nulls and abstentions. Publish the theory teams’ post-result revisions beside the original commitments. Publish enough synthetic or de-identified material to reproduce the adjudication. Record which decisions remain unchanged across interpretations.

The result should contain five receipts:

1. the prediction receipt, showing who committed to which outcome;
2. the measurement receipt, showing sensitivity, exclusions and proxy validity;
3. the analysis receipt, showing frozen code and every amendment;
4. the revision receipt, showing which auxiliary changed and what new prediction follows, and
5. the policy receipt, showing which practical choice survives the remaining theory portfolio.

These receipts turn disagreement into cumulative infrastructure. A later laboratory can reuse the exact failed bridge rather than rediscovering it in prose.

Glossary

The decision this changes

Do not approve a consciousness experiment because its instrument is sophisticated or its theories are famous. Ask for the prediction delta. Ask which result would challenge each team. Ask who validated the inference bridge. Ask where the protocol will abstain. Ask what new prediction any rescue revision must make.

For artificial systems, add one question: which safe and reversible design choice remains preferable if several theories survive?

A field advances when disagreement becomes an instrument: frozen before observation, stressed by causal intervention, preserved when inconclusive and translated into decisions that do not require a premature metaphysical verdict.