The trial the average threw away

A repeated stimulus does not create a repeated experiential context. Treating every within-person difference as noise can erase the variable that explains the measurement.

Imagine two trials with the same screen, timing and response. In the first, the participant anticipates the target and holds a stable visual expectancy. In the second, a half-formed memory occupies attention until the target interrupts it. If both produce a correct button press, a behavioural summary can make them look equivalent. A brain average can then blend two different preparation trajectories into a waveform that belongs to neither.

The reader’s likely intuition is sensible: keep the physical input constant, collect many repetitions and average away accidental variation. The hidden assumption is that the uncontrolled variation is irrelevant to the causal process of interest. That assumption is safe only when the variable being averaged does not systematically alter perception, action or physiology. When ongoing attention, expectancy or bodily readiness changes the response, averaging can destroy structure rather than reveal it.

Report is not one thing

Ask someone why they selected a face, and the answer may be a polished reason assembled after the choice. Ask what changed in the visual field just before the finger moved, and the answer may still be incomplete, but it targets a different object. Ask for a confidence rating, and a third object appears. These are all called self-report, yet they have different temporal distances, prompts and inferential demands.

Nisbett and Wilson’s classic critique showed that people can explain behaviour using plausible theories rather than direct access to the process that produced it.[7] Choice-blindness experiments made the problem vivid: participants sometimes accepted an outcome they had not chosen and then justified it.[8] The lesson is not that every description of experience is fictional. It is that a causal explanation of one’s behaviour and a description of what was experienced are not interchangeable reports.

This distinction prevents two symmetrical errors. The credulous error treats sincerity and detail as accuracy. The dismissive error treats known failure in causal explanation as proof that event-near sensory, affective or temporal distinctions cannot carry information. Neither conclusion follows. The correct unit of assessment is a configured report process attached to a declared claim.

Different errors require different instruments

“Introspection is unreliable” compresses several failures into one verdict. A participant can miss a feature that was present, remember a feature that was not, use one word for two structures, answer the interviewer’s expectation, or accurately describe a feeling while misidentifying its cause. These errors are not interchangeable. Omission calls for better sampling or training. Reconstruction calls for shorter delay. Vocabulary drift calls for source-preserving coding. Demand calls for prompt intervention and blinding. Causal misattribution calls for an external manipulation of the proposed cause.

The temporal grain also matters. A rating collected after a ten-minute block may be useful for a block-level mood claim and useless for a 200-millisecond transition. Conversely, an immediate interruption can capture a brief attentional state while altering the ongoing task. There is no report schedule that dominates every question. The instrument must match the timescale of the proposed mechanism and make its disturbance visible.

Descriptive Experience Sampling addresses one part of this problem by using random beeps to mark moments in everyday life, followed by interviews that focus on what was occurring at the sampled instant.[10] Micro-phenomenology uses a different route, often reconstructing the unfolding of a selected brief event in greater depth. Neither method is simply “more subjective” than a rating scale. Each trades coverage, interruption, delay and resolution differently. The scientific question is not which method sounds rigorous, but which error profile is acceptable for the causal claim.

A twelve-trial example

Suppose a participant completes twelve identical detection trials. After each trial, an event-near prompt records either stable preparation, category A, or interrupted preparation, category B. A pre-stimulus measure takes positive values in A and negative values in B. The pooled mean is zero. Nothing seems to happen. Partition the trials using labels fixed before the physiology is opened, and the contrast appears.

In a real study, this toy pattern would not be enough. Six trials per group are unstable. The measure could track eye movement or arousal rather than the reported structure. The labels could leak condition information. Yet the example exposes the causal opportunity. First-person data need not function as an unquestioned statement about private truth. They can function as trial-indexing information whose predictive value is assessed like any other candidate variable.

flowchart LR
  A["The trial the average threw away"] --> B["When a report becomes a constraint"]
  B --> C["Elicitation is an instrument,<br/>not a confession"]
  C --> D["The constraint lab"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#087A98,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

When a report becomes a constraint

A report constrains an experiment when removing, replacing or scrambling it changes a pre-specified prediction. This is stronger than a compelling narrative and weaker than certainty about experience.

The constraint-bearing partition

Consider a trial indexed by i. The physical task is X_i. A disciplined report procedure assigns a partition P_i, such as stable preparation or interrupted preparation. The third-person outcome is Y_i, perhaps reaction time, pupil response or a neural feature. Known contaminants are collected in C_i, including eye movement, motor timing and condition order.

The smallest useful test is not whether β differs from zero in the same trials used to invent the categories. It is whether a category scheme learned during a training phase predicts held-out trials, transfers to a new session, or identifies an intervention response. A report becomes more constraining when it survives a scrambled-label control, a simpler arousal explanation and independent coding.

This distinction creates three grades. At the first grade, a report is intelligible and sincere. At the second, its structure can be elicited and coded with traceable procedures. At the third, that structure generates a prediction or intervention that could fail. Only the third grade can carry the causal thesis of a neurophenomenological experiment. The earlier grades remain prerequisites, not conclusions.

Choice blindness shows both the danger and the opportunity

In the original choice-blindness task, a participant selected one of two faces. Through a card trick, the experimenter sometimes presented the unchosen face and asked for a justification. Many substitutions went undetected, and participants could provide reasons for the option they had not selected.[8] The result attacks a strong assumption: verbal confidence and causal fluency do not guarantee access to the decision process.

A later study reproduced the manipulation but used an elicitation interview for one manipulated choice. The authors reported much higher substitution detection in the elicitation condition than under the ordinary “why did you choose it?” prompt.[9] This finding is provocative rather than decisive. The interviewer, procedure and extra attention all changed together. Detection is an external criterion, which is valuable, but it does not show that every recovered detail accurately belonged to the original choice. Larger independent replications and dismantling studies would be needed to isolate which part of the procedure mattered.

The pair of studies nevertheless demonstrates the right argumentative form. One protocol generates confident confabulation. Another changes the report process and produces a measurable difference in detecting a covert intervention. The useful conclusion is not “introspection fails” or “micro-phenomenology succeeds”. It is that access claims should be evaluated through manipulations whose truth is known independently of the report. The hidden card supplies such a ground truth. In many consciousness studies no equivalent ground truth exists, so the claims must be narrower.

Four replacement tests

Because the private event cannot simply be placed beside the EEG trace, the method needs replacement tests. Each one changes one causal property while preserving the visible output as far as possible.

These cases share a logic. A useful report should be replaceable in ways that expose what it contributes. Scramble trial alignment, extend the delay, unblind the coder or coarsen the ruler. The resulting changes identify the report’s intervention fingerprint. Without such replacements, a report can decorate an experiment without constraining it.

The published pilot and its proper weight

The 2002 visual study remains instructive because it used first-person categories to guide a separate analysis of brain dynamics. Participants underwent training so that recurrent preparation states could be stabilised and named. During the task, a seven-second preparation period preceded presentation of the stereoscopic stimulus. Participants pressed when the three-dimensional form emerged and then briefly reported the cognitive context of that trial. The researchers used categories such as steady readiness, fragmented readiness and unreadiness, with subject-specific details rather than assuming one universal taxonomy.[3][4]

Within this pilot, the categories were associated with distinct pre-stimulus synchrony and amplitude patterns and with different response times. The result did not establish a general neural signature of readiness. There were only four participants, the categories were intensively trained, and the analysis involved complex choices. Its enduring contribution is methodological: instead of treating within-person state variation as nuisance, the study let disciplined report categories define contrasts that could be examined in the neural data.

Later micro-phenomenological work developed more explicit procedures for eliciting and analysing singular lived episodes. A concrete event is re-evoked, its temporal unfolding is examined, and descriptive units are separated from general beliefs or explanations. Generic structures are built iteratively across cases rather than assumed at the outset.[5][6] This can yield unusually fine descriptions. It does not by itself solve reliability. The categories still need traceability, independent coding and a result that could contradict them.

Elicitation is an instrument, not a confession

The interviewer’s task is not to reward a profound story. It is to recover the temporal and sensorial organisation of one event while minimising the routes by which explanation replaces description.

Begin with one recoverable episode

“How do you usually notice a thought?” invites theory. “Return to the moment just before you pressed the button on trial 42” identifies an event. The distinction is not cosmetic. General questions recruit semantic knowledge and self-image. A singular event can be located in time, revisited through cues and compared with its behavioural trace.

Micro-phenomenological interviews use evocation to help the participant reconnect with a concrete episode. The interviewer asks about sequence, sensory qualities, bodily orientation and changes in attention. Questions target how the event unfolded rather than why it happened. A few seconds of experience can take far longer to describe because the aim is not conversational efficiency. It is resolution.[5][6]

Resolution carries a risk. An hour of questioning can also construct a stable account. Repetition can train attention towards features that were not previously salient. The method therefore needs external constraints and a record of every transformation from utterance to category. Training may improve access, but it can also change the phenomenon being accessed. The right design treats that change as a testable intervention, not an embarrassment to hide.

Reliability is not a single coefficient

Three questions are often folded into one. Can the participant produce a similar description when an experience of the same type recurs? Can independent analysts transform the transcript in a comparable way? Does the resulting category track something outside the report process? These are, respectively, experiential repeatability, coding reproducibility and external validity. A high score on one does not guarantee the others.

Repeated wording can reflect a learned script rather than repeated experience. Coder agreement can be high because the codebook is simple, even when the categories have little causal relevance. A category can predict behaviour while the participant’s explanation of it is wrong. The protocol therefore records each layer separately. Confidence is not merged with description. Analyst agreement is not called accuracy. Prediction is not called identity.

Internal criteria such as a felt sense of certainty may help a participant discriminate strong from weak recollection, and exploratory work has investigated this possibility.[14] Yet certainty itself can be mistaken. Its best role is as another measured feature whose calibration can be tested, not as a final court of appeal. Disciplined first-person science replaces global trust with a set of local, inspectable warrants.

The Elicitation-to-Constraint Protocol

The following protocol is a practitioner synthesis, not a claim that one standard already governs the field. Its purpose is to expose the decisions that prose often hides. The sequence preserves the exploratory strength of phenomenology while adding a confirmatory gate before first-person categories are allowed to support a third-person claim.

The central asymmetry lies between stages five and six. Discovery is allowed to be rich, iterative and participant-specific. Confirmation must become narrower. The codebook can no longer absorb the result it was meant to predict. Registered Reports provide one institutional form for this separation because the question and analysis can receive in-principle acceptance before outcomes are known.[13]

Worked configuration: the faint-tone study

Consider a study of how a faint tone becomes present in noise. Every trial begins with six seconds of listening, followed by a possible tone near threshold. The participant responds when a tone is experienced. The same button press can arise through at least three routes: a gradual sensory emergence, a sudden perceptual onset, or a reasoned guess from contextual cues.

During an exploratory session, a trained interviewer elicits a subset of event-near trials. The participant’s language is coded for temporal onset, bodily orientation, auditory localisation and certainty. The research team derives participant-level structures, then maps only relations that recur across people. Before the confirmatory session, the team freezes three outcome predictions. Gradual emergence should show a longer pre-response accumulation profile; sudden onset should show a sharper transition; reasoned guesses should track condition probabilities more than sensory evidence. These are hypotheses, not known facts.

The confirmatory session uses new trials. Coders are blind to tone presence, response and physiology. Eye movement, button force, respiration and pupil size are recorded as rivals. Shuffled report labels establish a null distribution. A positive result would permit the claim that the report partition carries out-of-sample information about the measured dynamics. It would not show that the participant directly observed a neural accumulation process, nor that one phenomenological category is identical to one neural mechanism.

The hardest boundary: training can transform the target

A skilled contemplative may discriminate features that a novice cannot. That can improve resolution. It can also mean that expert and novice trials instantiate different attentional skills, expectations and learned concepts. Expertise is therefore not a universal upgrade applied to an unchanged mind. It is an intervention on the measurement system and perhaps on the phenomenon.

One study of micro-phenomenological self-inquiry found that giving untrained participants a form was not enough to reproduce the hoped-for access to decision processes; the authors argued that regular interview or training may be required.[11] This is a useful failure. It weakens the idea that a questionnaire can cheaply inherit an interview method’s validity. It also raises a transfer question: does the trained skill reveal an ordinary process, create a new process, or both?

stateDiagram-v2
  [*] --> Claim
  state "Figure 1. Where the two<br/>strands can meet" as Claim
  state "Report is not one thing" as Test
  state "Different errors require<br/>different instruments" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

The constraint lab

The interactive lab makes one hidden assumption visible: a true experiential contrast can disappear through noisy reports, while analytical flexibility can manufacture a contrast when none exists.

The simulation generates synthetic trials with two latent preparation states. The states have a configurable separation in a measured outcome. A report label recovers the state with configurable reliability. The chart compares the pooled mean, the true latent contrast, the contrast recovered by the report partition and a shuffled-label contrast. A post-hoc search option demonstrates how trying many category variants can inflate the apparent result.

Run the lab with true separation set to zero. A properly frozen report partition should hover near zero, as should shuffled labels. Turn on post-hoc search, and one of twenty recodings often looks more impressive simply because the largest result was selected. Next set separation to 1.2 and report reliability to 0.55. The latent contrast remains, but the report recovers little of it. A null report result now has at least two explanations: the experiential distinction may not matter, or the report process may not measure it well enough.

A positive result permits a conditional claim: under this elicitation, coding and task configuration, the report partition carried information about a measured outcome that survived declared rivals. A negative result permits less. It may weaken the proposed bridge, expose inadequate reliability, reveal a mismatched physiological grain or show that the proposed state taxonomy was wrong. The simulation cannot adjudicate which explanation holds. That requires interventions and new measures.

type EvidenceLayer =
  | "description"
  | "judgement"
  | "confidence"
  | "causal_explanation"
  | "tradition_interpretation";

type CodedUnit = {
  source_start: number;
  source_end: number;
  verbatim: string;
  layer: EvidenceLayer;
  code: string;
  coder_id: string;
  coder_blind_to_outcome: boolean;
};

type TrialReport = {
  participant_id: string;
  trial_id: string;
  event_anchor_ms: number;
  elicitation_delay_ms: number;
  prompt_version: string;
  transcript_hash: string;
  coded_units: CodedUnit[];
  participant_category: string | null;
  cross_participant_mapping: string | null;
  codebook_version: string;
  codebook_frozen_before_outcome: boolean;
  confidence_0_to_1: number | null;
  adverse_event_flag: boolean;
};

type ConfirmatoryReceipt = {
  report: TrialReport;
  prediction_id: string;
  outcome_feature_version: string;
  exclusion_rule_version: string;
  rival_generators_tested: string[];
  result: "supports" | "weakens" | "inconclusive";
  prohibited_inferences: string[];
};

Four failure signatures

Different failures leave different traces. Treating all of them as “subjectivity” prevents diagnosis.

The first failure is semantic capture. The interviewer names the expected state, and participants learn to answer in that vocabulary. The second is memory smoothing. Temporal gaps turn ambiguous fragments into coherent episodes. The third is analysis backflow. Categories are revised after outcome patterns are visible. The fourth is common-cause confounding. Arousal, movement or task expectancy changes both the report and the recorded signal.

A fifth failure deserves separate emphasis: cross-person flattening. The same word may mark different structures. “Openness” can mean expanded visual periphery for one participant, reduced effort for another and a tradition-specific interpretation for a third. A group code that ignores these differences can create artificial agreement. Begin with within-person structure, then map relations across people with explicit non-equivalence.

The strongest rival is often a strategy, not a state

Suppose “steady readiness” predicts faster responses and stronger pre-stimulus synchrony. The tempting explanation is that the reported experiential structure has a corresponding neural dynamic. A simpler rival may be that some trials use an anticipatory response strategy. The strategy changes eye position, muscle tension, expectation, decision threshold and the words used afterwards. Report and physiology then agree because both inherit a common task policy.

This rival is strong because it does not dismiss the report. The participant may accurately describe what the strategy feels like. What fails is the inference from felt structure to the proposed neural mechanism. A discriminating study could manipulate strategy instructions while asking whether the experiential category persists. It could match reaction time across categories, include electromyography and eye tracking, or intervene on expectancy while holding sensory input constant. If the report-outcome relation disappears when strategy is controlled, the bridge should be redrawn around strategy.

Another rival is measurement-induced state change. Immediate probes can redirect attention; extended interviews can teach new discriminations; repeated category use can stabilise a state. These effects are not always defects. A training intervention may be exactly what the study aims to understand. The mistake is to describe the post-training report as an untouched window onto the pre-training process. A neurophenomenological design must model the observer’s skill as part of the causal system.

What mutual constraint can and cannot mean

Varela’s neurophenomenological programme proposed reciprocal constraints between disciplined experience and cognitive science.[1] The phrase is powerful but easy to inflate. A neural result can challenge a report taxonomy by showing that two supposedly distinct states behave identically under every relevant intervention. A report can challenge an experimental contrast by revealing that trials grouped together contain recurrently different experiential trajectories. Neither direction turns one description into the other.

The metaphysical boundary is therefore strict. A successful bridge can be interpreted within physicalism, biological naturalism, functionalism, emergentism, idealism, neutral monism, panpsychism, dual-aspect theory or process philosophy. These positions disagree about what the relation ultimately is. The experiment shows that two disciplined descriptions covary or respond together under intervention. It does not show that experience is reducible to neural dynamics, that awareness is fundamental, or that a contemplative account has been mechanistically confirmed.

A compact decision table