The data do not carry a verdict

Imagine two navigation teams studying the same storm track. Team A says the storm followed a warm-water current. Team B says a pressure ridge deflected it. Both can draw a plausible path through the observed points. Looking only at fit, the argument stalls. The useful question is not “Can this story accommodate the track?” It is “Before seeing the track, how much probability did each story place near it?”

That change sounds small, but it alters the object called evidence. A hypothesis that could have explained almost any outcome risks little. A hypothesis that sharply expected one region and was right risks more. Evidence therefore belongs to a relation among an observation, a predictive model and at least one rival. It does not exist as a free-standing glow around a favoured claim.

The odds form reveals the mechanism

Bayes’ rule is often introduced as a formula for reversing a conditional probability. For model comparison, its most revealing form is an update of odds:

Here, M₁ and M₀ are rival models. The symbol y is the observation. Prior odds state their relative plausibility before this observation. The Bayes factor is the ratio of their predictive probabilities for the observation. Posterior odds are what remain after the transfer. This interpretation is standard in the Bayes-factor literature [Kass & Raftery, 1995].

Suppose the prior odds are one-to-one. If M₁ made the data ten times as probable as M₀, the posterior odds become ten-to-one. If the prior odds were one-to-nine, the same data produce posterior odds close to one-to-one. The observation did the same comparative work in both cases. It began from a different allocation.

Likelihood is not posterior probability

Another hidden switch occurs when people move from “the data would be likely if the story were true” to “the story is likely because the data occurred”. The first quantity is p(data | story). The second is P(story | data). They are not interchangeable. The second requires all relevant rivals and their prior weights.

This matters in human and machine intelligence research. A language model may produce a fluent self-report that is likely under a model with access to internal state. The same report may also be likely under a mimic model trained on discourse about introspection. Fluency raises little evidential pressure until a rival predicts something different under an intervention. The Bayesian lesson is not that one must assign a number to consciousness. It is that evidence must be designed around a prediction delta, not around how naturally a preferred story can narrate the result.

A model is a generator, not a label

Calling one explanation “reasoning” and another “memorisation” does not yet create statistical models. Each label must be expanded into a generator that assigns probabilities to observable outcomes. What would the reasoning account predict after a premise is replaced, an intermediate result is hidden or the problem is restated in an unfamiliar notation? What would the memorisation account predict under the same changes? Until those answers exist, the labels organise conversation but cannot carry a likelihood.

The generator also includes the observation channel. A latent fault does not directly produce the number on a dashboard; it changes vibration, the sensor transforms vibration into a signal, software thresholds the signal and an interface displays an alarm. A cognitive mechanism does not directly produce an experimental score; it interacts with instructions, memory limits, response format and scoring. The likelihood belongs to the complete path from mechanism to measurement. Leaving the channel implicit lets analysts attribute sensor, interface or scoring behaviour to the mechanism of interest.

This gives a practical test for a proposed rival. Ask whether it can simulate the data collection process, at least approximately. If it can only explain the observation after the fact, it is a narrative rival rather than a predictive rival. Bayesian model comparison needs the latter. The simulation may be simple, but it must state what could have happened and how often.

flowchart TD
  A["The coin that became less<br/>convincing when the<br/>alternative became more<br/>flexible"] --> B["The data do not carry a verdict"]
  B --> C["Flexibility pays through<br/>prediction"]
  C --> D["The frame can win while<br/>reality loses"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#5B5CF6,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Flexibility pays through prediction

The fixed 90 per cent coin model contains no adjustable parameter. The broad bias model contains a continuum of possible head probabilities. How should a single model make a prediction when it contains many parameter settings?

It averages. Before the tosses, each parameter value predicts possible sequences. The prior distribution says how much of the model’s predictive budget belongs to each value. The marginal likelihood, also called the model evidence, adds those weighted predictions:

The variable θ is a parameter inside model M. The likelihood describes what θ predicts. The prior distributes predictive mass over possible θ values before y is observed.

This integral is the mechanism behind Bayesian complexity control. A flexible model may achieve a high likelihood at one parameter value, but the comparison asks how well the model predicted on average across the parameter values it permitted. Regions that predicted different outcomes consume probability mass. The model wins only if its extra flexibility concentrated enough mass near the observation to repay that cost.

Occam’s razor is a probability budget

The phrase “Bayesian Occam’s razor” can mislead. It may sound as if the method rewards simple-looking equations or punishes parameter count by decree. The actual mechanism is predictive concentration. A parameter matters only through how it changes the distribution of possible observations. Two extra parameters that barely alter predictions may cost little. One parameter with a vast prior range may cost a great deal.

A local approximation makes the intuition visible. Imagine that the posterior occupies a narrow region of parameter space where the data fit well. The marginal likelihood is roughly the peak likelihood multiplied by the fraction of prior volume that remains plausible. This fraction is sometimes called an Occam factor. The approximation assumes a regular, well-identified model with a single dominant posterior region. Singular models, weak identification and multimodality can break the simple volume picture.

This is why diffuse priors are not automatically “uninformative” for model selection. A wider prior spreads probability over more possible datasets and can reduce marginal likelihood sharply. Improper priors, whose total mass cannot be normalised, do not define a marginal likelihood without additional construction. Prior sensitivity in Bayes factors is a documented practical issue, not a reason to conceal priors [Llorente et al., 2023].

Why better fit can produce weaker evidence

Maximum likelihood asks for the best parameter setting inside each model. Marginal likelihood asks for the model’s average pre-data prediction. These objectives can move in opposite directions. Add a parameter and the maximum likelihood cannot become worse because the old fit remains available. The marginal likelihood can fall because the new parameter opens predictive regions that the data did not occupy.

This distinction matters when a model is tuned repeatedly. If an analyst searches many priors, architectures or feature sets and reports only the one with the highest marginal likelihood, the search itself has created a larger effective model. The advertised comparison no longer describes the probability budget actually spent. One remedy is to include the search choices in the model hierarchy. Another is to reserve data for evaluating the completed process. The right remedy depends on whether the target is scientific comparison or future prediction.

A good report therefore shows both local fit and integrated evidence. The first reveals whether a model contains a parameter setting that matches the data. The second reveals whether that match was a concentrated prediction or a lucky point inside a broad possibility space. Neither number should be asked to do the other’s job.

Evidence accumulates through conditional prediction

For a sequence of observations, evidence can be written as a running contest. At each step, each model predicts the next datum conditional on what has already occurred. The log Bayes factors add:

This is the prequential view: judge a model through the sequence of probability forecasts it issues [Dawid, 1984]. A model may lose early, learn from data through its posterior, then recover. The trajectory reveals whether the final score came from steady predictive superiority or one decisive observation.

The frame can win while reality loses

Bayesian model comparison normalises probability across the models supplied. If two candidates are both poor, one still receives the larger posterior probability. This creates a decisive boundary: comparison answers which candidate predicted better within the frame. It does not by itself answer whether the frame can generate the important features of reality.

Suppose two maps omit an island. One has a slightly better coastline elsewhere. A posterior probability of 0.93 for that map does not create the missing island. Relative victory and absolute adequacy are different inferential operations. Model comparison redistributes belief; model checking asks whether the shared assumptions should survive.

Comparison, checking and prediction answer different questions

Three tools are often collapsed into “model evaluation”, but they create different evidence.

Leave-one-out cross-validation estimates out-of-sample predictive accuracy using each observation as held-out data; Pareto-smoothed importance sampling makes the calculation practical for many fitted Bayesian models and supplies diagnostics for unstable approximations [Vehtari, Gelman & Gabry, 2017] [Vehtari et al., 2024]. It is not a substitute for a Bayes factor when the scientific question is a prior comparison of sharply specified hypotheses. It answers a different question.

The difference becomes important in an M-open setting, where none of the candidates is expected to be literally true. Predictive stacking chooses weights that optimise held-out predictive performance rather than interpreting posterior model probability as the chance that one candidate is the data-generating truth [Yao et al., 2018]. Recent work in machine learning also shows that marginal likelihood can be misaligned with generalisation when it is repurposed for hyperparameter or architecture selection after repeated data-dependent choices [Lotfi et al., 2022].

The strongest boundaries

stateDiagram-v2
  [*] --> Claim
  state "The odds form reveals the<br/>mechanism" as Claim
  state "A rare fault alarm" as Test
  state "Change the rival, keep the data" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Build a rival-first evidence contract

The practical mistake usually occurs before calculation. A team starts with a favoured explanation, collects observations that sound compatible with it, then asks for a posterior. The calculation becomes a polished endpoint for a one-sided design.

A better process starts from the decision and works backwards to a discriminating observation. The proposed rival-first evidence contract is a compact specification for doing that. It is practitioner synthesis, not a published statistical theorem. Its purpose is to make hidden modelling choices reviewable before a consequential update.

Write rivals that can lose differently

A strong rival is more than the negation of the preferred model. “The system does not use evidence” covers many mechanisms with different predictions: memorised answers, prompt cues, a hidden database, majority-class guessing or a tool result copied after the fact. Combining them into one vague alternative makes the denominator impossible to interpret.

Start with the intervention that matters to the decision. Then work backwards. If the claim is causal use of retrieved evidence, change the evidence while preserving wording. If the claim is calibrated uncertainty, hold accuracy constant while changing base rates. If the claim is a stable internal representation, preserve output while perturbing the proposed representation. Each intervention should create a table of divergent predictions before data collection.

Do not require every rival to be equally detailed. Require each to be detailed enough to assign a probability to the decisive observation. When probabilities are hard to defend, use ranges and perform robust sensitivity analysis. The purpose is not false precision. It is to expose where the conclusion depends on a judgement that would otherwise remain verbal.

The seven fields

1. Decision. State what action can change, at what posterior or predictive threshold, and with which loss asymmetry. This prevents a probability estimate from being mistaken for a decision rule.

2. Rival generators. Describe at least two mechanisms capable of producing the observation. Include the strongest plausible mimic, not a straw alternative. A useful rival is generative: it says what would happen under interventions, not only how it labels existing data.

3. Prior predictive commitments. Specify model priors, parameter priors and the observations each model expects before seeing the test data. Simulate from them. Record which choices are substantive and which are convenience defaults.

4. Observation bridge. State how a latent mechanism becomes a measured datum. Include sensor error, annotation, prompt framing, missingness and selection. In possible-mind research, a report is an output channel, not a transparent window into experience.

5. Evidence units and dependence. Map shared sources, repeated transformations and conditional dependencies. Decide what can be multiplied and what must be modelled jointly.

6. Update and sensitivity. Report prior odds, Bayes factor and posterior odds separately. Vary plausible priors, likelihood noise and rival definitions. A conclusion that flips under modest changes should be carried forward as fragile.

7. Escape test. Name an observation that could reject the entire candidate frame, trigger a new rival or force abstention. This field stops posterior normalisation from masquerading as adequacy.

Executable lab: a browser Bayesian notebook

The lab below compares a fixed-probability model M₀ with a composite Bernoulli model M₁. Under M₀, the success probability is fixed at p₀. Under M₁, it has a Beta(α, β) prior. Enter observed successes and trials, then inspect the Bayes factor, posterior model probability and sensitivity to the prior.

What it tests. The notebook exposes how evidence changes when the observation, model prior or parameter prior changes. What a positive result permits. A Bayes factor above one permits a relative update toward M₁ under the entered models. What a negative result means. A value below one transfers odds toward M₀; it does not prove M₀. What it cannot establish. The notebook cannot detect omitted rivals, dependent trials, a wrong observation model or data-dependent prior selection.

from math import exp, lgamma, log


def log_beta(a: float, b: float) -> float:
    """Log of the beta function for positive a and b."""
    if a <= 0 or b <= 0:
        raise ValueError("Beta prior parameters must be positive")
    return lgamma(a) + lgamma(b) - lgamma(a + b)


def compare_binomial_models(
    successes: int,
    trials: int,
    p0: float = 0.5,
    alpha: float = 1.0,
    beta: float = 1.0,
    prior_m1: float = 0.5,
) -> dict[str, float]:
    """
    M0: fixed Bernoulli probability p0.
    M1: Bernoulli probability p with Beta(alpha, beta) prior.

    Invariants:
    0 <= successes <= trials; 0 < p0 < 1;
    alpha, beta > 0; 0 < prior_m1 < 1.
    """
    if not (0 <= successes <= trials):
        raise ValueError("Require 0 <= successes <= trials")
    if not (0 < p0 < 1 and 0 < prior_m1 < 1):
        raise ValueError("Probabilities must lie strictly between 0 and 1")

    failures = trials - successes
    log_m0 = successes * log(p0) + failures * log(1 - p0)
    log_m1 = (
        log_beta(successes + alpha, failures + beta)
        - log_beta(alpha, beta)
    )
    log_bf10 = log_m1 - log_m0
    bf10 = exp(log_bf10)

    prior_odds = prior_m1 / (1 - prior_m1)
    posterior_odds = prior_odds * bf10
    posterior_m1 = posterior_odds / (1 + posterior_odds)

    return {
        "bf10": bf10,
        "log_bf10": log_bf10,
        "posterior_m1": posterior_m1,
        "posterior_mean_p_m1": (successes + alpha) / (trials + alpha + beta),
    }


if __name__ == "__main__":
    print(compare_binomial_models(successes=9, trials=10))

Choosing the comparison tool

This division is consistent with a broader Bayesian workflow in which inference, checking, revision and comparison form an iterative process rather than a single calculation [Gelman et al., 2020]. Gelman and Shalizi argue that successful Bayesian practice includes model criticism and revision, not only conditional updating inside a fixed model [Gelman & Shalizi, 2013].