One distribution, three stories

Start with the final vector and the analogy looks irresistible. Start with the intervention and three different stories appear.

Temperature changes a comparison after scores exist

Suppose a model has produced three logits, 2, 1 and 0. A softmax converts their relative differences into positive values that sum to one. At temperature 1, the distribution is approximately 0.665, 0.245 and 0.090. Raise the temperature to 2 and it becomes 0.506, 0.307 and 0.186. Lower it to 0.5 and the leading class rises to roughly 0.867. The evidence, features and candidate labels did not change. Only the conversion from scores to a categorical distribution changed.

Two invariances matter. A positive temperature preserves the ordering of the logits, so a standard post-hoc temperature transform does not change the top class. Adding the same constant to every logit also changes nothing because only score differences survive normalisation. The odds between classes i and j are exp((z_i − z_j)/T). Temperature therefore governs the sensitivity of the output to already formed differences, not the truth of those differences.[1]

This single scalar is useful. Higher temperatures expose similarity structure in knowledge distillation, while a fitted temperature can improve the calibration of classifiers on held-out data.[2][3] Yet a useful calibration transform is not a universal uncertainty estimator. Its fitted value belongs to a model, dataset, scoring convention and calibration objective. Under distribution shift, a temperature learned on one distribution can become misleading, and newer architectures show calibration patterns that do not follow one simple historical rule.[4][5]

Precision changes the weight of a probabilistic source

Now replace the score vector with a measurement. A prior model says a crack length is likely to be near zero millimetres, but a camera estimates two millimetres. The question is more than how sharply to display the final answer. It is how much weight to give each source. In a Gaussian model, precision is the inverse variance. A source with lower variance receives more weight because the model represents it as more reliable.

Precision therefore does two jobs in this simple case. It narrows or widens the posterior, and it moves the posterior mean when prior and observation disagree. A softmax temperature applied to a completed posterior cannot generally reproduce that factor-specific reweighting. Global tempering raises both prior and likelihood to the same power. Likelihood precision changes one factor relative to another. They coincide only under restrictive choices, such as a uniform prior or a deliberately matched scaling of the same log factor.

Gain changes a response where it is implemented

Neural gain is broader still. In its cleanest form, one input changes a neuron’s sensitivity to another input without changing the feature to which the neuron is tuned. The modulation may look approximately multiplicative, but it can occur through several cellular, circuit and network mechanisms. It can scale an input before a nonlinearity, scale a response after a nonlinearity, alter thresholds, interact with divisive normalisation, or change recurrent stability.[11][20]

That variety matters. If a linear response g·z is passed directly into the same softmax, gain g is algebraically identical to inverse temperature β. If the gain multiplies a saturated sigmoid, changes correlated noise, or acts on only one population, the equivalence fails. At the organism level, arousal-linked gain may be associated with changes in sensory responsiveness, exploration, network integration and state transitions. Those effects cannot be reduced to a single final categorical sharpness.[15][16][17]

flowchart TD
  A["One distribution, three stories"] --> B["The narrow corridor of equivalence"]
  B --> C["Counterfactuals that break the<br/>spell"]
  C --> D["Audit the parameter before you<br/>trust the analogy"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#D95545,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

The narrow corridor of equivalence

The analogy becomes useful when it is written as a conditional statement. It becomes dangerous when the conditions disappear from view.

Put the same statistic at the same place

Let z be a vector of sufficient statistics for a fixed set of alternatives. A softmax with inverse temperature β produces softmax(βz). A linear neural stage with gain g followed by the same softmax produces softmax(gz). A categorical likelihood whose log evidence is z and whose precision parameter λ multiplies that evidence produces softmax(λz), provided the prior is uniform or held outside the comparison in an exactly matched way. Set β = g = λ and the three outputs are identical.

That is not a metaphor. It is an algebraic identity. It also has a small domain. The identity says nothing about what z represents, how it was obtained, whether gain changes its noise, whether precision belongs to the likelihood or prior, or whether the downstream normaliser is biologically implemented. Shared multiplication is a formal relation; shared causal role requires additional preservation tests.

Each condition has a reason. Change the candidate set and normalisation compares different alternatives. Move the scalar after normalisation and a common multiplicative gain cancels when probabilities are renormalised. Enter saturation and equal score ratios no longer produce equal response ratios. Scale the whole posterior instead of one likelihood and prior conflict is handled differently. Change correlated noise and response amplitude no longer has the same evidential meaning. Let the parameter alter learning and two systems that agree now can disagree after one update.

Factor placement is part of the meaning

In categorical inference, one can write a posterior as p(s|y) proportional to p(s)p(y|s). A likelihood precision λ may produce p(s|y,λ) proportional to p(s)p(y|s)^λ. A global inverse temperature β applied after combining factors produces q_β(s) proportional to [p(s)p(y|s)]^β. These expressions agree when the prior is uniform, when β = λ and the same normalisation follows. They do not generally agree when the prior contains information.

This is why the phrase “precision is inverse temperature” is both tempting and incomplete. In energy-based formulations, inverse temperature multiplies an energy or log-probability and can look exactly like a precision parameter. But a probabilistic model assigns that multiplier to a named distribution or prediction error. The factor placement determines which disagreement is being resolved. Without it, the same notation can hide different models.

A neural implementation needs its own commitments

Probabilistic population codes show how a bridge can be made responsibly. In a specified exponential-family code, neural population activity carries sufficient statistics of a probability distribution. Under particular Poisson-like response assumptions, total activity can vary with certainty while the activity profile carries the estimated value. Bayesian combination can then be implemented by operations on the population responses.[7][8]

The strength of this proposal is its specificity. The response model, noise family, kernel and prior assumptions are declared. Its limitation is the same specificity. Saturation, nuisance-dependent covariance, non-Poisson variability or a different coding scheme can break the gain-to-precision relation. Recent prediction-error circuit models continue to investigate how sensory and prediction uncertainty might be estimated and combined, which underscores that the biological implementation remains an active mechanistic question rather than a settled synonym.[18]

One fit can hide several scales

Choice models often estimate an inverse-temperature parameter from behaviour. Yet multiplying all latent values by a constant and dividing the inverse temperature by the same constant preserves the choice probabilities. Unless the value scale is fixed by independent constraints, the fitted parameter absorbs both decision sensitivity and representation scale. A high inverse temperature may reflect more deterministic choice, larger value differences, or a modelling convention.

This is an identifiability problem, not a statistical nuisance to mention after interpretation. Computational cognitive modelling can fit behaviour well while assigning the fitted parameter the wrong psychological role. Simulation and recovery tests are needed to determine whether the proposed manipulation uniquely changes the parameter of interest.[19][21]

Counterfactuals that break the spell

A good comparison predicts not only where systems match, but also the intervention under which they must separate.

The saturation cliff

Imagine that the neural path applies tanh(gz) before a softmax. Near zero, tanh(x) is approximately x. Gain g then behaves like inverse temperature. Increase the input contrast and tanh approaches ±1. Score differences stop growing even as g rises. The temperature path continues scaling the original differences. The two outputs diverge. This is a local-to-global failure: a relation valid around one operating point was mistaken for a system-wide identity.

Common offsets reveal where gain sits

Softmax ignores a common offset: softmax(z + c1) equals softmax(z). A linear pre-softmax gain also preserves that invariance because g(z + c1) differs from gz by a common offset gc. A saturating gain does not. Adding a baseline can push some units towards a ceiling and compress their differences. A common-mode intervention therefore distinguishes pure score scaling from a nonlinear response change.

A post-normalisation response gain behaves differently again. Multiply every probability by the same positive constant and renormalise, and the original distribution returns unchanged. If a report says that “gain sharpened the probabilities” but the intervention was a common multiplier after normalisation, either another operation occurred or the description is wrong. Placement can turn an apparently powerful parameter into a mathematical no-op.

A louder population need not be a more precise population

The gain-to-precision bridge also depends on the noise model. Imagine two populations with the same tuning curves. In the first, spikes fluctuate independently with approximately Poisson-like variance. In the second, every neuron receives a strong shared fluctuation. Doubling mean activity can improve the first population’s discriminability while doing little for the second, because shared noise does not average away. Precision concerns the shape of an encoded distribution or the information available for a declared estimate. It is not identical to total firing rate.

This distinction is built into careful probabilistic population-code work. The relationship among tuning curves, covariance and sufficient statistics must satisfy specific conditions. Contrast can sometimes act as a useful nuisance parameter because it changes means and covariances in a coordinated way. An arbitrary gain intervention need not preserve that coordination.[8] A circuit can therefore become more active while its estimate becomes no more reliable, or even less reliable if correlated variability grows.

Normalisation adds another hidden dependency. Divisive normalisation computes a response relative to pooled activity, so increasing one channel can suppress its competitors, while increasing every channel may largely cancel. A scalar described as gain before the pool has a different effect from the same scalar after the pool. This is not a minor implementation detail. The normaliser defines the comparison class against which a response becomes large.[20]

A snapshot is not a trajectory

Suppose three systems produce the same action distribution at time t. The temperature-controlled system changes only the current readout. The precision-learning system updates its estimate of sensor reliability after surprising observations. The gain-controlled recurrent system changes state occupancy and may alter which information reaches later stages. At t + 1, the systems need not remain matched even if the external input repeats.

Calibration under shift is a mechanism challenge

Temperature scaling is often fitted on a validation set by minimising negative log-likelihood. This can make confidence frequencies better aligned with accuracy on similar data. It does not repair the representation that produced the logits. If a shift changes class-conditional structure, label prevalence or shortcut use, one global scalar may no longer be appropriate. Empirical studies have found that calibration methods can degrade under shift, while later work shows architecture and pretraining can change the pattern.[4][5]

This is another reason not to interpret temperature as latent precision. A model can be sharply wrong because its scores encode a spurious feature. Softening those scores may improve aggregate calibration without identifying which observations are unreliable. Conversely, a probabilistic model with explicit input-dependent variance can alter its weighting case by case, though that variance model can itself be misspecified. The relevant distinction is between reshaping reported confidence and modelling the source of uncertainty.

stateDiagram-v2
  [*] --> Claim
  state "Temperature changes a<br/>comparison after scores exist" as Claim
  state "Precision changes the weight<br/>of a probabilistic source" as Test
  state "Gain changes a response where<br/>it is implemented" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Audit the parameter before you trust the analogy

The practical response is not to ban comparison. It is to make every comparison carry its locus, factorisation and break tests.

A configured system needs three separate controls

Consider a synthetic inspection assistant reviewing images of a bridge joint. The perception model estimates whether the joint is clear, needs inspection or requires a halt. The runtime also retrieves a prior from maintenance history. A policy proposes an action, while an authority kernel decides whether the proposal may execute. Three scalar controls may be present, but they belong to different contracts.

The perception layer may estimate observation precision from blur, occlusion or sensor diagnostics. That precision should control how current evidence combines with the prior. The policy layer may use a temperature to explore or calibrate among already formed action scores. A recurrent vision or attention module may have a gain parameter that changes feature sensitivity and network dynamics. None of these parameters should decide authority. A sharper proposal is still only a proposal; permission belongs to a deterministic policy and typed action contract.

Executable lab: a locus-first parameterisation audit

The interactive lab below creates three distributions from the same prior and evidence. Temperature scales the whole combined score. Precision scales only the evidence factor. Gain scales evidence before a selectable transfer function. The controls let you enter the equivalence corridor, introduce prior conflict, and push gain into saturation. Jensen-Shannon divergence measures how far the distributions separate.

A positive result permits a narrow claim: the tested parameterisations produce similar distributions under the tested interventions. A negative result means the mapping fails even at that grade. Neither result establishes that a biological population represents precision, that a temperature is calibrated under future shift, or that one scalar is the unique cause of observed behaviour.

from __future__ import annotations

import math
from dataclasses import dataclass
from typing import Callable, Sequence

Vector = tuple[float, ...]


def softmax(values: Sequence[float]) -> Vector:
    peak = max(values)
    weights = [math.exp(value - peak) for value in values]
    total = sum(weights)
    return tuple(weight / total for weight in weights)


def js_divergence(left: Vector, right: Vector) -> float:
    midpoint = tuple((a + b) / 2 for a, b in zip(left, right))

    def kl(p: Vector, q: Vector) -> float:
        return sum(a * math.log(a / b) for a, b in zip(p, q) if a > 0)

    return 0.5 * kl(left, midpoint) + 0.5 * kl(right, midpoint)


def transfer(name: str) -> Callable[[float], float]:
    if name == "linear":
        return lambda x: x
    if name == "tanh":
        return math.tanh
    if name == "threshold":
        return lambda x: max(0.0, x - 0.35)
    raise ValueError(f"Unknown gain transfer: {name}")


@dataclass(frozen=True)
class Audit:
    contrast: float = 1.0
    prior_bias: float = 0.0
    temperature: float = 1.0
    precision: float = 1.0
    gain: float = 1.0
    gain_transfer: str = "linear"

    def run(self) -> dict[str, Vector | float]:
        evidence = (self.contrast, 0.0, -self.contrast)
        prior = (0.0, 0.0, self.prior_bias)
        f = transfer(self.gain_transfer)

        temperature_path = softmax(
            [(p + e) / self.temperature for p, e in zip(prior, evidence)]
        )
        precision_path = softmax(
            [p + self.precision * e for p, e in zip(prior, evidence)]
        )
        gain_path = softmax(
            [p + f(self.gain * e) for p, e in zip(prior, evidence)]
        )

        return {
            "temperature": temperature_path,
            "precision": precision_path,
            "gain": gain_path,
            "js_temperature_precision": js_divergence(temperature_path, precision_path),
            "js_temperature_gain": js_divergence(temperature_path, gain_path),
            "js_precision_gain": js_divergence(precision_path, gain_path),
        }


if __name__ == "__main__":
    experiments = {
        "corridor": Audit(),
        "prior_conflict": Audit(prior_bias=1.25, temperature=0.5, precision=2, gain=2),
        "saturation": Audit(contrast=2, temperature=0.5, precision=2, gain=2, gain_transfer="tanh"),
    }
    for name, experiment in experiments.items():
        print(f"\n{name}")
        for key, value in experiment.run().items():
            print(f"  {key}: {value}")

The parameterisation audit

Before interpreting a parameter, write a one-page audit. The first line names the scaled quantity in physical or computational terms. The second marks the causal locus. The third states what remains fixed. The fourth specifies the downstream normalisation. The fifth lists the timescale. Only then record the proposed meaning, such as calibration sensitivity, likelihood precision or synaptic responsiveness.

Next, add break tests. Swap the prior while holding evidence fixed. Add a common offset. Vary input contrast to test linearity and saturation. Change noise covariance without changing the mean. Interrupt recurrent state. Repeat after one learning update. If the proposed translation survives the interventions it claims to preserve, promote it. If not, retain the narrower description.

Applied to the bridge-joint assistant, the record might say: the source is a per-image log variance from the perception model; the map treats its negative as likelihood precision; a blur intervention should lower precision without changing the maintenance prior; a label-frequency shift should challenge calibration without necessarily changing blur; an alternative explanation is that the variance head merely tracks image brightness; and the permitted decision is to route low-precision cases for inspection, not to let the model authorise a halt. This compact record prevents one fluent word, “certainty”, from swallowing several contracts.

The strongest boundary

A designed architecture can make the bridge stronger than it is in ordinary systems. An exponential-family code may deliberately represent natural parameters in neural activity. A precision controller may deliberately modulate the gain of selected prediction-error units. A policy may deliberately use the same sufficient statistic and normaliser. In such a system, a formal mapping can support real mechanistic work.

Even there, the claim remains scoped. A mechanism can implement a Bayesian precision term without making every firing-rate increase a precision estimate. A temperature can be part of a generative model rather than a post-hoc decoder. Gain can be measured at one layer while another layer compensates. Biological arousal can produce an inverted performance curve rather than monotonic sharpening. The audit is not scepticism for its own sake. It preserves the conditions under which a comparison earns explanatory force.

sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: One distribution, three stories
  M->>E: Temperature changes a comparison after scores exist
  E-->>M: compare rival explanations
  E->>D: Locate the dial before translating the label
  D-->>Q: revise scope or act

Locate the dial before translating the label

When a paper, model card or experiment says that a system became more precise, more confident, lower-temperature or higher-gain, do not begin by asking whether the words are approximately synonymous. Ask four concrete questions: what quantity was scaled, where was it scaled, what competing factor remained fixed, and what changed over time?

If the answer is “all logits at the final readout”, interpret the parameter as output sensitivity or a calibration transform. If a named likelihood or prediction error is weighted relative to a prior, interpret it as precision within that declared probabilistic model. If a response function or recurrent circuit is modulated, interpret it as gain at that locus and measure its effects on noise, saturation and dynamics. Translate among them only after the mapping survives the interventions that should leave it unchanged.

This changes model design as well as language. Keep sensor reliability separate from policy decisiveness. Keep policy decisiveness separate from action authority. Keep a fitted behavioural inverse temperature separate from a claim about neuromodulation. When one scalar is used for several roles, record the coupling as a design choice and test its failure modes rather than treating the coupling as natural law.

The same rule improves research communication. Report “temperature-scaled logits” rather than “increased certainty”. Report “likelihood precision in this Gaussian observation model” rather than “attention became more precise”. Report “multiplicative response gain in this population and operating range” rather than “the brain lowered its temperature”. When the evidence reaches only a structural comparison, say so. That wording preserves a useful bridge while keeping rivals alive.

The practical rule is locus before label. A matching distribution permits a local computational shortcut. A matching intervention fingerprint permits a stronger structural comparison. Only a preserved causal role permits a mechanism claim. The three dials can be coupled, but the coupling is something to establish, not something their names provide.