The promise inside a probability

The simplest mistake is to read confidence as a private feeling inside the model. A classifier emits a number because its final computation places mass over possible labels. A language system may expose token probabilities, a consistency score or an auxiliary estimate of answer correctness. None of these quantities arrives with its meaning guaranteed.

Meaning comes from a comparison. Collect cases that received scores near 0.80. If roughly 80 per cent were correct, the score is calibrated for that population, outcome definition and time window. Brier’s early work on probabilistic forecasts made this operational: a probability forecast is evaluated by how its stated probabilities meet realised events, not by how confident the forecaster sounds.^[1]

Calibration is therefore a relationship among scores, outcomes and a reference population. Remove any one of the three and “80 per cent confident” becomes under-specified. Change the label policy, the input mix or the time horizon, and the old relationship may no longer hold.

Two uncertainties can demand different refusals

Consider two packets that both receive 0.70. Packet A contains a blurred photograph. Even an expert cannot recover the missing characters from the available pixels. Packet B is sharp, but it uses a layout absent from training. The same numerical uncertainty now points towards different interventions.

For Packet A, more examples of the same blurred image do not restore the information. The useful abstention asks for a new scan. For Packet B, a specialist may label the novel layout, after which retraining or retrieval could reduce future uncertainty. The first problem lies largely in the observation; the second lies in the model’s limited coverage of the world.

These are often discussed as aleatoric and epistemic uncertainty. The names are helpful when they guide an intervention, but the decomposition is not handed to us by an ordinary softmax score. Different modelling assumptions can assign the same residual error to different sources. A low confidence number alone does not tell the system whether to gather another sensor reading, consult a person, retrieve evidence or improve the model.

The gate needs a route-relevant reason, not only an uncertainty magnitude. A practical abstention record should include the triggering score, the suspected failure mode, the requested next action and whether that action can change the evidence. This prevents every refusal from collapsing into a generic review queue. It also creates testable hypotheses: rescans should resolve observation failures; additional labelled layouts should resolve coverage failures; neither should be credited without outcome readback.

Accuracy, ranking and calibration answer different questions

Imagine two models that each classify eight of ten parcels correctly. Model Flint assigns 0.80 to every answer. Model Glass assigns 0.99 to every answer. Their accuracy is identical. Their promises are not. Flint is roughly calibrated on this batch. Glass is severely overconfident.

Temperature scaling offers a clean illustration. It divides a model’s logits by one learned positive temperature before applying softmax. The argmax usually stays fixed, so accuracy and score ordering stay fixed, while the numerical probabilities spread or contract. Guo and colleagues found this simple post-processing step effective on many studied datasets.^[3] Later work showed that the historical story is not “newer models are always worse calibrated”. Architecture, pretraining and evaluation conditions matter, and some more recent vision models were among the better calibrated systems studied.^[4]

This correction matters. Calibration is not a permanent personality trait of an architecture. It is an empirical property to be measured for a configured model, dataset and scoring rule.

Calibration estimates have their own uncertainty

Now change only the sample size. In one evaluation, nine of ten cases scored near 0.90 are correct. In another, nine hundred of one thousand are correct. Both bins show the same point estimate, but they support very different confidence in the claim. A reliability diagram that omits bin counts can make these observations look equivalent.

This matters most where abstention is supposed to protect the edge cases. Rare classes, new suppliers and low-resource languages often occupy the smallest bins. Their apparent calibration can swing after only a few outcomes. A gate that releases on a point estimate may therefore become most certain exactly where its evidence is thinnest.

There is a second layer of estimation. The calibrator was fitted from data, so it can overfit, especially when the mapping is flexible and the sample is small. Separate fitting from evaluation unless the method supplies a valid alternative. Work on verified and distribution-free calibration makes this statistical burden explicit rather than treating a low empirical ECE as self-authenticating.^[15][16]

A calibrated score should travel with its support. At minimum, record the number of examples behind each operating region, uncertainty around the measured error, and the slices for which the estimate is too weak to authorise action. “Insufficient evidence” is a different reason to abstain from “high predicted error”, and the ledger should preserve that distinction.

Reliability diagrams expose this by plotting observed frequency against stated probability. Proper scoring rules add another constraint: a model should not improve its expected score by reporting a probability other than its genuine estimate. The Brier score and log loss are common examples.^[1][2] Expected calibration error can be useful, but its value depends on bins and aggregation. It should be treated as a diagnostic, not as a release certificate.

flowchart TD
  A["The promise inside a probability"] --> B["The threshold moves with<br/>consequences"]
  B --> C["How safe gates become unsafe"]
  C --> D["Build the right-to-abstain<br/>contract"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#D95545,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

The threshold moves with consequences

Suppose a model is well calibrated. Should every case below 0.80 be rejected? There is no answer until we know what an error costs, what abstention costs and what happens after refusal.

Take binary correctness. Let p be the calibrated probability that the proposed answer is correct. Let C_e be the loss of an error and C_a the loss of abstaining. Assume a correct answer has zero loss. Acting has expected loss (1 − p)C_e. Abstaining has loss C_a. Accept when the former is no larger than the latter.

With error cost 8 and abstention cost 1.5, the threshold is 1 − 1.5/8 = 0.8125. At 0.85, expected action loss is 0.15 × 8 = 1.2, so acting is cheaper. At 0.75, expected action loss is 2.0, so abstaining is cheaper.

From a confidence threshold to an action matrix

The binary equation is useful because its assumptions are visible. Real systems often have several labels and several possible actions. Then the top probability is not enough. Let p(y) be the calibrated probability of each outcome y, and let L(a,y) be the loss of taking action a when y is true. Choose the action with the lowest expected loss, including abstention as one candidate action.

This extension also reveals a design choice. “Abstain” need not mean “ask a person”. It can mean request one missing field, retrieve another source, return a set of plausible labels, place the case in a safe hold, or execute a reversible diagnostic action. Each route has its own cost and information value. The gate should compare real alternatives rather than a fictional choice between automatic action and nothing.

Coverage is the price paid for lower selective risk

Selective prediction represents a classifier as two functions: a predictor f and a selection function g. The selector accepts or rejects each case. Coverage is the fraction accepted. Selective risk is the error rate among accepted cases. The complete profile is a risk–coverage curve, not one chosen threshold.^[6][7]

If confidence ranks cases well, accepting fewer cases should reduce error among those retained. The curve reveals how much work the system must defer to reach a target risk. A gate that achieves 1 per cent error at 2 per cent coverage may be statistically impressive and operationally useless.

The distinction explains why a monotone calibrator can improve probability quality without improving who gets rejected. It changes the labels on the confidence axis but usually preserves the order. Large empirical comparisons have also shown that uncertainty quality is multi-dimensional: calibration, ranking and selective performance can disagree across models and metrics.^[11]

How safe gates become unsafe

A calibrated gate is fitted to a relationship observed somewhere. It fails when deployment changes that relationship, when aggregation hides a local mismatch, or when the route for rejected cases stops behaving as assumed.

Shift changes the meaning of the same score

Ovadia and colleagues compared uncertainty methods under increasing dataset shift and found that accuracy and calibration can deteriorate as inputs move away from the training distribution. Post-hoc calibration fitted before the shift did not provide a general cure.^[5] The causal point is simple: the calibrator learned P(correct | score) in one environment. Shift can change that conditional relationship.

Out-of-distribution detection can help when the new inputs look different in a detectable way. It cannot guarantee safety. Some shifts preserve superficial features while changing labels, costs or base rates. A gate should therefore monitor both input change and outcome change. The first offers early warning. The second tells us whether the promise actually broke.

Global calibration can hide local damage

Return to a bin scored 0.80. Suppose group North is correct 92 per cent of the time while group South is correct 68 per cent. If both groups are equally common, the combined accuracy is 80 per cent. The model appears perfectly calibrated globally while being underconfident for one group and overconfident for the other.

Selection creates another distribution. Once the gate accepts only high scores, calibration should be checked on the accepted region and along the actual selection rule. A model can be calibrated overall and poorly calibrated after a complex selector filters cases using language, class, route or a second model. The selected population is the one that acts.

Outcome readback can become selectively blind

Calibration requires outcomes, but operating systems rarely observe every outcome in the same way. Accepted cases may finish quickly and produce an automated status. Rejected cases may take days to review. Some mistakes are corrected silently by a downstream worker. Others become visible only after an appeal. If the dashboard joins only fast, machine-readable outcomes, it measures a convenience sample rather than the gate’s real population.

The remedy is an outcome manifest. Record which cases lack labels, why they are missing, when they are expected, and which routing decision affected their visibility. Maintain a small random audit sample across accepted cases, even when ordinary operations would not review them. Without that counterfactual sample, the system may never learn the error rate of automation because automation itself suppresses observation.

Readback latency is part of the calibration horizon. A gate can remain provisionally open while labels are delayed, but it should carry an evidence-age limit and conservative bounds. Once missingness or delay becomes associated with score, supplier, language or outcome severity, the current calibration claim should be downgraded.

Abstention can fail after the model has behaved correctly

Free-form language generation adds another complication. Token likelihood is not automatically the probability that a paragraph is factually correct. Different wordings can express the same claim, and a system may be consistently wrong rather than variably uncertain. Semantic-entropy work targets one subset of failures, arbitrary confabulations, by grouping generations at the level of meaning. Its authors explicitly note that this does not catch every systematic error.^[12] The general lesson is unchanged: define the event being calibrated before treating any uncertainty score as a gate input.

stateDiagram-v2
  [*] --> Claim
  state "Two uncertainties can demand<br/>different refusals" as Claim
  state "Accuracy, ranking and<br/>calibration answer different<br/>questions" as Test
  state "The twin models" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Build the right-to-abstain contract

A deployable gate needs more than a threshold. It needs a versioned score, a calibrator, a loss policy, a fallback route, monitoring and an outcome ledger. These elements form one configured system. Evaluating only the model weights answers the wrong question.

Three clocks govern one permission

A calibrated abstention gate is supported by three clocks that rarely tick at the same speed. The evidence clock measures how long the score-to-outcome relationship remains defensible. It advances when the input mix, label policy, model, calibrator or scoring interface changes. The consequence clock measures how long the approved loss model remains current. It can change when a payment limit, regulatory duty, recovery cost or tolerance for delay changes. The capacity clock measures whether the fallback can absorb the next interval of deferred work. It can expire within minutes as arrivals, staffing and service times move.

Consider Nacre-39 at 10:00. Its calibrator was fitted on the previous month and remains inside its seven-day evidence window. The loss policy was approved that morning. Review has room for 400 packets. By 13:00 a supplier sends an unfamiliar template burst. The score distribution changes only slightly, so a global drift alert stays quiet. Yet the abstention queue reaches 390 and its arrival rate exceeds the review team’s service rate. The score may still be a reasonable estimate of correctness, but permission to defer is about to fail. At 14:00 the organisation raises the value threshold above which a completeness error becomes irreversible. The same probability now supports a different action even if both calibration and capacity recover.

The practical rule is conjunctive: the gate carries separate version identifiers and expiries for evidence, loss and fallback. Each decision receipt records which versions were used. A fast capacity monitor can close or narrow the gate without pretending that the model became less calibrated. A policy change can demand a new threshold without retraining the model. A distribution trigger can revoke the calibrator while leaving the fallback route intact.

The gate’s permission expires when the first supporting clock expires. This distinction prevents operational overload from being mislabelled as model uncertainty, and prevents a stable model metric from authorising action under stale consequences. It also makes recovery precise: renew only the failed premise, then re-evaluate the conjunction before restoring coverage.

A realistic configured-system example

Nacre-39 classifies synthetic application packets as complete, incomplete or ambiguous. Its model proposal contains a label and raw logits. A calibrator fitted on a recent, held-out sample maps the top-label score to estimated correctness. A policy service supplies class-specific error costs and the current review cost. A deterministic gate compares expected losses. Accepted packets move to automated routing. Rejected packets enter a review queue with a capacity limit and maximum age. Later corrections return to an outcome ledger.

The model cannot authorise its own action. It proposes a score. The gate owns the decision. The review service owns the fallback. The outcome ledger owns the evidence used to renew or revoke permission.

The smallest useful implementation

The minimum credible build uses one held-out calibration set, one proper scoring rule, one reliability view, one risk–coverage curve, one explicit cost model, one capacity limit and delayed outcome readback. It does not require a Bayesian neural network or an elaborate ensemble. Those methods may improve the score, but they do not replace the contract.

A practical protocol follows.

Release against a bound, not a flattering point

Suppose a shadow run accepts one hundred cases and observes four errors. The point estimate of selective risk is 4 per cent. A second run accepts ten thousand cases and observes four hundred errors. It has the same point estimate but far stronger evidence about the operating region.

A release rule should therefore predeclare both a target and a confidence requirement. Selective-classification work has framed this directly: choose a selector intended to keep true risk below a target with high probability, while maximising coverage.^[7] The exact bound depends on assumptions and method, but the engineering consequence is stable. A threshold should not graduate from shadow mode because its observed risk happens to fall just below a target on a small sample.

Use three statuses rather than a binary pass. Release means the upper risk bound, slice checks and capacity tests pass. Shadow means the point estimate is promising but support is insufficient. Block means the measured or bounded risk, the fallback load or the evidence quality violates the contract. This keeps statistical uncertainty from being hidden inside an optimistic green cell.

Runnable Python version

The script below uses only the Python standard library. It generates overconfident synthetic scores, fits one temperature on a calibration split, calculates Brier score and binned ECE, derives the gate threshold from costs, and checks fallback capacity. Run it once without shift, then again with --shift 0.12.

#!/usr/bin/env python3
"""Synthetic calibration and abstention gate lab for Paper 39.

No third-party packages are required.
Run:
    python calibrated_abstention_lab.py
    python calibrated_abstention_lab.py --shift 0.12 --error-cost 8 --abstain-cost 1.5
"""
from __future__ import annotations

import argparse
import math
import random
from dataclasses import dataclass
from typing import Iterable, Sequence

EPS = 1e-12


@dataclass(frozen=True)
class Case:
    logit: float
    correct: int


def sigmoid(z: float) -> float:
    return 1.0 / (1.0 + math.exp(-z))


def clip(p: float) -> float:
    return min(1.0 - EPS, max(EPS, p))


def generate_cases(n: int, seed: int, shift: float = 0.0) -> list[Case]:
    """Generate overconfident scores with optional deployment degradation.

    The model emits sigmoid(logit), while correctness is sampled from
    sigmoid(logit / 2) - shift. The mismatch makes the raw model overconfident.
    """
    rng = random.Random(seed)
    cases: list[Case] = []
    for _ in range(n):
        logit = 4.0 * rng.random()
        true_p = min(0.99, max(0.01, sigmoid(logit / 2.0) - shift))
        cases.append(Case(logit=logit, correct=int(rng.random() < true_p)))
    return cases


def probability(case: Case, temperature: float) -> float:
    return clip(sigmoid(case.logit / temperature))


def log_loss(cases: Sequence[Case], temperature: float) -> float:
    total = 0.0
    for case in cases:
        p = probability(case, temperature)
        y = case.correct
        total -= y * math.log(p) + (1 - y) * math.log(1 - p)
    return total / len(cases)


def fit_temperature(cases: Sequence[Case]) -> float:
    """Small deterministic grid search, sufficient for this teaching lab."""
    candidates = [0.50 + i * 0.01 for i in range(351)]
    return min(candidates, key=lambda t: log_loss(cases, t))


def brier_score(cases: Sequence[Case], temperature: float) -> float:
    return sum(
        (probability(case, temperature) - case.correct) ** 2 for case in cases
    ) / len(cases)


def expected_calibration_error(
    cases: Sequence[Case], temperature: float, bins: int = 10
) -> float:
    """Top-label ECE. Useful as a diagnostic, not a complete guarantee."""
    buckets: list[list[tuple[float, int]]] = [[] for _ in range(bins)]
    for case in cases:
        p = probability(case, temperature)
        index = min(bins - 1, int((p - 0.5) / 0.5 * bins))
        buckets[index].append((p, case.correct))

    ece = 0.0
    for bucket in buckets:
        if not bucket:
            continue
        mean_p = sum(p for p, _ in bucket) / len(bucket)
        accuracy = sum(y for _, y in bucket) / len(bucket)
        ece += len(bucket) / len(cases) * abs(mean_p - accuracy)
    return ece


def gate_threshold(error_cost: float, abstain_cost: float) -> float:
    """Accept when expected error loss is no greater than abstention loss."""
    if error_cost <= 0:
        raise ValueError("error_cost must be positive")
    return min(1.0, max(0.0, 1.0 - abstain_cost / error_cost))


@dataclass(frozen=True)
class GateReport:
    threshold: float
    coverage: float
    selective_risk: float
    abstention_rate: float
    average_cost: float
    capacity_ok: bool


def evaluate_gate(
    cases: Sequence[Case],
    temperature: float,
    error_cost: float,
    abstain_cost: float,
    fallback_capacity: float,
) -> GateReport:
    threshold = gate_threshold(error_cost, abstain_cost)
    accepted = [case for case in cases if probability(case, temperature) >= threshold]
    rejected = len(cases) - len(accepted)
    coverage = len(accepted) / len(cases)
    selective_risk = (
        sum(1 - case.correct for case in accepted) / len(accepted)
        if accepted
        else 0.0
    )
    average_cost = (
        sum((1 - case.correct) * error_cost for case in accepted)
        + rejected * abstain_cost
    ) / len(cases)
    abstention_rate = 1.0 - coverage
    return GateReport(
        threshold=threshold,
        coverage=coverage,
        selective_risk=selective_risk,
        abstention_rate=abstention_rate,
        average_cost=average_cost,
        capacity_ok=abstention_rate <= fallback_capacity,
    )


def pct(value: float) -> str:
    return f"{100.0 * value:6.2f}%"


def print_report(
    name: str,
    cases: Sequence[Case],
    temperature: float,
    error_cost: float,
    abstain_cost: float,
    fallback_capacity: float,
) -> None:
    report = evaluate_gate(
        cases, temperature, error_cost, abstain_cost, fallback_capacity
    )
    print(f"\n{name}")
    print("-" * len(name))
    print(f"temperature       {temperature:6.2f}")
    print(f"accuracy          {pct(sum(c.correct for c in cases) / len(cases))}")
    print(f"mean confidence   {pct(sum(probability(c, temperature) for c in cases) / len(cases))}")
    print(f"Brier score       {brier_score(cases, temperature):6.3f}")
    print(f"ECE (10 bins)     {expected_calibration_error(cases, temperature):6.3f}")
    print(f"gate threshold    {report.threshold:6.3f}")
    print(f"coverage          {pct(report.coverage)}")
    print(f"selective risk    {pct(report.selective_risk)}")
    print(f"abstention rate   {pct(report.abstention_rate)}")
    print(f"average cost      {report.average_cost:6.3f}")
    print(f"fallback capacity {'PASS' if report.capacity_ok else 'FAIL'}")


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser()
    parser.add_argument("--shift", type=float, default=0.0)
    parser.add_argument("--error-cost", type=float, default=8.0)
    parser.add_argument("--abstain-cost", type=float, default=1.5)
    parser.add_argument("--fallback-capacity", type=float, default=0.35)
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    calibration = generate_cases(1_200, seed=39, shift=0.0)
    deployment = generate_cases(2_000, seed=390, shift=max(0.0, args.shift))
    fitted_temperature = fit_temperature(calibration)

    print(f"Fitted temperature on calibration data: {fitted_temperature:.2f}")
    print_report(
        "Raw model on deployment data",
        deployment,
        temperature=1.0,
        error_cost=args.error_cost,
        abstain_cost=args.abstain_cost,
        fallback_capacity=args.fallback_capacity,
    )
    print_report(
        "Calibrated model on deployment data",
        deployment,
        temperature=fitted_temperature,
        error_cost=args.error_cost,
        abstain_cost=args.abstain_cost,
        fallback_capacity=args.fallback_capacity,
    )

    print("\nInterpretation")
    if args.shift > 0:
        print(
            "The calibrator was fitted before the injected shift. A worse ECE or "
            "selective risk is evidence to block or revalidate the gate, not to "
            "retune the threshold silently."
        )
    else:
        print(
            "A lower proper score and a workable risk-coverage trade-off support "
            "this gate only for data represented by the calibration sample."
        )


if __name__ == "__main__":
    main()

What conformal prediction changes, and what it does not

Conformal prediction can wrap a model to produce prediction sets or intervals with finite-sample marginal coverage under exchangeability. That is a stronger statistical statement than an informal confidence threshold, and it can turn uncertainty into a set rather than a forced singleton.^[9] Risk-controlling variants extend the idea to broader losses.^[10]

Yet the guarantee is only as relevant as its assumptions and target. Marginal coverage does not promise coverage for every subgroup. Exchangeability can fail under time dependence or shift. A prediction set containing the correct class may still be too broad to act on. Conformal methods strengthen the score-validity layer; they do not price an error or staff the fallback.