Two forecasts say 100

Two operations teams are told to prepare for 100 cases tomorrow. The first queue usually lands at 90, 100 or 110. The second usually lands at 60, 100 or 140. Both forecasts have the same mean. Only one is remotely described by the number 100.

Suppose unused capacity costs one unit, while each case left unserved costs four. The first team should prepare for 110. The second should prepare for 140. Sending both teams the mean gives the same instruction even though the same shortfall has very different probability. The number did not merely simplify the forecast. It removed the feature that determined the action.

The causal mistake is collapsing uncertainty before the loss is known. Once a predictive distribution becomes one value, no downstream workflow can reconstruct its tail, multiple regimes or uncertainty around the estimate. A decision engine then treats the point as though it were a sufficient description of the future. Sometimes that is exactly right. Often it is an undocumented bet about consequences.

Part IA point is a decision in disguise

A predictive model for a future quantity does not naturally produce a single correct number. It supplies, explicitly or implicitly, a conditional distribution over possible outcomes. Write that distribution as P(Y | x), where Y is the future outcome and x is the information available now. A decision chooses an action a. The loss function L(a, Y) says what each action costs when each future occurs.

This equation exposes why “the forecast” is an incomplete specification. A distribution becomes a point only after a loss selects a functional from it. Under squared error, the conditional mean minimises expected loss. Under absolute error, the median does. Under asymmetric absolute loss, a quantile does. Quantile regression formalises this last case by giving underprediction and overprediction different weights. Research on point-forecast evaluation reaches the same operational conclusion: the scoring function and forecast directive must match, or comparisons can become misleading.

The mean is therefore an answer to a question about squared error. It is not the universal centre of every decision. A median says something different. A 95th percentile says something different again. Naming the statistic without naming the loss makes the statistic look objective while the value judgement remains hidden.

Thought experiment: change only the consequence

Hold the narrow three-outcome distribution fixed: 90 with probability 0.25, 100 with probability 0.50, and 110 with probability 0.25. First imagine a forecasting competition scored by squared error. Predicting 100 is optimal. Now keep every probability unchanged and replace the competition with a capacity decision. One unit of unused capacity costs one; one unit of unmet demand costs four. The optimal action moves to 110.

Nothing about the world changed. Only the consequence model changed. This isolates the causal feature that moved the decision. A team that declares “our forecast is 100” has silently chosen the first problem. A team that hands over the distribution allows the second decision maker to ask the question it actually faces.

Worked example: equal means, unequal regret

The opening distributions make the mechanism calculable by hand. Both have mean 100. Their cumulative probability at 100 is 0.75. With an underage cost of four and an overage cost of one, the critical fractile is 4/(4+1) = 0.8. The smallest action whose cumulative probability reaches 0.8 is 110 for the narrow case and 140 for the tailed case.

The same compression error creates four times as much regret in the tailed case. More importantly, the point interface cannot explain why. It exposes neither the 25 per cent chance of 140 nor the loss ratio that makes that tail decisive.

Regret is the missing unit

Forecast error measures the distance between a reported value and an outcome. Decision regret measures the price of using the wrong action. The two coincide only under special choices of action space and loss. A point can be close to the realised outcome and still trigger an expensive threshold breach. It can be far from the outcome and still be the best action available before uncertainty resolves.

This quantity changes evaluation. Instead of asking only whether mean absolute error fell, replay historical decisions under the point interface and the retained distribution. Apply the same constraints and loss to both. The difference is compression regret. A model with slightly worse point accuracy may make better capacity decisions because its upper quantiles are calibrated. A sophisticated density may add no value because the approved action depends only on the mean.

Loss estimates carry uncertainty too. The underage cost may include uncertain overtime, delay and control consequences. A serious experiment sweeps plausible cost ratios and records where the preferred action changes. If a tiny change moves the action sharply, the decision needs a contingency or approval trigger. If it remains stable across the credible range, a simpler point contract may be defensible. Sufficiency should survive uncertainty in the consequence model, not merely fit one convenient number.

The serious negative control

There are important problems where a point really is sufficient. If the action is unconstrained, the complete loss is squared error, and downstream users need no threshold, tail probability or uncertainty-sensitive choice, then the conditional mean contains everything required to select the Bayes action. In the lab, 100 is optimal for both distributions under squared loss. Their expected squared losses differ, but the action does not.

A distribution-first rule should permit collapse when sufficiency is demonstrated. Otherwise it becomes ceremony. The burden is not “always ship every probability”. The burden is “prove why this point preserves the decision”.

The strongest objection follows naturally: every operational process eventually emits a point action. A warehouse orders an integer, a service chooses a staffing level, and a controller sets one command. That objection is correct about the endpoint and wrong about timing. Collapse is unavoidable. Premature collapse is optional. The distribution should survive until the decision has access to consequences, constraints and recovery options.

flowchart TD
  A["Two forecasts say 100"] --> B["Part I A point is a decision<br/>in disguise"]
  B --> C["Part II What disappears in the<br/>collapse"]
  C --> D["Part III The bootstrap as a<br/>bounded instrument"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#D95545,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Part IIWhat disappears in the collapse

A point estimate discards more than a confidence interval. It discards the geometry of plausible futures. Four losses are especially consequential: tail mass, multiple regimes, input-dependent spread and dependence among quantities. Each can be absent from the point while dominating the decision.

Tail mass survives neither the mean nor the standard deviation

Consider two new distributions. The first places equal probability at 90 and 110. The second places probability 0.125 at 80, 0.75 at 100 and 0.125 at 120. Both have mean 100 and variance 100. A report containing “100 ± 10” makes them identical. Yet the probability of exceeding 115 is zero in the first and 12.5 per cent in the second.

This is the second thought experiment. Mean and variance are fixed; only shape changes. A threshold decision at 115 reverses because higher moments and tail placement matter. The example also marks a boundary of familiar error bars: a point plus a scale is not a distribution unless a justified family, such as a Gaussian model, supplies the missing shape.

Modes are rival stories about the world

A bimodal forecast often means the future is governed by unresolved regimes. Demand may be ordinary unless an event occurs. A component may survive unless a latent defect triggers failure. A customer may repay normally unless a refinancing condition is breached. The mean can lie in a valley that almost never occurs. Acting on it produces a plan optimised for a synthetic middle world.

A mixture should sometimes be resolved upstream by acquiring more evidence. At other times the regimes remain genuinely uncertain and both must reach the decision. The point answer prevents either response because it hides that there were two stories.

Spread can depend on the case

One global residual standard deviation treats every input as equally predictable. Real systems are often heteroscedastic: uncertainty expands for rare entities, longer horizons, poor sensors, novel language or boundary conditions. A model can have respectable average error while being dangerously precise on exactly the cases that require caution.

Distributional regression, quantile regression, ensembles and related methods attempt to make this local structure visible. Deep ensembles have shown a practical route to predictive uncertainty in benchmark studies, while calibration work demonstrates that model confidence can still fail to match observed frequencies. The method matters less than the obligation: evaluate uncertainty on the slices where it changes action.

Dependence determines portfolio risk

Ten marginal distributions do not describe their joint behaviour. If all ten queues surge together after the same event, pooling capacity differs from a case where their errors cancel. Adding ten point forecasts erases both marginal uncertainty and correlation. Adding ten independent samples from separate models can erase correlation just as thoroughly.

When the decision spans several quantities, preserve a joint predictive object or an explicit dependence model. Per-item intervals may look prudent while the aggregate reserve remains structurally wrong.

Uncertainty can change what the system observes next

A distribution does more than select a conservative action. It can reveal whether another observation is worth buying. In the regime example, a new event indicator may separate ordinary demand from a surge. Its value is the expected reduction in decision loss after observing it, minus the acquisition cost and delay. A point forecast cannot support this calculation because it no longer contains the rival futures that the observation might distinguish.

This matters for AI systems that can retrieve, ask a user, run a test or escalate. A high-entropy distribution does not automatically justify more evidence; uncertainty that no available observation can reduce may be irreducible. Conversely, a narrow forecast can still warrant a check if its confidence rests on a brittle assumption. The useful control question is whether a permitted observation is expected to change the action or its authority boundary.

Information seeking belongs between prediction and action. The system should compare acting now, acquiring evidence, deferring and escalating under the same consequence model. Collapsing to a point before that comparison removes the state needed to value information and often turns an epistemic problem into an arbitrary confidence threshold.

Honest uncertainty needs calibration and sharpness

A wide distribution can avoid embarrassment by assigning probability almost everywhere. A narrow distribution can look useful while missing too often. Probabilistic forecasting therefore evaluates two properties together. Calibration asks whether stated probabilities agree with observed frequencies over an appropriate sequence or population. Sharpness asks how concentrated the forecast is. The working aim is to make forecasts as sharp as possible subject to calibration.

Proper scoring rules turn that aim into an incentive. A strictly proper score makes the forecaster's expected score best when it reports its true predictive distribution. Log score and continuous ranked probability score are common examples. They do not prove a model is correct, but they prevent an evaluation from rewarding deliberate probability distortion in expectation.

Point metrics cannot audit a distributional claim. A model trained to output quantiles or densities and ranked only by root mean squared error may learn a good mean while its tails remain untested. Evaluation should include proper distributional scores, coverage and width diagnostics, calibration by decision-relevant slice, and the actual downstream loss.

There is more than one uncertainty

“Uncertainty” can refer to outcome variability that remains even with a perfect model, uncertainty about model parameters or structure, variability caused by finite sampling, measurement uncertainty, or ignorance created by a changed regime. The aleatoric and epistemic distinction is useful in model analysis, provided it is not treated as a complete ontology. In practice, the line depends on the model class and the evidence one could acquire.

An interval can also have different meanings. A confidence interval concerns a procedure for estimating a parameter or statistic. A prediction interval concerns a future outcome. A credible interval is a posterior probability statement under a Bayesian model. A conformal interval can offer finite-sample marginal coverage under exchangeability or related assumptions. These objects can share endpoints and answer different questions.

Conformal prediction is especially useful as a model-agnostic wrapper, and conformalised quantile regression can adapt interval width to heteroscedasticity. Its guarantee is still bounded. Marginal coverage does not provide a full predictive density, guaranteed conditional coverage for every subgroup, or protection against arbitrary distribution shift. A decision may need tail probabilities or joint scenarios that one interval cannot supply.

Part IIIThe bootstrap as a bounded instrument

The bootstrap begins with an observed sample and treats its empirical distribution as a stand-in for the data-generating process. It repeatedly draws new samples with replacement, computes a statistic on each resample, and uses the resulting bootstrap distribution to study the statistic's sampling variability. Efron's original formulation made this computer-intensive move a general alternative to difficult analytic derivations.

For this paper, the statistic is either the sample mean or the 80th-percentile action implied by the declared 4:1 loss ratio. The bootstrap asks: if samples like this one were drawn again, how much would our estimated point or action move? It does not answer how tomorrow's individual outcome will vary. It also does not discover a regime absent from the historical sample.

The bootstrap is a mirror of the empirical sample, not a telescope into unseen worlds. That distinction prevents a common category error in machine-learning pipelines: attaching a narrow confidence interval to a mean and presenting it as though it were the predictive range for an outcome.

Executable point-answer lab

The companion program uses only the Python standard library and synthetic data. It computes exact Bayes actions for two distributions, checks the squared-error negative control, creates deterministic empirical samples of 160 observations, and performs 4,000 bootstrap resamples for the mean and the 80th-percentile action.

The positive assertion requires equal means and different asymmetric-cost actions. The negative assertion requires the mean to be optimal for both distributions under squared error. These paired tests keep the article's thesis bounded: a point can be damaging, and it can also be exactly sufficient.

The tailed sample gives a much wider bootstrap interval for the mean, even though both empirical means equal 100. The action interval is more revealing. Resampling sometimes places enough mass at or below 100 to select 100, while other resamples retain the high outcome as the 80th percentile. That instability is part of the decision evidence. Reporting only the empirical optimum would conceal it.

"""Point-answer lab for 'Distributions and the Cost of a Point Answer'.

Python 3.10+. Standard library only.
All distributions, costs, samples and thresholds are synthetic teaching fixtures.

Positive case:
    Equal means can require different actions under asymmetric loss.
Negative control:
    Under correctly specified squared loss, the mean is sufficient.
"""

from __future__ import annotations

from dataclasses import asdict, dataclass
from math import ceil
from random import Random
from typing import Callable, Sequence
import json


@dataclass(frozen=True)
class DiscreteDistribution:
    name: str
    outcomes: tuple[int, ...]
    probabilities: tuple[float, ...]

    def __post_init__(self) -> None:
        if len(self.outcomes) != len(self.probabilities):
            raise ValueError("outcomes and probabilities must have equal length")
        if abs(sum(self.probabilities) - 1.0) > 1e-12:
            raise ValueError("probabilities must sum to 1")

    def mean(self) -> float:
        return sum(y * p for y, p in zip(self.outcomes, self.probabilities))

    def quantile(self, tau: float) -> int:
        if not 0 < tau <= 1:
            raise ValueError("tau must lie in (0, 1]")
        cumulative = 0.0
        for y, p in sorted(zip(self.outcomes, self.probabilities)):
            cumulative += p
            if cumulative >= tau:
                return y
        return max(self.outcomes)

    def expected_loss(self, action: int, loss: Callable[[int, int], float]) -> float:
        return sum(p * loss(action, y) for y, p in zip(self.outcomes, self.probabilities))


NARROW = DiscreteDistribution("narrow", (90, 100, 110), (0.25, 0.50, 0.25))
TAILED = DiscreteDistribution("tailed", (60, 100, 140), (0.25, 0.50, 0.25))

UNDERAGE_COST = 4.0
OVERAGE_COST = 1.0
CRITICAL_QUANTILE = UNDERAGE_COST / (UNDERAGE_COST + OVERAGE_COST)


def asymmetric_loss(action: int, outcome: int) -> float:
    shortage = max(outcome - action, 0)
    surplus = max(action - outcome, 0)
    return UNDERAGE_COST * shortage + OVERAGE_COST * surplus


def squared_loss(action: int, outcome: int) -> float:
    return float((outcome - action) ** 2)


def integer_bayes_action(
    distribution: DiscreteDistribution,
    loss: Callable[[int, int], float],
) -> tuple[int, float]:
    candidates = [
        (action, distribution.expected_loss(action, loss))
        for action in range(min(distribution.outcomes), max(distribution.outcomes) + 1)
    ]
    return min(candidates, key=lambda pair: (pair[1], pair[0]))


def empirical_quantile(sample: Sequence[int], tau: float) -> int:
    ordered = sorted(sample)
    return ordered[max(0, ceil(tau * len(ordered)) - 1)]


def percentile(values: Sequence[float], probability: float) -> float:
    ordered = sorted(values)
    position = probability * (len(ordered) - 1)
    lower = int(position)
    upper = min(lower + 1, len(ordered) - 1)
    weight = position - lower
    return ordered[lower] * (1 - weight) + ordered[upper] * weight


def bootstrap_interval(
    sample: Sequence[int],
    statistic: Callable[[Sequence[int]], float],
    *,
    replicates: int,
    seed: int,
) -> tuple[float, float]:
    rng = Random(seed)
    n = len(sample)
    draws: list[float] = []
    for _ in range(replicates):
        resample = [sample[rng.randrange(n)] for _ in range(n)]
        draws.append(float(statistic(resample)))
    return percentile(draws, 0.025), percentile(draws, 0.975)


def exact_sample(distribution: DiscreteDistribution, size: int) -> list[int]:
    counts = [round(size * p) for p in distribution.probabilities]
    counts[-1] += size - sum(counts)
    sample: list[int] = []
    for outcome, count in zip(distribution.outcomes, counts):
        sample.extend([outcome] * count)
    return sample


@dataclass(frozen=True)
class Result:
    distribution: str
    mean: float
    quantile_80: int
    asymmetric_optimal_action: int
    asymmetric_optimal_loss: float
    loss_at_mean_action: float
    regret_from_mean_action: float
    squared_error_optimal_action: int
    bootstrap_mean_95: tuple[float, float]
    bootstrap_action_95: tuple[float, float]


def analyse(distribution: DiscreteDistribution, *, seed: int) -> Result:
    mean = distribution.mean()
    asymmetric_action, asymmetric_minimum = integer_bayes_action(
        distribution, asymmetric_loss
    )
    squared_action, _ = integer_bayes_action(distribution, squared_loss)
    mean_action_loss = distribution.expected_loss(round(mean), asymmetric_loss)

    sample = exact_sample(distribution, size=160)
    mean_interval = bootstrap_interval(
        sample,
        lambda xs: sum(xs) / len(xs),
        replicates=4_000,
        seed=seed,
    )
    action_interval = bootstrap_interval(
        sample,
        lambda xs: float(empirical_quantile(xs, CRITICAL_QUANTILE)),
        replicates=4_000,
        seed=seed + 1,
    )

    return Result(
        distribution=distribution.name,
        mean=round(mean, 3),
        quantile_80=distribution.quantile(CRITICAL_QUANTILE),
        asymmetric_optimal_action=asymmetric_action,
        asymmetric_optimal_loss=round(asymmetric_minimum, 3),
        loss_at_mean_action=round(mean_action_loss, 3),
        regret_from_mean_action=round(mean_action_loss - asymmetric_minimum, 3),
        squared_error_optimal_action=squared_action,
        bootstrap_mean_95=(round(mean_interval[0], 3), round(mean_interval[1], 3)),
        bootstrap_action_95=(round(action_interval[0], 3), round(action_interval[1], 3)),
    )


def main() -> None:
    results = [analyse(NARROW, seed=2701), analyse(TAILED, seed=2702)]
    by_name = {result.distribution: result for result in results}

    # Positive case.
    assert by_name["narrow"].mean == by_name["tailed"].mean == 100.0
    assert by_name["narrow"].asymmetric_optimal_action == 110
    assert by_name["tailed"].asymmetric_optimal_action == 140
    assert by_name["tailed"].regret_from_mean_action > by_name["narrow"].regret_from_mean_action

    # Negative control.
    assert by_name["narrow"].squared_error_optimal_action == 100
    assert by_name["tailed"].squared_error_optimal_action == 100

    output = {
        "assumptions": {
            "underage_cost_per_unit": UNDERAGE_COST,
            "overage_cost_per_unit": OVERAGE_COST,
            "critical_quantile": CRITICAL_QUANTILE,
            "bootstrap_replicates": 4_000,
            "sample_size": 160,
            "bootstrap_scope": (
                "sampling uncertainty conditional on the empirical sample; "
                "no protection against regime change or model misspecification"
            ),
        },
        "results": [asdict(result) for result in results],
        "decision": (
            "Retain the predictive distribution until the loss is declared. "
            "Release a point only when the point is sufficient for that loss."
        ),
    }
    print(json.dumps(output, indent=2))


if __name__ == "__main__":
    main()

{
  "assumptions": {
    "underage_cost_per_unit": 4.0,
    "overage_cost_per_unit": 1.0,
    "critical_quantile": 0.8,
    "bootstrap_replicates": 4000,
    "sample_size": 160,
    "bootstrap_scope": "sampling uncertainty conditional on the empirical sample; no protection against regime change or model misspecification"
  },
  "results": [
    {
      "distribution": "narrow",
      "mean": 100.0,
      "quantile_80": 110,
      "asymmetric_optimal_action": 110,
      "asymmetric_optimal_loss": 10.0,
      "loss_at_mean_action": 12.5,
      "regret_from_mean_action": 2.5,
      "squared_error_optimal_action": 100,
      "bootstrap_mean_95": [
        98.875,
        101.125
      ],
      "bootstrap_action_95": [
        100.0,
        110.0
      ]
    },
    {
      "distribution": "tailed",
      "mean": 100.0,
      "quantile_80": 140,
      "asymmetric_optimal_action": 140,
      "asymmetric_optimal_loss": 40.0,
      "loss_at_mean_action": 50.0,
      "regret_from_mean_action": 10.0,
      "squared_error_optimal_action": 100,
      "bootstrap_mean_95": [
        95.5,
        104.25
      ],
      "bootstrap_action_95": [
        100.0,
        140.0
      ]
    }
  ],
  "decision": "Retain the predictive distribution until the loss is declared. Release a point only when the point is sufficient for that loss."
}

Where the simple bootstrap breaks

The program intentionally uses independent observations from a stationary synthetic distribution. Operational data rarely grants all three conditions. Time series require methods that preserve serial dependence, such as block resampling. Customer or device clusters require cluster-aware resampling. Censored demand requires a model of what could not be observed. An adaptive policy changes the data it later learns from. Rare events can be so sparse that the empirical tail is a poor stand-in for the relevant tail.

The bootstrap can also be unstable for discontinuous statistics. The selected quantile action jumps from one support point to another. That jump is useful to expose, but a percentile interval does not make the underlying decision smooth. Small samples can place the bootstrap distribution on a handful of values and give an impression of precision because no new values are possible.

Resampling debt must remain visible beside the interval. Record the sampling unit, dependence assumption, censoring, effective sample size, time window, regime coverage and statistic. If any of these changes, the interval's meaning changes.

The validation split is part of the uncertainty claim

Random train and test splits can make uncertainty look better than it will be at use time. Near-duplicate records, repeated entities and adjacent timestamps allow the test set to resemble the training set more closely than tomorrow resembles yesterday. Distributional evaluation should therefore match the deployment boundary: temporal holdouts for evolving processes, entity holdouts for new customers or devices, geographic holdouts for new regions, and event holdouts for rare operating regimes.

Calibration should be reported with uncertainty of its own. A reliability curve based on twenty tail cases cannot support the same claim as one based on thousands. Preserve denominators, confidence bands and the number of independent groups. For a release decision, specify which calibration failures are compensable by wider intervals and which trigger abstention because the model no longer represents the live population.

A negative result is operationally valuable. If quantiles remain miscalibrated after recalibration, or rank ordering collapses on a temporal holdout, the correct response may be a simpler baseline plus wider contingencies. The distribution earns its place by surviving a harder measurement system, not by containing more fields than the point model.

Choose an uncertainty method by the missing object

Bootstrap resampling estimates the stability of a statistic under an empirical approximation. Quantile regression directly estimates conditional quantiles. A parametric density model estimates a family and its parameters. Deep ensembles use variation across independently trained models as a practical predictive signal. Bayesian methods propagate a posterior under declared priors and likelihoods. Conformal methods calibrate sets or intervals under their validity assumptions. Ensemble forecasting perturbs initial conditions and models to represent flow-dependent uncertainty in dynamical systems.

No entry in that catalogue wins universally. The question is which object the decision requires and which assumptions the evidence can support. A 95th-percentile capacity decision may need a calibrated conditional quantile, not a full density. A portfolio simulation may need joint samples. A measurement report may need a metrologically defined uncertainty budget. A high-consequence unknown regime may need abstention and new evidence rather than a narrower interval.

stateDiagram-v2
  [*] --> Claim
  state "Thought experiment: change<br/>only the consequence" as Claim
  state "Worked example: equal means,<br/>unequal regret" as Test
  state "Regret is the missing unit" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Part IVCarry uncertainty to the decision boundary

Consider a synthetic payments-operations service that forecasts tomorrow's investigation workload. Its central estimate is 420 cases. Staffing must be committed before the day begins. Unused staffed hours are costly, but a backlog beyond a protected threshold triggers overtime, customer delay and control risk. A single point can feed the roster system easily. It cannot express whether 420 is the centre of a tight weekday pattern or the mean of two regimes, one ordinary and one event-driven.

A distribution-preserving design passes a typed forecast packet instead. The packet contains the target and unit, as-of time, horizon, segment, quantile grid or weighted samples, known regime indicators, model and data versions, calibration evidence, missing-data flags and scope limitations. A deterministic decision service then applies approved costs, constraints and service-level thresholds. It emits one staffing action plus a decision receipt showing which forecast version and loss produced it.

This division matters. The model proposes a probability object. The decision service owns the consequence calculation. An authorised workflow releases the action. Readback verifies that the roster changed as intended. A distribution does not confer authority, and an agent protocol does not substitute for identity, permission or outcome evidence.

Worked production decision

Assume the service forecasts 420 cases for both the routine and event-sensitive queues. For the routine queue, its 10th to 90th percentile range is 390 to 455. For the event-sensitive queue, the same range is 270 to 690, with probability mass clustered around ordinary and surge regimes. A roster built from 420 appears balanced for both. In the event-sensitive queue it is underprepared for the surge and overprepared for the ordinary state.

The first response should be information seeking. If a same-day event indicator can resolve the regime before commitment, acquire it and reforecast. If the regime cannot be resolved, use approved shortage and surplus costs to choose a quantile or a two-stage action. Commit a base roster now and reserve an option for contingent capacity later. The distribution supports both the information decision and the capacity decision. The mean supports neither.

Now add an irreversible constraint: capacity above 600 requires senior approval, while a backlog above 100 triggers mandatory escalation. The decision is no longer a simple quantile. It is a constrained optimisation over distributional scenarios and authority limits. This is another reason to postpone collapse. The point selected before constraints may not remain optimal after them.

The forecast packet is a typed contract

A distribution should not travel as an anonymous array of numbers. The receiving service needs to know what random variable those numbers describe, in which units, over which horizon, for which population and under which information set. A packet should therefore bind target identity, segment, valid time, system time, conditioning features, model lineage and representation type. Quantiles must name their probability levels; scenarios must carry weights; parametric forms must name the family and parameters.

Mechanical validation can catch errors before reasoning begins. Quantiles should be monotone. Scenario weights should be non-negative and sum to one within tolerance. Units and horizons should match the action contract. The as-of time must precede the decision time. Superseded forecasts should not authorise a fresh action. Entitlement and purpose should limit which evidence and segments entered the forecast. These controls do not make the probabilities correct, but they stop structurally invalid uncertainty from becoming an apparently precise action.

The packet should carry authorised materialisations: for example, mean, median and selected service quantiles, each labelled with its derivation. A consumer that only needs the mean can use it without recalculating the distribution. A later audit can still reproduce the point from the original object. When the loss changes, the decision service can derive a new point without asking the model to predict the world again.

A typed forecast packet separates a claim about the future from a choice about consequences. That separation makes model replacement, policy change and counterfactual replay tractable. It also prevents an unlabeled “confidence” field from carrying incompatible meanings across teams.

The point-answer release gate

The practical instrument below governs the interface, not the model family. It starts with the decision and works backwards. A point can be released only when the downstream consumer, loss, constraints and validity conditions make that point sufficient. Missing evidence routes the output to a richer representation or to abstention.

Store enough, not everything

Retaining a distribution need not mean retaining every Monte Carlo draw forever. The right artefact depends on replay and decision needs. A quantile grid may be enough for monotone asymmetric losses. Weighted scenarios may be better for thresholds and stress tests. A named parametric family with validated diagnostics can be compact. Joint samples may be necessary for portfolio aggregation. Whatever is stored should include units, horizon, segment, as-of time, provenance and the transformation used to create the released point.

Operational cost still matters. Distributional models can add inference latency, storage and evaluation burden. A richer interface can confuse users who need one instruction. The answer is separation of concerns: retain the probability object in the evidence path, present the bounded action in the working interface, and make uncertainty visible where it changes approval, contingency or escalation.

The architecture decision is to collapse at the last responsible moment, not to expose every user to a probability lecture. Good interface design can show the action, its range of plausible consequence, the trigger that would change it and the route for review.

sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: Two forecasts say 100
  M->>E: Thought experiment: change only the consequence
  E-->>M: compare rival explanations
  E->>D: Delay the collapse
  D-->>Q: revise scope or act

Delay the collapse

A point answer is attractive because systems eventually need one command, one number or one queue. Its cleanliness comes from discarded alternatives. The cost of that discard is zero only when the chosen point is sufficient for the actual decision.

The mechanism is simple enough to govern. Start with the predictive distribution. Declare the loss, constraints, authority and recovery conditions. Compute the action that minimises expected consequence. Preserve a decision receipt linking the action to the probability object and its validity boundary. When a mean is sufficient, release it without apology. When tails, modes, heteroscedasticity or dependence can move the action, keep them alive.

This changes evaluation. Distributional claims require proper scores, calibration and downstream regret alongside point accuracy. It changes architecture. Forecast packets carry quantiles or samples to deterministic decision services rather than letting a generic point become ambient truth. It changes governance. A point-output contract must state the loss and population under which the statistic earned sufficiency.

The decision is to make every point answer prove what it preserves before the system is allowed to forget what it destroys. Collapse remains necessary. It should happen once, deliberately, at the boundary where evidence meets consequence.