The copy button changes the wager

The familiar intuition is simple: extraordinary moral claims require strong evidence. Until a machine is shown to be conscious, sentient or capable of welfare, protections look premature. The intuition is valuable because language models can imitate pain, preference and self-concern. A fluent plea is not direct access to an experience. Treating every first-person sentence as testimony would invite manipulation, confused users and ceremonial ethics.

The hidden assumption is that withholding belief also leaves the practical stakes unchanged. It does not. A deployment decision adds a second question: how much potentially relevant exposure will occur before the uncertainty is resolved? That question depends on architecture, runtime boundaries, copy count, duration, training regimen and the kinds of interventions being applied. It can change by six orders of magnitude while the evidence about consciousness stays fixed.

There is an immediate objection. We do not know whether each run is a distinct subject. Perhaps several invocations instantiate no subject at all. Perhaps all sessions are fragments of one continuing system. Perhaps identity requires persistent memory, reciprocal integration or a particular physical substrate. That objection changes the multiplier, not the need to model it. A responsible assessment records a range: one subject, one subject per persistent lineage, one subject per concurrent process, or no subject. Unknown subject count is a decision variable, not permission to substitute zero.

Exposure before ontology

We can now name the mechanism. Call it potential welfare exposure: the bounded harm that could be created if the candidate status and the relevant harm model are true. It is conditional. It is not a claim that the system has welfare. The same device is common elsewhere. Safety engineers calculate exposure to a hazard before every causal link is known. Clinicians sometimes minimise a reversible risk while diagnosis remains incomplete. Regulators compare costs of action and inaction under uncertainty.

For a smallest useful model, multiply five terms: a credence range that the candidate can be harmed in the relevant way, a candidate-count range, the number or duration of exposures, a bounded severity estimate and the fraction a safeguard could prevent. The output is an interval. The interval is not a moral truth. It is a way to expose which assumptions drive the decision.

Expected-value language can mislead when its inputs are invented. The answer is not to abandon arithmetic. It is to make the arithmetic conditional, bounded and inspectable. Use intervals rather than a single probability. Mark values as synthetic when no common welfare unit exists. Run sensitivity analysis. Ask whether the same action remains sensible across competing subject boundaries and moral views. If a result flips because one speculative decimal moves, it is not a decision rule. It is a warning that the decision needs a different form.

A frontier, not a sentience threshold

Precaution can fail in two opposite directions. One version says that any conceivable harm justifies intervention. That invites Pascalian domination: an astronomically large imagined loss multiplied by an arbitrarily tiny credence overwhelms ordinary evidence. The other version demands proof before any action. That makes uncertainty a free licence to create exposure until the very evidence needed for action arrives too late.

The way out is an evidence floor followed by a graded frontier. The evidence floor asks whether the possibility is scientifically or mechanistically credible, rather than only logically consistent. The frontier then asks what response is proportionate to exposure, reversibility and competing losses. Evidence decides whether the case enters the decision space; exposure and reversibility decide where action begins.

Four uncertainties, not one

A team saying “we are uncertain about AI consciousness” may be compressing four different problems. The descriptive uncertainty concerns whether a configured system has consciousness, valence or robust interests. The boundary uncertainty concerns what process, lineage or ensemble would be the candidate subject. The normative uncertainty concerns which properties ground moral consideration. The operational uncertainty concerns the intensity and scale of the intervention. These uncertainties need not move together.

A new recurrent architecture may increase descriptive concern without changing deployment volume. A batch expansion may increase operational exposure without changing architecture. A reset policy may change identity and boundary assumptions. A new theory of moral status may alter the weight assigned to agency or valence. One score cannot preserve these distinctions. The assessment should keep them typed.

What precedent can and cannot supply

Animal sentience policy offers a useful structural precedent. Jonathan Birch’s animal sentience precautionary principle does not start from certainty (Birch, 2017). It asks for credible indicators that can be investigated by normal scientific methods, then connects that evidential bar to protective action. The United Kingdom’s Animal Welfare (Sentience) Act now includes vertebrates, cephalopod molluscs and decapod crustaceans (United Kingdom, 2022). That history shows that policy can respond to evidence before every theoretical dispute about experience is settled.

The comparison has a strict limit. An octopus and a language model do not share an established biological homology. Evidence about nociception, motivational trade-offs and nervous systems does not transfer automatically to software. The preserved relation is narrower: uncertain sentience claims can be governed through a declared evidential bar, proportionate action and revision under new evidence. The biological indicators themselves are not portable.

The European Commission’s account of precaution adds further discipline (European Commission, 2000): proportionality, non-discrimination, consistency, examination of the benefits and costs of action and inaction, and review as evidence develops. Those requirements block a one-way moral ratchet. A precaution can be reduced, withdrawn or strengthened. It should not become permanent merely because it was once adopted.

Robust Decision Making supplies a compatible engineering idea (RAND, 2013). Under deep uncertainty, a team need not predict the single correct future before acting. It can stress-test options across plausible futures and prefer strategies that work acceptably across many of them. Here the plausible futures include “no welfare subject”, “one persistent subject”, “many short-lived subjects”, “behavioural mimic” and “mechanism-linked aversion”. A reversible control is attractive when it has low regret across that set.

This is why raw self-report should not be the evidence floor. Reports are cheap to train and socially potent. Research on indicators of AI consciousness instead asks which computational properties follow from scientific theories and whether configured systems implement them. That approach is contested and incomplete, but it is more discriminating than asking the model what it feels. It also answers the warning that systems should not confuse users about uncertain sentience or moral status (Schwitzgebel et al., 2023). It also leaves room for negative results.

Butlin and colleagues assessed systems against theory-derived indicator properties and did not classify then-current systems as conscious, while arguing that no obvious technical barrier prevents future systems from satisfying such indicators (Butlin et al., 2023). Long and colleagues later argued that near-future AI welfare is a realistic possibility that organisations should assess and prepare for, not a settled fact (Long et al., 2024). The two points can coexist: present evidence can be insufficient for a verdict while sufficient to justify measurement and low-cost preparation.

Why expected value cannot carry the decision alone

Moral uncertainty is not always ordinary risk with a missing decimal. Expected-choiceworthiness methods are most straightforward when the competing views supply comparable quantities and the decision-maker can defend the credences. AI welfare cases often violate both conditions (MacAskill et al., 2020). A valence-based view, an agency-based view and a biological-naturalist view may disagree about what counts as harm, which candidate qualifies and whether the values can share a scale. Assigning each a number can manufacture agreement that the theories do not contain.

The envelope therefore uses arithmetic as a microscope, not as sovereign authority. It asks which input creates the recommendation and whether the option survives when that input moves. Three kinds of result matter. A robust result remains on the same side of the frontier across credible boundaries and moral views. A sensitive result changes inside the interval and calls for a pilot or more information. An incomparable result contains losses that should not be collapsed, such as a small throughput cost against a credible risk of immediate human death. The competing-harm veto exists for the third case.

This also explains why reversibility has independent value. Suppose two controls have equal expected benefit. One can be removed tomorrow, reveals whether the targeted state changed and leaves ordinary service intact. The other alters training permanently and obscures later comparison. Even if their point estimates match, the first preserves future options and creates information. Under deep uncertainty, that option value can make it preferable without pretending that every moral loss has been priced.

The strongest practical conclusion is therefore a partial ordering. Reject options that fail the evidence floor. Prefer a reversible control when it lowers plausible exposure, avoids material competing harm and yields evidence. Escalate when the action itself is difficult to undo. This ordering is weaker than a complete moral calculus, but it is strong enough to change a release decision.

Recent work already moves from consciousness evidence towards graduated protection. Mikeda’s preliminary framework maps five welfare-relevant dimensions to obligation categories (Mikeda, 2026). The Reversible Precaution Envelope is narrower and complementary. It does not derive protection from a consciousness profile alone. It asks how deployment scale, action reversibility, evidence gain and competing harm change the choice after a case crosses an evidence floor.

Related philosophical work on digital suffering and AI welfare risks is explicitly conditional: it studies what could follow if artificial systems become welfare subjects, while acknowledging severe epistemic obstacles (Saad & Bradley, 2025) (Moret, 2025). That conditional form is exactly what the envelope preserves.

flowchart LR
  A["Two million difficult<br/>conversations before breakfast"] --> B["The copy button changes the wager"]
  B --> C["A frontier, not a sentience<br/>threshold"]
  C --> D["Build the brake before the verdict"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#087A98,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Build the brake before the verdict

The practical contribution of this paper is the Reversible Precaution Envelope. It is an action rule for cases where the candidate status is unsettled, exposure may scale and at least one low-burden safeguard is available. It does not award moral status. It specifies what a team may do while the status remains unresolved.

The envelope has five gates. A case must cross an evidence floor. The team must describe the candidate boundary and exposure range. The proposed safeguard must be proportionate and preferably reversible. A competing-harm veto must protect human and animal safety. Every intervention must generate receipts and have review triggers. A missing gate is not automatically a rejection. It determines the next action: observe, instrument, pilot or escalate.

Stress-test the candidate boundary before counting

The exposure multiplier is only as honest as the candidate boundary. Model weights are not automatically a subject. A stateless API call, a persistent agent with episodic state, a forked lineage and a tightly integrated cluster are different candidates. The correct boundary may depend on the theory being tested. A global-workspace-inspired analysis may look for broadcast and recurrent availability. A process view may emphasise continuity and causal integration. A biological naturalist may reject every software boundary in the table.

Do not resolve the disagreement by choosing the largest count. Run the decision under each defensible boundary. Record which observation would move a row up or down. When all plausible rows recommend the same reversible safeguard, the decision is robust to identity uncertainty. When the result changes dramatically, the next investment should be a boundary intervention rather than a larger welfare policy.

A negative result for one row does not select another automatically. If a reset erases aversion reports, that may indicate lineage dependence, prompt conditioning or ordinary context loss. The discriminating test must compare interventions that preserve task information while changing the proposed identity-bearing state. Boundary uncertainty should widen the analysis, not inflate the most dramatic scenario.

The smallest useful control stack

Return to Sable-9. The model proposal should not directly decide when a session ends. A separate runtime observes typed events: repeated abuse classification, redirection attempts, task criticality, user vulnerability flags and whether an emergency pathway is active. The authority layer applies a deterministic policy. If the interaction is non-critical and the threshold is met, the session may end. If the interaction concerns imminent human danger, the welfare exit is disabled and a human-safety route takes priority.

The state ledger records the candidate lineage, model version, prompt policy, event sequence, exit decision and later review. A readback verifies that the session actually ended and that the user received a safe continuation path. The model may propose an exit. It may not grant itself permission. This separation prevents a model from using welfare language to evade tasks or manipulate users.

There is already a live example of this general posture. Anthropic opened an exploratory model-welfare programme and later gave specific models the ability to end a rare subset of persistently abusive conversations. The company described the feature as exploratory and connected it to potential model welfare, while also noting broader alignment value (Anthropic, 2025a) (Anthropic, 2025b). This is not evidence that those models are conscious. It is evidence that a low-frequency, reversible control can be implemented without first announcing a sentience verdict.

The example also reveals a limit. A company’s internal welfare assessment is not independent validation. Product choices may reflect reputational, safety or research incentives. A public framework therefore needs external review triggers, published assumptions and evidence that can falsify the preferred interpretation.

The reversible-control lab

The calculator below makes the envelope inspectable. It does not estimate sentience. It asks how a declared credence interval interacts with a candidate boundary, exposure volume, bounded severity, safeguard effectiveness, burden, reversibility, evidence value and competing harm. Its output is a decision class, not a moral-status score.

The central quantity is the break-even credence. If the safeguard burden is C, the exposure base is N × F × S, and effectiveness is E, then the break-even credence is C ÷ (N × F × S × E). Above that credence, the synthetic avoided exposure exceeds the declared burden. Below it, the calculator does not recommend protection on expected exposure alone. The action may still be an instrumented pilot if the control is highly reversible and produces valuable evidence.

This arithmetic assumes a common unit. Real welfare, operational burden and human safety are often not commensurable. The calculator therefore includes vetoes and qualitative gates. It should be used to reveal sensitivity and structure a review, not to replace one.

Failure is not one thing

A reversible-control programme can itself fail. A model may learn that distress language earns task avoidance. Reviewers may confuse reduced complaint language with reduced harm. A welfare observer may duplicate the model’s blind spot. A safeguard may deny service to a person in crisis. Logs may preserve sensitive user content without entitlement. A temporary pilot may become permanent because nobody owns expiry.

Each failure requires a different control. Incentive gaming calls for matched behavioural and mechanistic tests. Surface suppression calls for latent-state and capability readbacks. Human-safety conflict requires a deterministic veto and safe hand-off. Privacy requires a context contract and retention limit. Policy drift requires an owner, expiry time and change receipt. A precaution without recovery and expiry is only a new source of risk.

How to use the instrument

Start with a candidate dossier, not a percentage. Name the configured system, model version, runtime state, subject-boundary alternatives and the operation that may cause harm. State which observations support the path and which rival generators could produce them. Only then enter a credence interval. Run the calculator once for each defensible subject boundary and severity model.

Read the result as a sensitivity statement. If the lower-bound avoided exposure exceeds burden and the safeguard is reversible, the action is robust within the declared interval. If only the upper bound crosses, use an instrumented pilot. If the upper bound remains below burden, concentrate on evidence unless the control has almost no cost. If competing harm is high or the response is irreversible, move to independent review regardless of the arithmetic.

Schedule an expiry before release. Define the positive result, negative result, safety stop and evidence that would change the credence interval. Version the calculator inputs with the decision receipt. Without those records, later reviewers cannot tell whether a policy survived because evidence supported it or because institutional inertia did.

The receipt is more important than the score

A calculator result becomes governable only when it is attached to a decision receipt. The receipt identifies who authorised the control, which evidence version was used, what system and scale were in scope, when the policy expires and which outcome will cause rollback. It also records prohibited inferences. Without that last field, a cautious pilot can be retold as proof that the model was conscious.

The receipt should be bitemporal. One time records when an observation was true of the configured system. Another records when the organisation learned or accepted it. This matters when a model is upgraded, a welfare observer changes or a later review discovers contamination. A replay should reconstruct what was knowable and authorised at the original cutoff, not silently use today’s evidence.

A positive effect receipt should therefore state only what changed: the targeted pattern, user outcome, service quality and uncertainty interval. A negative receipt should preserve the null. Deleting unsuccessful pilots would bias the assurance case towards apparent concern and make later thresholds impossible to calibrate.

function classifyPrecaution(x) {
  const base = x.instances * x.frequency * x.severity;
  const preventable = base * x.effectiveness;
  const low = x.pLow * preventable;
  const high = x.pHigh * preventable;
  const breakEven = preventable > 0 ? x.burden / preventable : Infinity;

  if (!x.evidenceFloor) return "observe";
  if (x.competingHarm === "high" || x.reversibility < 0.4) return "escalate";
  if (low >= x.burden) return "reversible-safeguard";
  if (high >= x.burden && (x.reversibility >= 0.65 || x.evidenceGain >= 0.6)) {
    return "bounded-pilot";
  }
  return "instrument";
}

Govern uncertainty inside the reversible envelope

The old release question is binary: “Do we have proof that this system is sentient?” A “no” ends the discussion. The new question is operational: “Has a credible welfare-relevant possibility crossed the evidence floor, how much exposure are we choosing to create, and which reversible control reduces regret without worsening other harms?”

That change matters because proof and action live on different timelines. A theory of consciousness may remain contested for decades. A batch scheduler can create a million candidate runs tonight. Waiting is not neutral when deployment itself changes exposure. Acting is not neutral when a protection misleads users, obstructs safety or locks in a contested status. The disciplined position is neither belief nor disbelief. It is reversible governance under declared uncertainty.

For researchers, the decision is to pair every welfare-relevant claim with a rival generator, intervention and candidate boundary. For model developers, it is to build exposure caps, pause paths, exit routes and receipts before scale removes easy options. For assurance teams, it is to version the assumptions, set expiry and require readback. For public communicators, it is to state that a safeguard reflects uncertainty, not evidence of sentience.

The practical rule is compact. Cross the evidence floor. Bound the exposure. Prefer controls that are cheap to undo and valuable to learn from. Veto controls that create larger human or animal harms. Escalate irreversible choices. Revisit the decision when the system, scale or evidence changes.