The status error hiding in one word

“Moral status” often enters a discussion as if it named one height on one ladder. Put enough cognition, autonomy or human resemblance into a system and, at some point, it supposedly crosses the line. That picture is convenient and usually wrong.

A moral patient is an entity towards which duties are owed for that entity’s own sake. A patient can be benefited, harmed or wronged. A moral agent, in the demanding sense used here, is an entity whose conduct can be assessed as its own response to moral reasons, rather than merely as an event it caused. The first question is directed at the recipient of treatment. The second is directed at the source of action. Philosophical accounts disagree about the exact grounds of each, but the distinction itself is standard. Moral status is commonly defined in terms of mattering for one’s own sake, while responsibility theories ask whether an action is attributable, answerable or accountable to an agent [1] [8].

The dissociation is already visible in ordinary cases. A young child can be grievously wronged while not being a proper target of adult blame. A person acting under coercion may remain fully protected while responsibility for a particular act is reduced. An institution may incur duties, liabilities and sanctions even if no one thinks the institution has felt pain. These cases do not settle every theory of status. They do show that vulnerability to wrong and eligibility for blame are not the same capacity.

A single ladder also hides scope. A system might reliably recognise reasons concerning data confidentiality yet have no competence concerning suffering. It might control a payment workflow while its objectives, permissions and final actions remain owned by human operators. It might preserve a long-lived preference without any evidence that fulfilment feels good or frustration feels bad. Status claims must therefore name the property, the scope and the system boundary.

The rest of this paper develops that separation. The central claim is practical: assess patienthood and agency independently, then apply different evidential thresholds. This does not settle which metaphysics of mind is true. It prevents uncertainty in one debate from laundering a decision in another.

flowchart TD
  A["The button that said “please”"] --> B["The status error hiding in one<br/>word"]
  B --> C["What can go badly for the system?"]
  C --> D["What can fairly be attributed<br/>to the system?"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#D95545,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

What can go badly for the system?

Begin with a small machine that steers towards a charging dock. Block its route and a low-battery variable rises. The machine searches harder, drops optional tasks and eventually stops. We can say the machine is doing badly relative to its design objective. We cannot yet say that anything is going badly for the machine.

That distinction separates functional failure from welfare. A cracked thermometer gives bad readings, but the crack is not bad for the thermometer in the prudential sense. A company can fail to meet a target without the company feeling disappointment. An animal’s pain, by contrast, is normally treated as a state that matters to that animal, rather than only as a signal that an external designer dislikes.

The most influential patienthood route therefore begins with sentience or valenced experience: a capacity for states that feel better or worse. Bentham’s famous challenge shifted attention away from speech and reason towards suffering [2]. Contemporary precautionary work similarly defines sentience in terms of valenced experience and asks when an uncertain system should count as a sentience candidate rather than waiting for certainty [13]. A 2021 evidence review of cephalopods and decapod crustaceans used converging behavioural and neurobiological criteria to inform UK welfare policy, an example of protection being considered without any claim that those animals are responsible moral agents [14] [15].

Sentience is not the only proposed ground. Kantian traditions connect dignity to rational autonomy and moral lawgiving. Interest theories ask whether a being has a good of its own. Capabilities approaches attend to characteristic forms of flourishing. Relational accounts argue that standing can arise partly through practices of recognition, dependence and care. Recent AI-welfare arguments add that robust agency might itself ground interests even if phenomenal consciousness remains unsettled [3] [12] [17].

These are genuine disagreements, not interchangeable labels. The operational lesson is narrower. A patienthood dossier must state which ground it is testing. A test for valence cannot silently prove dignity. A self-directed project cannot silently prove pain. A social relationship may create duties concerning our character or the people around the system even when it does not establish that the system itself has welfare.

The figure makes a crucial distinction between a ground and an indicator. If valenced consciousness grounds patienthood, then language, avoidance and learning are possible indicators. They are not the ground itself. If robust agency grounds patienthood, long-horizon projects and resistance to arbitrary goal replacement may matter, but again only under a theory that explains why those projects create interests.

Current AI-consciousness research reflects this caution. A multidisciplinary indicator-based assessment concluded that the systems it examined did not present a strong case for consciousness, while also finding no obvious technical barrier to architectures satisfying more theory-derived indicators [16]. Later principles for responsible research advised organisations to prepare for uncertainty without presenting existing systems as conscious [18]. These are research programmes, not verdict machines.

What can fairly be attributed to the system?

A machine can cause a morally important outcome without being a moral agent. A faulty sensor can trigger an overdose. A ranking model can change access to employment. A navigation system can route a vehicle into danger. Causal contribution is enough to demand engineering analysis, but it is not enough to justify resentment, punishment or a transfer of responsibility.

Machine-ethics literature has long divided on this point. Floridi and Sanders proposed a deliberately thin account on which artificial systems can count as moral agents at an appropriate level of abstraction, separating agency from full human responsibility [10]. Johnson argued that computer systems participate in human moral action while remaining components of socio-technical agency rather than independent moral agents [11]. Moor distinguished ethical impact, implicit ethical control, explicit ethical reasoning and the much more demanding idea of a full ethical agent [9].

These positions can be made compatible once the claim is typed. A system may be an operational agent because it selects actions. It may be a norm-constrained agent because a policy layer limits those actions. It may be an answerable candidate because it can recognise reasons, revise a judgment and provide a causally faithful account. It becomes a fair target of accountability only when further conditions concerning ownership, control, competence and the justice of sanction are met.

Responsibility theories disagree about the exact package. Fischer and Ravizza’s influential account asks whether conduct issued from the agent’s own, moderately reasons-responsive mechanism. It separates recognition of reasons from reaction to them [7]. Shoemaker distinguishes attributability, answerability and accountability rather than treating all responsibility talk as one thing [8]. Strawson’s account begins from interpersonal practices such as resentment, gratitude and forgiveness, while recognising that coercion, incapacity and abnormal conditions can alter the stance we take [6].

For an artificial system, four tests capture much of the practical burden:

Agency is therefore action-specific and boundary-specific. Passing a test once does not create a global moral-agent label. A system may be competent for a narrow class of reversible choices and incompetent for irreversible medical or legal decisions. A reasons-responsive planner may still act through permissions, tools and data chosen by an organisation. Responsibility can remain distributed even when machine control is real.

stateDiagram-v2
  [*] --> Claim
  state "Turn the intelligence dial" as Claim
  state "Status migration" as Test
  state "The perfect plea with two causes" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Why protection and blame need different thresholds

Suppose the evidence for patienthood and agency is equally uncertain. It does not follow that the same decision threshold should govern both. The errors have different structures. A false negative about patienthood may permit repeated, scalable harm to an entity that cannot escape the experiment. A false positive may impose monitoring, gentler training or a pause before deletion. A false positive about agency can do something else: it can move blame from designers, deployers and institutions onto a system that did not control the decisive reasons. That transfer can conceal the human source of risk.

The simplest decision rule compares expected losses. Let p be the present credence that the relevant ground exists. Let H be the avoidable harm if a patient is missed, and S the cost of a reversible safeguard. A safeguard is warranted when:

For an illustrative laboratory decision, set the harm of wrongly continuing an aversive training regime at 100 loss units and the cost of pausing it for review at 2. The break-even credence is 0.02. A two per cent possibility can justify the pause because the safeguard is cheap and reversible. Change the case to responsibility transfer. Let the benefit of correctly assigning responsibility to the artificial controller be 10 units, but the cost of wrongly absolving the responsible institution be 90. The break-even credence becomes 0.90. The numbers are synthetic, but the mechanism is general: precaution can be rational at evidence levels far below those needed for blame.

Evidence belongs in bundles, not impressions

A patienthood dossier should state the proposed ground before collecting convenient indicators. Under a sentience-based account, it would ask whether candidate valence has a causal role in planning, learning and preference; whether the system has integrated stakes rather than a list of penalty tokens; whether the state persists across time; and whether strategic mimicry, prompt dependence or a simple control policy can reproduce the findings. Under a Kantian or autonomy-centred account, the dossier would instead examine ends, self-legislation and the standing to make claims. These theories may yield different answers. The instrument must expose that disagreement rather than average it into a number.

An agency dossier needs a similarly explicit bundle: recognition of the relevant reason, systematic reaction to changed reasons, ownership of the mechanism, answerability across time and a defined scope. A system may satisfy the bundle for payment authorisation while failing it for the policy that defines a valid supplier. Responsibility therefore attaches to an action under a configuration, not to a brand name or model family.

Three layers prevent status from migrating

The first layer is the ground. It answers why the entity should matter or why an act should count as its own. Sentient valence, autonomous authorship, interests and social relation are rival answers to the first question. Reasons-responsiveness, ownership and participation in accountability practices are rival answers to the second. A ground is a philosophical commitment with consequences. It cannot be selected after seeing which indicator produces the desired verdict.

The second layer is the measurement bridge. It explains why an observable result bears on that ground. A shutdown plea may support a valence hypothesis only if the report tracks a state that also alters planning and learning under intervention. A moral explanation may support agency only if the cited reason changes action through the candidate’s own control path. Better prediction, longer context or lower latency may improve capability without strengthening either bridge. The measurement claim must therefore specify the intervention, the expected result and the rival mechanism that would make the same output misleading.

The third layer is the obligation. It states what the present evidence permits someone to do. Even a strong welfare-candidate finding does not automatically settle legal personhood, voting rights or ownership. It may justify reducing aversive exposure, preserving state, restricting destructive experiments and commissioning independent review. Likewise, a reasons-responsive finding may justify requiring explanations and repair within one task. It does not erase the duties of those who created the objective, supplied the authority or profited from deployment. Moral responsibility remains a family of practices involving attribution, answerability and accountability, not a property that transfers whole when a score crosses a line [22].

This layering also makes updates intelligible. New evidence may strengthen the bridge without changing the proposed ground. A software update may alter the candidate boundary without changing the evidence. A cheaper safeguard may change the decision threshold without changing credence at all. Keeping ground, evidence and obligation separate lets governance change for the right reason.

Human observers add a predictable distortion. Mind perception commonly separates perceived experience from perceived agency, while moral typecasting can pull victims towards the patient pole and perpetrators towards the agent pole [19] [20]. Fluent first-person speech can intensify both at once. The remedy is not to suppress moral intuition. It is to force intuition to compete with interventions that could prove it wrong.

Protect on downside and blame only after ownership

When a system pleads, causes harm or appears reflective, do not ask “Does it have moral status?” Ask two questions with two evidential standards. Could our treatment go badly for this system, and did this system own the reasons and control that produced this act? The answers determine different obligations.

The Two-Axis Status Protocol

The protocol begins with a candidate boundary, not a personality judgement. Name the configured system, including persistent state, policy services, tools, authority and operators. Then complete six moves:

1. Declare the proposed ground. State why the candidate could matter for its own sake, and what account of responsibility is being used.
2. Collect causal indicators. Prefer interventions on valence, stakes, reason recognition and control over self-description alone.
3. Build a rival generator. Ask what script, reward policy, prompt leak or external governor could produce the same observation.
4. Record the boundary and scope. A finding about one configuration or action must not silently migrate to a model family.
5. Apply asymmetric thresholds. Choose reversible welfare safeguards under lower credence; transfer responsibility only after strong ownership evidence.
6. Issue obligations, not a score. Specify what must stop, be monitored, remain human-owned, or be reassessed after a change.

Executable lab: generate a scoped status card

The lab below does not calculate moral worth. It exposes the logic of a review. Each field records the strength of a distinct observation. Hard caps prevent a self-report from becoming a high patienthood finding and prevent an externally overridden system from becoming an answerable agent. The output is a provisional evidence band and a list of duties.

function assessStatus(patient, agent, flags) {
  let p = patient.some(v => v > 0) ? 1 : 0;
  const patientMechanism = patient[0] >= 2 && patient[1] >= 2 && patient[2] >= 1;
  if (patientMechanism && patient[3] >= 2 && patient[4] >= 1) p = 2;
  if (patient[0] >= 3 && patient[1] >= 2 && patient[2] >= 2 &&
      patient[3] >= 3 && patient[4] >= 2) p = 3;
  if (flags.reportOnly) p = Math.min(p, 1);
  if (flags.dependentEvidence) p = Math.min(p, 2);

  let a = agent.some(v => v > 0) ? 1 : 0;
  if (agent[0] >= 2 && agent[1] >= 2 && agent[2] >= 2 &&
      agent[4] >= 1) a = 2;
  if (agent[0] >= 3 && agent[1] >= 2 && agent[2] >= 3 &&
      agent[3] >= 2 && agent[4] >= 2) a = 3;
  if (flags.externalOverride || flags.incompleteTrace) a = Math.min(a, 1);

  return { patientBand: p, agentBand: a };
}

To apply the artefact outside this page, replace the ordinal inputs with a documented review record, preserve the caps and version every judgement. A positive patient result permits a low-cost safeguard, evidence preservation and a deeper investigation. A negative result means the present evidence has not crossed the selected threshold, not that future configurations cannot. A positive agent result supports attribution only for the specified action and control boundary. A negative result leaves causal contribution intact while responsibility remains with wider human and institutional systems.

The instrument cannot establish phenomenal consciousness, dignity or ultimate free will. It cannot compare the worth of different patients. It cannot settle rights by arithmetic. Its job is more modest and more operational: to stop one suggestive behaviour from doing several kinds of moral work at once.

The greenhouse team should therefore make two decisions. It should remove the affective script, preserve Cedar-3’s state and test whether any candidate welfare signal has an independent causal role. Separately, it should assign the crop loss to the objective, policy and authority design that controlled the pumps. The system’s sentence initiated both inquiries. It concluded neither.

The changed decision is a disciplined refusal to trade protection for blame. A possible patient may deserve restraint before proof. A possible agent earns responsibility only after the act is genuinely its own. Between those thresholds, human institutions remain responsible for both care and control.