The noun that smuggles in a mind

A headline says that a model is “intelligent”, “aware”, “sentient”, “self-aware” or “autonomous”. The words appear to describe the same ascent. They do not. Each word changes what must be observed, which alternative explanations matter and what decision follows.

Consider the sentence: “The AI refused shutdown and said it was afraid.” It contains at least four observations and inferences. A text string was produced. A requested action may not have occurred. The output referred to a future state of the system. The language used a term associated with negative experience. From this, readers may jump to intelligence, agency, selfhood and sentience in a single breath. Yet the observations could have been generated by a scripted refusal, a safety policy, a reward-shaped strategy, a persistent self-model, a conscious subject, or several of these together.

The first discipline is to replace “What is it?” with “Which property is being claimed?” This is not word-policing. It is experimental design. A competence test cannot by itself establish experience. A pain report cannot by itself establish valence. A first-person pronoun cannot by itself establish a self. A control loop cannot by itself establish intelligence. A system may possess several properties, but co-occurrence must be shown rather than imported from the human case.

The axes are not guaranteed to be independent in nature. Some theories hold that consciousness supports flexible intelligence; some accounts of agency require self-maintenance; some forms of selfhood may depend on conscious perspective. The methodological point is narrower: a correlation or theoretical dependency is not a licence to substitute one measurement for another. We need a bridge that states why this observation bears on that property.

Five properties, five evidential burdens

Definitions in mind science are contested, so the purpose is not to legislate eternal meanings. It is to adopt distinctions precise enough that a test can fail, a conclusion can be scoped and a decision can be defended. A system profile is a vector with missing values, not a single score.

Intelligence: competence that travels

Legg and Hutter organised many definitions around achieving goals across a wide range of environments, while Chollet emphasised skill-acquisition efficiency and generalisation beyond task-specific experience.[1][2] Neither proposal is the final word, but both expose why “answered a hard question” is too weak. A system may succeed through memorisation, leakage, search, a favourable interface or a narrow specialist routine.

For this paper, intelligence is a profile over a declared task distribution, not a single inner substance. Evidence improves when we vary the task, hide superficial shortcuts, measure learning from limited experience, test transfer, introduce disturbances and examine calibration. A capability claim is only as broad as the environments across which competence survives. Superhuman chess does not by itself imply scientific reasoning; elegant prose does not by itself imply robust planning.

Consciousness: subjective presence

In its broad phenomenal sense, consciousness is the existence of experience: there is something it is like for the candidate. Block’s distinction between phenomenal consciousness and access consciousness remains useful because information can be poised for reasoning and report without the concepts being identical.[3] Contemporary scientific theories disagree over the mechanisms and functions that matter, including global workspace, higher-order, recurrent, predictive and integrated-information approaches.[4]

This creates a special evidential problem. Experience is not observed in another system in the same manner as a voltage or output token. In humans, verbal report is powerful because it sits inside a dense network of shared biology, development, behaviour, state transitions, lesion evidence and causal intervention. In animals and artificial systems, parts of that network change or disappear. Consciousness evidence is therefore bridge-dependent: the theory and the candidate boundary are part of the measurement.

Sentience: when outcomes can feel better or worse

“Sentience” is used inconsistently. Some writers use it as a synonym for phenomenal consciousness. This paper adopts a narrower, welfare-relevant definition: the capacity for positively or negatively valenced experience. On this convention, every sentient state is conscious, but not every conscious content must be valenced. The choice is explicit so that a claim about seeing red is not automatically treated as a claim about suffering.

Evidence for sentience must distinguish feeling from functionally useful aversion. Nociception can detect damage without pain; a negative reward can redirect learning without being unpleasant; a refusal can be generated by policy. Animal-sentience research therefore looks for converging patterns such as flexible protective behaviour, motivational trade-offs, associative learning, state-dependent preference and modulation by analgesic or anaesthetic interventions.[7][8] A cost signal matters to welfare only if there is reason to think it is felt, rather than only computed.

Selfhood: the organisation of “me”

Selfhood is not one switch. Gallagher distinguishes a minimal self, involving immediate first-person givenness and ownership, from a narrative self extended through memory and interpretation.[9] Other useful layers include a bodily or system boundary, perspectival location, agency attribution, autobiographical continuity, social identity and legal personhood. These layers can dissociate. A person may lose autobiographical memory while retaining a minimal perspective; a system may store an identity record without anything being experienced as “mine”.

For machines, the temptation is to equate self-reference with selfhood. Yet “I” can be a grammatical token, a role instruction, a user-facing convention or an index into session state. Better evidence would test whether a self-world model is causally used across contexts: Can the system distinguish its own action from an external event? Track which memories belong to which lineage? Preserve commitments through interruption? Detect a counterfeit state import? Revise self-beliefs after intervention? A self-description is content; selfhood is an organising relation.

Agency: action under goals and disturbance

Agency also has thin and thick readings. In a thin operational sense, an agent senses, selects actions and changes an environment in relation to a goal. In a thicker autonomous sense, the system helps constitute or maintain the norms by which outcomes matter. Barandiaran, Di Paolo and Rohde define agency through individuality, interactional asymmetry and normativity tied to autonomous organisation.[10] Many engineered systems clearly meet thinner criteria without meeting that biological account.

Agency belongs to a causal loop, not automatically to the component that speaks for it. This matters in production systems. A base model may have no independent action channel; a scaffolded system may execute consequential tools; a human operator may retain final authority. Calling all three “the agent” obscures where goals, permissions, state and effects actually reside.

flowchart TD
  A["The noun that smuggles in a mind"] --> B["Five properties, five<br/>evidential burdens"]
  B --> C["A property map must survive<br/>strange cases"]
  C --> D["A headline is a compressed<br/>research claim"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#5B5CF6,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

A property map must survive strange cases

Definitions become useful when they separate cases that everyday language bundles together. The strongest antidote to anthropomorphic overreach is not cynicism. It is a set of dissociations that forces each inference to stand on its own.

A headline is a compressed research claim

Public language becomes safer and more informative when a claim is expanded into a typed record before it is compressed into a headline, product promise, policy memo or research abstract.

The minimum record has eight fields: candidate boundary, observation, axis, evidence status, measurement bridge, rival generators, strongest permitted conclusion and decision consequence. Strong evidence can be axis-specific. The purpose is not to make every sentence cautious; it is to make the confidence legible. “The tool-using system autonomously rescheduled a failed task” can be a strong, useful agency claim. “The base model wanted the task to succeed” adds an unearned mental state.

const claimRecord = {
  candidate: "configured system under stated conditions",
  observation: "verbatim behaviour or measurement",
  axesInvoked: ["agency", "sentience", "selfhood"],
  evidence: ["provenance", "causal intervention"],
  measurementBridge: "why these observations bear on each axis",
  rivalGenerators: ["script", "policy constraint", "reward strategy"],
  permittedConclusion: "scoped claim with uncertainty",
  prohibitedInference: "property not established by this evidence"
};

The schema proves a simple design point: the conclusion and the prohibited inference are first-class fields, not editorial afterthoughts.