Insight Editions · No. 247
The Mind’s I
Fantasies and Reflections on Self and Soul
Douglas R. Hofstadter and Daniel C. Dennett
Twenty-seven stories, dialogues and essays on self and soul with the editors’ reflections, arguing that a mind is an abstract, self-representing pattern not tied to any particular stuff.
A study guide to The Mind’s I by Douglas R. Hofstadter and Daniel C. Dennett. Shared free for personal reading; not for sale and not endorsed by the authors or publishers. Please read and buy the original.
A self is a pattern, not a substance: minds can in principle be copied, moved, spread out or run on other hardware, and the stories that make this seem absurd are intuition pumps to be examined, not proofs.
Personal identity, other minds, the Turing test, animal and machine suffering, genes and memes, levels of description, brains in vats, the Chinese Room, what it is like to be a bat, free will and created beings. 34 sections: introduction, 6 part openers, 27 selections with Reflections.
Editors Douglas Hofstadter and Daniel Dennett; authors Borges, Turing, Lem, Dawkins, Searle, Nagel, Smullyan, Nozick, Harding, Morowitz, Miedaner, Wheelis, Zuboff, Sanford, Leiber, Rucker, Cherniak.
Anyone asking whether AI systems could think or feel, what "I" refers to, and how thought experiments shape (and mislead) our intuitions about minds.
Douglas Hofstadter and Daniel Dennett's The Mind's I (1981) is an anthology of twenty-seven stories, dialogues and essays on self and soul, each followed by the editors' Reflections, arranged in six Parts that move from the sense of self through other minds, biology and software to created selves and the first-person view. Its argument, carried by fiction as much as by philosophy, is that a mind is an abstract, self-representing pattern: real, but not tied to any particular stuff, place or body, so that our deepest intuitions about selves must be tested against cases where those ties come apart. The editors set out "to make the obvious strange and, perhaps, to make the strange obvious", and they argue openly with the authors who disagree.
- Having a body versus being one. Dennett's Introduction teleports an astronaut home from Mars and asks whether she survived; Borges splits from his public name; Harding finds no head on his own shoulders; Morowitz traces explanation round a circle from psychology to physics and back to mind.
- Other minds from the outside. Turing replaces "Can machines think?" with the imitation game and answers nine objections; Hofstadter's coffeehouse dialogue tests the test; Lem's dream-princess and Miedaner's poisoned chimpanzee and whimpering Mark III Beast show how feeling for a creature follows its behaviour.
- From hardware to software. Wheelis's "spirit", Dawkins's replicators and memes, and Hofstadter's Aunt Hillary, a conscious ant colony, climb level by level to minds made of mindless parts; Zuboff's brain, spread neuron by neuron across the galaxy, asks where experience happens.
- Mind as program. Dennett's brain stays in a vat in Houston while his body goes to Tulsa ("Where Am I?"), Sanford moves only the point of view, Leiber's taped minds face body rejection, Rucker's robot is offered immortality as software, and Cherniak's Riddle is an idea that kills whoever understands it.
- Created selves and free will. Lem's personoids and Trurl's kingdom in a box put creators on trial; Smullyan's God explains why he could not make us free but sinless; Borges's magician dreams a man and discovers he is dreamed himself.
- The inner eye. Searle's Chinese Room denies that running a program gives understanding, with six replies and rebuttals; Nagel argues that what it is like to be a bat escapes any objective science; Smullyan's cerebrescope and Hofstadter's Einstein's brain as a book probe first-person authority; Nozick ends by telling the reader that you are a fictional character too.
Section map
| # | Selection | One-line payload |
|---|---|---|
| In | Introduction: Private Lives and Other Minds | Teleporting home from Mars: having versus being a body; the book's plan |
| I | Part I: A Sense of Self | Three landmarks: a split name, no head, mind at both ends of science |
| 1 | Borges and I | The writer and the man things happen to |
| 2 | On Having No Head | Harding's "headless" vision of the first-person view |
| 3 | Rediscovering the Mind | Physics brings the observer's mind back into nature |
| II | Part II: Soul Searching | What could reveal another mind from outside? |
| 4 | Computing Machinery and Intelligence | Turing's imitation game and nine objections |
| 5 | The Turing Test: A Coffeehouse Conversation | The test defended, doubted and tested in dialogue |
| 6 | The Princess Ineffabelle | Love for a simulation, and the price of joining her |
| 7 | The Soul of Martha, a Beast | A speaking chimpanzee poisoned in court |
| 8 | The Soul of the Mark III Beast | A robot beetle nobody can bring herself to kill |
| III | Part III: From Hardware to Software | From biology up the levels to self-designing representations |
| 9 | Spirit | Wheelis's prose poem of life rising to self-reflection |
| 10 | Selfish Genes and Selfish Memes | Replicators build survival machines; ideas replicate too |
| 11 | Prelude . . . Ant Fugue | Aunt Hillary: a mind made of ants; holism and reductionism |
| 12 | The Story of a Brain | Splitting a brain until experience has no place |
| IV | Part IV: Mind as Program | A mind as movable, copyable software |
| 13 | Where Am I? | Dennett's brain in Houston, body in Tulsa |
| 14 | Where Was I? | Moving only the point of view |
| 15 | Beyond Rejection | Taped minds and the rejection of new bodies |
| 16 | Software | Immortality by extraction; a robot's self-symbol |
| 17 | The Riddle of the Universe and Its Solution | An idea that puts its knowers into a coma |
| V | Part V: Created Selves and Free Will | What do creators owe their creations? |
| 18 | The Seventh Sally | Trurl's kingdom in a box is a real crime |
| 19 | Non Serviam | Personoids argue about their creator |
| 20 | Is God a Taoist? | God cannot make us free but sinless |
| 21 | The Circular Ruins | A dreamer discovers he is dreamed |
| VI | Part VI: The Inner Eye | Can the inside of a mind be captured from outside? |
| 22 | Minds, Brains, and Programs | The Chinese Room and six replies |
| 23 | An Unfortunate Dualist | A soulless body that still complains |
| 24 | What Is It Like to Be a Bat? | Subjective character escapes objective description |
| 25 | An Epistemological Nightmare | A brain-reading machine overrules first-person reports |
| 26 | A Conversation with Einstein's Brain | A book that is a brain: is it Einstein? |
| 27 | Fiction | You are a fictional character too |
Hofstadter and Dennett set out (Preface, April 1981) to provoke, disturb and befuddle: "to make the obvious strange and, perhaps, to make the strange obvious." Dennett's Introduction opens with a stranded astronaut who teleports home from Mars and then cannot be sure she survived the trip, and uses her to pry apart having a body, having a brain and being a self. It then frames the book's two great puzzles, consciousness seen from the inside and from the outside, and argues that cognitive science has resurrected the problem of other minds inside every single head. What is needed is not more data but a revolution in our habits of thought, and the fantasies and thought experiments that follow are exercises designed to break those habits.
The Preface: a book built to jolt
The Preface asks the four questions the whole anthology circles: What is the mind? Who am I? Can mere matter think or feel? Where is the soul? The editors do not promise answers. Their aim is to jolt two audiences at once: the hard-nosed believers in a no-nonsense scientific world view and the believers in a religious or spiritual vision of the soul. Neither camp, they think, yet holds the key, and a consensus on the meaning of the word "I" will need radical rethinking of the issues.
The book grew out of conversations in 1980 at the Center for Advanced Study in the Behavioral Sciences in Palo Alto, where Dennett was a Fellow working on artificial intelligence and philosophy (supported by the National Science Foundation and the Alfred P. Sloan Foundation). It was finished while Hofstadter, fresh from the Pulitzer for Gödel, Escher, Bach, was a Guggenheim Fellow doing AI research at Stanford. The division of labour shows throughout: every selection is followed by Reflections signed D.R.H. or D.C.D., and the Introduction itself is Dennett's.
Douglas R. Hofstadter: computer scientist and author of Gödel, Escher, Bach (1979); writes the Reflections signed D.R.H., plus three selections of his own.
Daniel C. Dennett: philosopher of mind (Brainstorms, 1978); writes the Reflections signed D.C.D., the Introduction and the selection "Where Am I?".
Sarah's mother: the Introduction's stranded Mars astronaut, who teleports home and then doubts she is who she remembers being.
Sarah: her daughter, about eight and a half, whose joyous assumption that "Mother has come home" becomes a moral problem.
E. R. John: neurophysiologist whose objective definition of consciousness Dennett uses as a test case.
Karl Lashley: psychologist who claimed that "no activity of the mind is ever conscious".
James Lackner and Merrill Garrett: psycholinguists whose dichotic-listening experiment revealed unconscious sentence comprehension.
The Teleclone: home from Mars, or dead on Mars?
You watch two moons rise from opposite horizons: you are on Mars, your spaceship broken beyond repair, protected from the killing cold only by fragile membranes of technology. In the communication compartment you find a Teleclone Mark IV teleporter. Step into the chamber and it will dismantle your body swiftly and painlessly, producing a molecule-by-molecule blueprint beamed at the speed of light to a receiver on Earth, whose well-stocked atom reservoirs will build, from the instructions, you. With nothing to lose you flip the switches: 5, 4, 3, 2, 1, FLASH, and you step out into the sunny air of Earth.
Thought experiment: the Teleclone Mark IV. Setup: (1) your body on Mars is scanned and destroyed; (2) the information travels to Earth; (3) a new body is assembled from local atoms, with all your memories. The intuition it pumps: at first, "I've come home"; then, on reflection, "maybe I am a brand-new human being, only several hours old, and Sarah's mother died on Mars." What it shows: our concept of personal identity rests on two things that normally travel together, physical continuity and psychological continuity (memories, character, plans), and the machine pulls them apart. Dennett leaves the verdict open; the book returns to it in Parts IV and V.
The celebration brings the first crack. Your daughter Sarah is now about eight and a half, almost three years older, taller, wiser. You do not so much recognise her as extrapolate from memory and deduce her identity; most of the cells in her body were not there when you left, yet she is "still the same little person" you kissed goodbye. Then it hits you: am I the same person who kissed her? Did I die on Mars? "No, certainly I did not die on Mars, since I am alive on Earth." Perhaps someone died on Mars, Sarah's mother; "then I am not Sarah's mother. But I must be!" Is the machine a tele-porter, a mode of transport, or, as the brand name hints, "a sort of murdering twinmaker"?
The narrator tries to settle it from the inside. Sarah's mother entered the chamber with hope, not suicidal resignation; her act was both altruistic (giving Sarah back her mother) and selfish (escaping a jam). How do I know how it seemed to her? "Because I was there; I was Sarah's mother thinking those thoughts; I am Sarah's mother. Or so it seems." The days that follow bring relief and joy balanced by soul searching, and a feeling of being an impostor: what will Sarah think when she figures out what really happened, as she once figured out about Santa Claus?
Having a body, having a brain
So, with more than idle curiosity, you pick up The Mind's I. "Here I am reading page 5 of this book; I see my hands." How do I know they are my hands? They are fastened to my body, and I control it. Do I own it? In a sense: I may not legally sell it while alive, but I may bequeath it to a medical school. If I have this body, then I seem to be something other than this body; "I own my body" does not mean "this body owns itself", which is probably meaningless (though does the moon own itself, since nobody else does?). I and my body are "intimately connected and yet distinct. I am the controller, it is the controlled. Most of the time."
The book next asks whether you could exchange your body for a stronger or more beautiful one, and insists it is perfectly imaginable, hence possible in principle. Reincarnation and the transmigration of souls are one version, though their mechanics are "always left in the dark"; a brain transplant is a clearer one, and if your brain went into another body, it seems you would go with it. That raises the harder test. Say the two sentences aloud and see which rings true:
"I have a brain." Then there is a you distinct from the brain, and in principle you could trade it in for another, as you might trade a body.
"I am a brain." Then you always travel with your brain in a body switch, and a "brain switch" makes no sense; yet the Mars traveller left her old brain behind and still feels like herself.
Dennett then asks how you even know you have a brain. You have never seen it, cannot see it in a mirror, cannot feel it. You know it as you know you have a liver: you are human, all humans have one, and trusted books say so. For centuries people did not know what livers were for, nor brains: Aristotle reportedly thought the brain an organ for cooling the blood (which, to be fair, it does). Dennett's sharpest twist: suppose our livers were in our skulls and our brains in our ribcages. Would we find it plausible that we think with our livers? Your thinking seems to happen behind your eyes and between your ears, but is that because your brain is there, or because you locate yourself roughly at the place you see from? It is just as mind-boggling to imagine thinking with "those soft grayish cauliflower-shaped things" as with "those soft reddish-brown liver-shaped things".
Try this: the transposed organs. Imagine a species whose brain sits in the belly and whose eyes and ears sit on a stalk on top. Where would its members say they "are"? If they say "up on the stalk", where they see from, the felt location of the self is a point of view, not an organ. Dennett's own "Where Am I?" (Selection 13) runs exactly this test by separating brain and body by hundreds of miles.
Souls that flourish in the garden of science
Many people think the idea that you are more than a living body, a soul or spirit, is unscientific: science has banished leprechauns and ghosts, and the "ghost in the machine" (Ryle's sneer at Cartesian dualism) "will itself soon give up the ghost". Dennett replies that not all versions of the idea are so vulnerable; "some versions, as we shall see, actually flourish in the garden of science."
His evidence is a catalogue of ordinary things that are neither ghostly nor simply constructed out of the building blocks of physics. Do you believe in voices? In haircuts? What, in the physicist's language, is a hole in a piece of cheese? Where in space and time does "The Star-Spangled Banner" exist? Not merely as ink in the Library of Congress: destroy that paper and the anthem survives. Latin still exists though no longer living; the language of the cave people of France exists no more. The game of bridge, less than a hundred years old, is "not animal, vegetable or mineral".
Abstract-but-historical objects. These things have no mass or chemical composition, yet they are not purely abstract like the number π, immutable and nowhere. They have birthplaces and histories; they change, things happen to them, and they move about, as a species, a disease or an epidemic moves. To assume that everything worth taking seriously is a collection of particles is "a lack of scientific imagination, not hard-headed sophistication". "One doesn't have to believe in ghosts to believe in selves that have an identity that transcends any particular living body."
Back to Mars. Consider the improved Teleclone Mark V, which uses noninvasive CAT-scanning to obtain its blueprint without destroying the original. Sarah's mother might still press the button, for Sarah's sake and to get the story home through "an eloquent spokeswoman", but she would expect to step out still on Mars. "Could someone, some one, literally be in two places at once?" Not for long: the two would soon accumulate different memories and different lives, and be "as distinct as any two people could be".
Critical lens. The Mark IV/Mark V pairing is the core of Derek Parfit's argument in Reasons and Persons (1984), which appeared three years later: Parfit concludes that identity is not what matters, only psychological connectedness and continuity, so teleportation is "about as good as ordinary survival". Opponents (Bernard Williams, and later "animalists" such as Eric Olson) hold that the Mark IV kills you. Dennett's Introduction poses the case without choosing; the book's later selections lean towards the pattern view of the self, which is a substantive and contested commitment, not a neutral result.
Private lives: consciousness from the inside
The next section asks what makes you you, and where your boundaries lie. Part of the answer seems obvious: you are a centre of consciousness. Consciousness is "both the most obvious and the most mysterious feature of our minds". Nothing is more certain to each of us than being a subject of experience, an enjoyer of perceptions, a sufferer of pain, an entertainer of ideas, a deliberator. Yet magnetism, photosynthesis and digestion are equally accessible to any observer with the right apparatus, whereas each case of consciousness seems to have a favoured or privileged observer whose access beats anyone else's. So there is no good theory of consciousness, no agreement on what one would look like, and some deny the term names anything at all.
That such a familiar thing has resisted characterisation suggests our conception of it is at fault. Dennett lists the baffling questions: are other animals conscious, and in our way; could a computer or robot be; can there be unconscious thoughts, pains or perceptions; is a baby conscious at or before birth; are we conscious when we dream; might one brain harbour more than one conscious subject? Empirical findings will matter, but for each we must ask what its bearing on consciousness is, and why. These are conceptual questions, and thought experiments are the tool.
From the inside, consciousness seems all-or-nothing, "an inner light that is either on or off". We grant drowsiness, inattention and sleep, but that we are conscious seems not to admit of degrees. This perspective sunders the universe into two kinds of thing: subjects, beings "it is like something to be", and the rest. It is not like anything to be a brick, a pocket calculator or an apple: they have insides, but not the right sort, "no inner life, no point of view". It is certainly like something to be me, almost certainly to be you (you have told me so), probably to be a dog or a dolphin, "and maybe even like something to be a spider."
Other minds: consciousness from the outside
Other creatures we meet from the outside. They react to events within their senses, recognise things, avoid painful experiences, learn, plan, solve problems. But the very vocabulary ("senses", "painful") prejudges the issue: describe a robot that way and the polemical intent would be obvious. So we lean on biological similarity to us, the paradigm conscious beings, and our intuitions about which similarities count are untrustworthy.
| Creature | Feature that sways us | Dennett's point |
|---|---|---|
| Dolphin | its "fishiness" | lowers our confidence, though it should not |
| Chimpanzee | a face like ours | would keep it in "the charmed circle" even if it were as dull as a sea slug |
| Housefly | small, cold-blooded | if it were our size or warm-blooded, we would be sure pulling off its wings hurt |
| Clever robot | talks about its "inner life" | might be admitted, but how could we tell we were not fooled? |
The obvious answer is that outside indicators are more or less reliable symptoms of the whatever-it-is each subject knows from inside. But confirming that requires the impossible: checking the coincidence of inner and outer in someone else. Their testimony only adds "more coincidence of outer with outer". This is the notorious problem of other minds, and the robot case makes it look unanswerable: is "that special inner light really turned on, or is there nothing but darkness inside?" Dennett notes that his own unworried use of "we" in these paragraphs, and the reader's acceptance of it, shows that nobody takes the problem seriously for ordinary humans. Perhaps, then, "we have taken a misstep already."
Two anchors of the ordinary concept of consciousness
- From the inside: obvious, pervasive, intimately known; determines "what it is like to be me"; seems all-or-nothing, an inner light on or off; divides the world into subjects and objects.
- From the outside: inferred from behaviour (reacting, recognising, avoiding pain, learning, planning) and from biological similarity to us; a matter of degree; open to deception by a clever enough robot.
E. R. John's objective definition
Some theorists expect that better theories of brain organisation will let us sort conscious from nonconscious entities by outside facts. Neurophysiologist E. R. John offered a definition: consciousness is a process in which information from multiple sensory modalities is combined into a unified multidimensional representation of the system and its environment, integrated with memories and the organism's needs, generating emotions and programs of behaviour that adjust the organism to its world. Establishing that this occurs in a creature would be hard but empirical; and once it was established, "we will not find any room to wonder further". Doubting would be like inspecting a car engine in detail and asking, "But is it really an internal combustion engine?"
Any proper science of consciousness, Dennett grants, must take this "somewhat doctrinaire step" of treating the phenomenon as objectively accessible. Yet one may still wonder whether, once the step is taken, the truly mysterious phenomenon is left behind. Before dismissing that hunch as romantic, consider a revolution in thinking about the mind.
What changed by 2026. John's definition reads like a sketch of two theories that now dominate the science: global workspace theory (Baars; Dehaene's global neuronal workspace), in which content becomes conscious when it is broadcast system-wide, and integrated information theory (Tononi), which measures how unified a system's causal structure is. In 2023 the COGITATE adversarial collaboration, pre-registered by proponents of both, presented results that challenged key predictions of each; the full paper appeared in Nature in April 2025. The same year, 124 researchers signed a letter calling IIT pseudoscience, a sign that Dennett's worry (we lack agreement on what a theory would even look like) is still live.
Freud's crutch and its removal
For John Locke and his successors nothing was more essential to the mind than consciousness, especially self-consciousness. The mind was transparent to itself; to discover what went on in it one just "looked", introspected, and the limits of what one found were the boundaries of the mind. Unconscious thinking was dismissed as self-contradictory nonsense, and Locke even struggled to explain how memories could be "in" the mind while not "present to consciousness". So when Freud proposed unconscious mental processes he met stark denial and incomprehension. He won converts because the "conceptual impossibility" explained otherwise inexplicable psychopathology.
Freud's crutch. The new view kept a pale version of the Lockean creed by assigning unconscious thoughts to other selves within the psyche: as I can keep secrets from you, my id can keep secrets from my ego. Splitting the subject into many subjects preserved the axiom that every mental state is someone's conscious state, with the weird question of "whether it was like anything to be a superego" kept at bay "in the mists of jargon".
Freud's expansion of the thinkable revolutionised clinical psychology and paved the way for cognitive experimental psychology. We now accept without a twinge that hypothesis testing, memory searching and inference, in short information processing, go on in us entirely inaccessible to introspection. This is not repressed activity driven out of sight; it is simply beneath or beyond the ken of consciousness. As Freud overruled his patients' sincere denials, the cognitive psychologist shows people performing sophisticated reasoning of which they can give no account: "some mental activities are more accessible to outsiders than to the very 'owners' of those minds."
In the new theorizing the crutch has been thrown away. The metaphors survive (subsystems like little people sending messages, asking for help, obeying, volunteering), but the actual subsystems are deemed nonconscious organic machinery, "as utterly lacking in a point of view or inner life as a kidney or kneecap". The advent of "mindless" but "intelligent" computers sped this dissolution. Now Locke is turned on his head: we are losing our grip on conscious mentality. If subjectless information processing can achieve all the ends conscious minds were for, the theories could be true of zombies or robots and have no way of distinguishing us from them.
Lashley's hexameter and the return of other minds
Karl Lashley suggested provocatively that "no activity of the mind is ever conscious". Asked to think a thought in dactylic hexameter, anyone who knows the rhythm can oblige ("How in the world did this case of dactylic hexameter come to me?"), but how we do it is inaccessible. Far from killing consciousness as a topic, the remark highlights the difference between all the unconscious processing and the conscious thought itself, which is somehow directly accessible. Accessible to what? Saying "to some subsystem" does not distinguish it from unconscious events, which are also accessible to subsystems. If some special subsystem's traffic with the rest makes it the case that there is "one more self in the world", one more thing it is like something to be, that is far from obvious.
The old chestnut is back: the problem of other minds, now inside one head. The split-brain cases show it most graphically. After the corpus callosum is cut, it is unproblematic to say the patient has two somewhat independent minds, one per hemisphere, since we already think of a mind as an organisation of communicating subminds whose lines have been cut. What remains problematic is whether both have an inner life.
Deny the minor hemisphere consciousness: it merely processes information and controls behaviour, as unconscious subsystems do. But then what reason is there to grant consciousness to the dominant hemisphere, or even to the whole intact brain?
Grant it consciousness: then the "newly discovered person whose brain is the non-dominant hemisphere" is real, and what of all the other information-processing subsystems? Do we populate our heads with hosts of subjects and hand Freud his crutch back?
The unattended channel (Lackner and Garrett). In dichotic listening, subjects hear two channels through earphones and attend to one. They can report the attended message but of the other only that a voice spoke, perhaps male or female, perhaps in their native tongue. In the attended ear: the ambiguous "He put out the lantern to signal the attack." In the unattended ear, one group heard a disambiguating sentence ("He extinguished the lantern"), a control group something irrelevant. The first group could not report the unattended sentence, yet favoured the suggested reading significantly more. What it shows: the ignored sentence was processed all the way to meaning. Was that unconscious comprehension, or evidence of two only partly communicating consciousnesses, one of which "consciously comprehended the sentence and relayed a hint" to the one that answers our questions? Asked what it was like, subjects sincerely say: not like anything.
Dennett concludes that we are back at an unanswerable question, which suggests we need different ways of looking. A view of consciousness that does justice to these complications will demand a revolution in our habits of thought, and the selections are games and exercises to help break bad habits.
What changed by 2026. Split-brain research has grown more puzzling, not less. In 2017 Yair Pinto and colleagues reported in Brain that two split-brain patients could respond to stimuli anywhere in the visual field with either hand or verbally, suggesting split perception with undivided consciousness; other studies still find classic dissociations, and the issue is unresolved. The Lackner and Garrett finding fits today's broad consensus that semantic processing can occur outside awareness, though the size of such effects is debated after psychology's replication crisis. For anyone building AI systems the Lockean reversal is daily life: large models perform elaborate "reasoning" with no introspective access to how, and their self-explanations, like the patients', can be fluent and wrong.
The road map: six Parts and three roadblocks
The book's itinerary, as the Introduction sets it out
- Part I, A Sense of Self: swift forays into the territory, noting landmarks but mounting no campaigns.
- Part II, Soul Searching: the mind's I surveyed from the outside: what reveals other minds and souls to a searcher?
- Part III, From Hardware to Software: the biological foundation of mind, then up several levels to internal representations; the mind emerges as a self-designing system of representations embodied in the brain. First roadblock: "The Story of a Brain".
- Part IV, Mind as Program: mind as software, independent of any physical embodiment; prospects of transmigration and Fountains of Youth.
- Part V, Created Selves and Free Will: traditional metaphysical problems in untraditional costumes; reality challenged by dreams, fictions, simulations, illusions; free will under an unusual spotlight. Second roadblock: Searle's "Minds, Brains, and Programs".
- Part VI, The Inner Eye: past the third roadblock, Nagel's "What Is It Like to Be a Bat?", into the inner sanctum, relocating the self in the metaphysical and physical world.
Critical lens. (1) The Introduction is not neutral: by calling Zuboff, Searle and Nagel "roadblocks" to be got around, the editors announce in advance that the functionalist, pattern-based view of mind is the road. Readers should judge the roadblocks on their merits. (2) Dennett's later work hardened the line sketched here: Consciousness Explained (1991) argued against a single "Cartesian theatre" where everything comes together, and he came to call the felt inner light a "user illusion"; critics such as David Chalmers (the "hard problem", 1995) say this explains away rather than explains. Dennett died in April 2024, aged 82, without that dispute settled.
Three swift forays into the territory, each from a different direction: a writer split from his public name, a mystic who finds no head on his shoulders, and a biophysicist who traces explanation round a circle from mind to physics and back to mind. The Part mounts no campaigns; it plants landmarks that the rest of the book keeps returning to.
Part I sets up the book's central tension in miniature: the self seen from the inside against the self seen from the outside. Borges finds his first-person "I" and the third-person "Borges" drifting apart, so that he cannot tell which of them wrote the page. Harding pushes the first-person view to its limit: from where he stands, he has no head, only a void that is filled by the world, while everyone else has a head. Morowitz goes to the other extreme of objectivity, the chain of scientific reductions from psychology to biology to chemistry to physics, and finds that the chain closes on itself, because quantum physics (on one reading) needs a mind to complete a measurement.
The editors' Reflections bind the three together. Dennett's "Pete" in the department store, who watches a man on the security monitor have his pocket picked before realising the man is himself, shows that thinking about yourself and thinking about yourself as yourself are different acts. Hofstadter answers Harding's headlessness with the syllogism of mortality, the moment the abstract category "human being" folds back to engulf the one who formed it. And he answers Morowitz with Everett's many worlds, which turns the quantum measurement problem into a problem of personal identity: why does this "me" feel itself to be travelling down this one branch?
Part I at a glance
- Selection 1, "Borges and I": the private self and the public persona; first person against third person.
- Selection 2, "On Having No Head": the first-person view taken literally; the subject is a void filled by the world.
- Selection 3, "Rediscovering the Mind": the circle of explanation from mind through physics back to mind; quantum measurement and the observer.
Jorge Luis Borges's one-page prose piece (from Labyrinths, in James Irby's translation) is a confession by a man who feels that the famous writer "Borges" is someone else, the one "things happen to". The narrator gives his life over to the other, who falsifies and magnifies everything, and ends unable to say which of them wrote the page. The piece dramatises the gap between knowing yourself as "I" and knowing yourself as a name, a reputation, a third person. The Reflections turn the literary mood into a precise philosophical point about first-person thought, using a pickpocket, a self-describing book and a robot.
The selection: two Borgeses
The narrator walks the streets of Buenos Aires, stopping, "perhaps mechanically now", to look at an entrance arch and a gate's grillwork. He knows of Borges from the mail and from lists of professors and biographical dictionaries. They share tastes: hourglasses, maps, eighteenth-century typography, the taste of coffee and the prose of Stevenson. But Borges shares them "in a vain way that turns them into the attributes of an actor". The relationship is not hostile: "I live, let myself go on living, so that Borges may contrive his literature, and this literature justifies me."
The pages Borges has achieved cannot save the narrator, "perhaps because what is good belongs to no one, not even to him, but rather to the language and to tradition". The narrator is destined to perish, and only "some instant of myself" can survive in the other. Little by little he gives everything over, aware of Borges's "perverse custom of falsifying and magnifying things". He invokes Spinoza: all things long to persist in their being, "the stone eternally wants to be a stone, and the tiger a tiger". He will remain in Borges, not in himself ("if it is true that I am someone"), yet he recognises himself less in Borges's books than in many others, or in "the laborious strumming of a guitar".
The flight that fails. Years ago the narrator tried to escape by changing subjects, moving from "the mythologies of the suburbs" (Borges's early stories of Buenos Aires knife-fighters) to "the games with time and infinity" (the later labyrinths and libraries). But those games now belong to Borges too, so he will have to imagine other things. What it shows: every act of the private self, once made public, is captured by the persona. "Thus my life is a flight and I lose everything and everything belongs to oblivion, or to him."
I do not know which of us has written this page.
The last line is the trapdoor. The page is a public text by the famous author, so "Borges" wrote it; but it is a private confession in the first person, so "I" wrote it. The reader cannot settle it either, which is the point.
Reflections (D.C.D. And D.R.H.)
The Reflections, signed by both editors, begin with the "curious effect" of international fame: Borges seems to himself to be two people, the public personage and the private person. Fame magnifies the feeling, but we all share it. You read your name on a list, see a candid photograph, or overhear people talking about someone and suddenly realise it is you; the mind must leap from third person ("he", "she") to first person ("I"). Comedians exaggerate the leap in the classic double-take: Bob Hope reads in the paper that Bob Hope is wanted by the police, comments casually, then jumps up: "That's me!"
Robert Burns may be right that seeing ourselves as others see us is a gift, but it is not a condition we could or should hold at all times. Several philosophers, the editors note, have recently given "brilliant arguments" that there are two fundamentally and irreducibly different ways of thinking of ourselves. The arguments are technical; the examples are vivid.
Thought experiment: Pete and the security monitor. Setup: (1) Pete queues to pay in a department store under a closed-circuit television monitor; (2) on screen he sees a man in an overcoat with a large paper bag having his pocket picked by the man behind him; (3) he raises his hand to his mouth in astonishment and sees the victim's hand do the same; (4) he realises he is the victim. The intuition: something genuinely new was learned at step 4, though all along Pete was thinking about a man who was in fact himself. What it shows: before the shift he was thinking about himself but not "as himself", not "in the right way". Only the first-person thought galvanises him into defensive action; without the capacity for such thoughts "he would hardly be capable of action at all".
The second example is a self-referential trap. Imagine someone reading a book in which a descriptive noun phrase of about three dozen words portrays an unnamed person of indeterminate sex doing something everyday. The reader forms a vague mental image, which sharpens as details are added, until something "clicks" and the reader realises with an eerie, sheepish, tickled feeling that he or she is the person being described. Now suppose the book was The Mind's I. "There now, doesn't your mental image of the whole scenario come into a little sharper focus?" What page was the reader on? What might he or she be doing right now? The paragraph has just done to you what it describes.
Try this: the dossier. A colleague hands you an anonymised profile for a risk review: "a forty-something architect, frequent traveller, writes long technical reports, drinks too much coffee". You assess the person coldly for two minutes before seeing your own employee number in the footer. Notice what changes at that instant: not one fact about the person, only which facts now bear on what you do next. That shift is the whole of the first-person "click".
The editors close with a robot. Shakey, the famous robot at SRI International in California, was steered by a computer over radio links. The computer held an internal representation of the robot and its surroundings, updated as the robot moved, so it could control the robot's "body" with current information. Suppose the representation places the robot in the middle of an empty room, and you are asked to "translate into English" the computer's state. Should it read "It (or he, or Shakey) is in the centre of an empty room" or "I am in the centre of an empty room"? The question, they promise, resurfaces in Part IV.
Third-person translation: "Shakey is in the centre of the room." Accurate, but it treats the representation as a description from outside, the way Pete first saw the man in the overcoat.
First-person translation: "I am in the centre of the room." Fits how the representation is used: it is the one model the system consults to move its own body, which is what makes a thought "self-locating".
Critical lens. The "brilliant arguments" the editors allude to are John Perry's "The Problem of the Essential Indexical" (1979), with its shopper following a trail of sugar round a supermarket until he realises the leaking bag is his own, and David Lewis's "Attitudes De Dicto and De Se" (1979); Hector-Neri Castañeda had raised the issue in the 1960s. Pete is a close cousin of Perry's shopper. Whether the first-person element is a special kind of content (Lewis) or merely a special way of grasping ordinary content (Perry) is still argued.
What changed by 2026. The Shakey question is now practical. Large language model agents are routinely given a system prompt describing "the assistant" in the third person and then asked to act; developers debate whether to write "You are..." or "The assistant is...", and models trained on human text readily slip into first-person self-description. Whether that first person marks a genuine self-model or a stylistic habit is exactly the line between Pete before and after the click. In enterprise architecture terms, a system that holds a model of itself (its own state, limits and role) and uses it to steer behaviour is doing something structurally first-person, whatever the English says.
Douglas E. Harding's essay (from his short book On Having No Head, first published by the Buddhist Society in London) reports the day, walking in the Himalayas at thirty-three, when he stopped thinking, looked, and found no head on his shoulders: only a void that held the whole world. He then defends the "vision" against common-sense objections about noses, punches and eyes, recruits an honest observer, a camera and the cinema to his side, and ends with a mystical claim that true seeing is eyeless. Taken literally the essay is absurd; taken as a description of the first-person point of view it is exact, and Hofstadter's Reflections use it to ask how any of us ever comes to believe "I am a human being, therefore I will die".
D. E. Harding: English architect and philosopher (1909–2007) who spent his life teaching the "headless way".
The honest sceptic: an imagined observer invited to walk towards Harding and report only what he sees.
The camera: a witness "lacking memory and anticipation", which records only what is where it stands.
Film directors: "practical people" who already know that a first-person shot needs a headless actor.
The selection: a rebirthday in the Himalayas
"The best day of my life, my rebirthday, so to speak, was when I found I had no head." Harding insists this is not a literary gambit: "I mean it in all seriousness: I have no head." It happened about eighteen years before he wrote, when he was thirty-three, after months absorbed in the question "What am I?" He was walking in the Himalayas on a still, clear day, looking from a ridge over misty blue valleys to the highest range on Earth, with Kangchenjunga and Everest "unprominent among its snow peaks". The setting probably had little to do with it, he says, though unusual states of mind are said to come easily there.
What happened was "absurdly simple and unspectacular: I stopped thinking." An odd alert limpness came over him; reason, imagination and mental chatter died down; past and future dropped away. He forgot his name, his manhood, his animalhood, "all that could be called mine", as if born that instant, brand new, mindless, innocent of all memories. There was only the Now. "To look was enough." And what he found was khaki trouser legs ending downwards in brown shoes, khaki sleeves ending sideways in pink hands, and a khaki shirtfront ending upwards in "absolutely nothing whatever! Certainly not in a head."
A nothing that holds everything. The hole where a head should be was "no ordinary vacancy, no mere nothing". It was "a nothing that found room for everything": grass, trees, shadowy hills, and snow peaks "like a row of angular clouds riding the blue sky". "I had lost a head and gained a world." The scene shone "alone and unsupported", "utterly free of 'me', unsustained by any observer". "Its total presence was my total absence, body and soul."
The vision felt like waking, not dreaming: "self-luminous reality for once swept clean of all obscuring mind", "the revelation, at long last, of the perfectly obvious", "naked, uncritical attention to what had all along been staring me in the face, my utter facelessness". It raised no questions, only peace, quiet joy and "the sensation of having dropped an intolerable burden".
Describing the vision afterwards
As the wonder wore off, Harding began describing it to himself. He had vaguely thought of himself as inhabiting a house, his body, looking out through two round windows. Now he finds nothing to tell him how many eyes he has here, "two, or three, or hundreds, or none". On this side of the façade there is one window, "wide open and frameless, with nobody looking out of it". "It is always the other fellow who has eyes and a face to frame them; never this one."
So there are two species of man. The first, of which he observes countless specimens, carries a head ("a hairy eight-inch ball with various holes in it"); the second, of which he observes only one specimen, carries no such thing. Till now he had overlooked this "considerable difference", a victim of "a lifelong hallucination", which by the dictionary means "apparent perception of an object not actually present". He had always seen himself as pretty much like other men, never as "a decapitated but still living biped".
What is actually here, he says, is a "luminous and absolutely pure void, which nevertheless is, rather than contains, all things". However carefully he attends, he finds no blank screen on which things are reflected, no lens or aperture, "still less a soul or a mind to which they are presented, or a viewer (however shadowy) who is distinguishable from the view". Even distance vanishes: how can the blue sky or the pink-edged snows be remote "when there's nothing to be remote from"? A measuring rod from a head to the peak of Everest, read end-on (the only way he can read it), shrinks to a point. "All twoness, all duality of subject and object, has vanished."
He admits that every such description misrepresents the experience by complicating it: "the longer the postmortem examination drags on the further it gets from the living original". At best words remind one of the vision or invite its return, just as no menu tastes like the dinner and no book about humour makes you see a joke. Yet one cannot stop thinking for long, and some attempt to relate lucid intervals to a confused life is inevitable, lest life "disintegrate into two quite alien, idea-tight compartments". So he takes on the objections, while insisting no argument can add to or subtract from an experience "as plain and incontrovertible as hearing middle C or tasting strawberry jam".
Common sense objects; Harding replies
| Objection | Harding's reply |
|---|---|
| My head is missing, but not my nose: it precedes me everywhere. | What I see are two fuzzy, pinkish, transparent clouds, left and right. The single opaque protuberance in the middle of your face is a nose; calling both "nose" is dishonest. Other men have a nose apiece; I have none. |
| A sceptic who punched midway between the two clouds would hurt me. | Granted: the result would be as unpleasant as if I owned "the most solid and punchable of noses". |
| Here are tensions, pressures, itches, tickles, aches, warmths, throbbings. | Real sensations, but they do not amount to a head. To make one you must add what is plainly missing here: coloured three-dimensional shapes, eyes, mouth, hair. |
| My hand can feel my head when I explore here. | Groping for the head, I lose the hand too: it is "swallowed up in the abyss", a beacon-fire that consumes all that approaches, so the world's brilliance is never obscured. |
| Without eyes, how do I see you? | "See" has two meanings. Between third persons it is visual communication, a physical chain of light, lenses, retinas, cortex, with no chink for "mind". True seeing is first-person and eyeless. |
His summary: "my world and my head are incompatibles, they won't mix"; there is no room for both on these shoulders, "and fortunately it is my head with all its anatomy that has to go". This is not argument or acumen but simple sight: "LOOK-WHO'S-HERE instead of THINK-WHO'S-HERE". If he fails to see what he is, it is because he is "too busily imaginative, too 'spiritual', too adult and knowing". What he needs is "a kind of alert idiocy": "It takes an innocent eye and an empty head to see their own perfect emptiness."
The honest observer, the camera and the cinema
Thought experiment: the approaching observer. Setup: (1) a sceptic starts across the room and sees Harding as a full-length man with a head; (2) approaching, he sees half a man, then a head, then a blurred cheek, eye or nose, then a blur, then (at contact) nothing at all; (3) with scientific instruments the blur resolves into tissues, cell groups, a single cell, a nucleus, giant molecules, and finally empty space; (4) if he turns round and looks out with Harding instead of in at him, he finds the vacancy "filled to capacity with everything imaginable". The intuition it pumps: what you are at the place you are is an empty viewpoint, and what you are from elsewhere is a man. "This central Point exploding into an Infinite Volume, this Nothing into the All, this Here into Everywhere." The camera, lacking memory and anticipation, agrees: over there a man, midway bits of a man, here no man and nothing, or, pointed outward, the universe.
So "this head is not a head, but a wrong-headed idea". Whether one finds a human head here, an ass's head, a fried egg or a bunch of flowers makes no difference: "to have any topknot at all is to suffer from delusions." Yet over there he is far from headless: he has "more heads than I know what to do with". They turn up in observers and cameras, picture frames, shaving mirrors, door knobs, spoons, coffeepots, "shrunken and distorted, twisted back-to-front, often the wrong way up, and multiplied to infinity". The one place none can appear is here, where it would blot out "this Central Void which is my very life-source". The man behind the looking-glass, "that small, dull, circumscribed, particularized, ageing, and oh-so-vulnerable gazer", is the opposite in every way of his real Self, "this ageless, adamantine, measureless, lucid, and altogether immaculate Void".
Film directors, practical people, already know this. A film of a car crash driven by someone else leaves us mildly interested spectators on the pavement. A film shot from the driver's seat grips us: swaying knees, foot hard down, hands at the wheel, the long bonnet, telegraph poles whizzing past, oncoming cars looming, the crash, a flash, an empty silence. "I have been taken for a ride." Such shots use a headless dummy with the camera in place of the head, or an actor whose head is pulled back or aside: to make me identify with him, his head must be got out of the way. "A picture of me-with-a-head is no likeness at all, it is the portrait of a complete stranger."
Harding finds it curious that one learns the deepest and simplest truths about oneself from the advertising man, and that the cinema helps cure an illusion that young children and animals are free of. Old cults and legends of loose and flying heads, one-eyed or headless monsters, bodies with animal heads, and martyrs who (like King Charles in the ill-punctuated sentence) "walked and talked after their heads were cut off" are, he says, "nearer than common sense ever gets to a true portrait of this man". The essay ends in mysticism: "In the language of the sages, only the Buddha Nature, or Brahman, or Allah, or God, sees or hears or experiences anything at all."
Try this: the video call. On a video call your own face sits in a small tile in the corner, a head "over there" like Harding's heads in spoons and coffeepots. Notice that you do not look from that tile; you look from a place that never appears on the screen at all. Turn off "self view" and nothing about your seeing changes. Harding would say the tile was always the stranger.
Critical lens. Harding's argument trades on a slide from "I do not see a head here" to "there is no head here". The first is true of every seeing creature; the second is false, as the punch test he concedes shows. His reply, that "head" must mean what I can see, simply redefines the word. What survives is a phenomenological observation (the visual field has no visible owner at its origin) that philosophers from Ernst Mach (whose famous drawing of his own visual field shows eyebrow, nose and moustache, but no head) to Wittgenstein ("the eye does not see itself") had made more soberly. The mystical conclusion, that only God sees, does not follow from it.
Reflections (D.R.H.)
Hofstadter calls the essay "a charmingly childish and solipsistic view of the human condition", one that at an intellectual level "offends and appalls us". Yet it speaks clearly to a primitive level in us: the level at which we cannot accept our own death. In most of us that level has been submerged so long that we forget how incomprehensible personal nonexistence is. We extrapolate easily from others' deaths to our own. "Yet how can it be a day when I die? After all, a day is a time with light and sounds; when I die, there will be none of those." An inner voice protests that this is solipsistic, and, "coerced by the power of a simple syllogism", overrides the notion that I am a necessary ingredient of the universe.
The syllogism of mortality, and where each part comes from
- Premise 1, "All human beings are mortal": needs the abstract category "human being" (an advanced feat: bees seem to have "flower" but not "chimney" or "human"; dogs and cats coin "food dish", "door", "toy"; people pile category on category, a profound source of joy) and the general concept of death, built from early vanishings: food leaving the spoon, the rattle falling, Mommy going away, the balloon popping, the newspaper burning, a house razed, then the swatted fly and sprayed mosquito.
- Premise 2, "I am a human being": the tricky one. The child forms "human being" from things outside; that the class can "fold back" and engulf its maker comes later and must be shocking, though few remember it.
- Conclusion, "Therefore I am mortal": the conjunction of the two "slaps us in the face", "an ugly, brutal blow that sends us reeling", for days, months, "for our whole lives". We suppress the conflict and turn it elsewhere.
Hofstadter wonders whether higher animals see themselves as members of a class, whether a dog could wordlessly think "I bet I look like those dogs over there". He imagines a gory test: a ring of twenty animals of one kind; an evil human spins a dial, walks to the designated animal and knifes it in front of the rest. Will each think, "That animal is just like me, and my goose may soon be cooked just as his was"? He suggests this ability to "snap oneself onto others" belongs only to higher species, and links it to Nagel's "What Is It Like to Be a Bat?" (Selection 24).
The process, he proposes, starts with partial mappings ("I have feet, you have feet; I have hands, you have hands") that eventually induce a total mapping: I conclude from your having a head that I have one too, though I cannot see mine. This stepping outside myself is "gigantic and, in some ways, self-denying"; it contradicts much direct knowledge, like Harding's two verbs "to see". The sheer weight of mappings wins: logic overrides intuition, just as we came to believe the Earth is round like the alien moon without people falling off. Only "a powerful vision such as Harding's" returns us to the primordial sense of self and otherness at the root of the problems of consciousness, soul and self.
Hofstadter's conspiracy theory. Every imaginative person sometimes suspects that life is a huge hoax, perhaps a psychology experiment run by an inconceivable superbeing to see how far we can be pushed into believing absurdities: that sounds I cannot understand mean something; that someone could hear Chopin or eat chocolate ice cream without loving it; that light has the same speed in every reference frame; that I am made of inanimate atoms; that I will die. The theory undermines itself, since it explains away mysteries by postulating another, inconceivable mind. "There seems to be no alternative to accepting some sort of incomprehensible quality in existence. Take your pick."
We all fluctuate delicately between a subjective and objective view of the world, and this quandary is central to human nature.
What changed by 2026. The "headless" first-person view has become an engineering spec. Virtual-reality headsets render exactly Harding's scene (hands and torso, no head) because a visible head breaks body ownership; experiments since the rubber-hand illusion (Botvinick and Cohen, 1998) and Henrik Ehrsson's out-of-body and body-swap illusions (2007–08) show the sense of where "I" am can be moved with a camera and a little synchronous touch, which is Harding's film-director point made in the lab. Hofstadter's question about animals and death also has new data: field observations of elephants, chimpanzees and corvids responding to dead conspecifics fed a growing field of comparative thanatology, though whether any animal grasps its own mortality remains open.
Harold J. Morowitz, a Yale biophysicist, argues in Psychology Today (August 1980) that biologists and physicists have swapped places: biology has marched towards nineteenth-century materialism while physics, forced by quantum mechanics, has put the observer's mind back into nature. Chain the reductionists together (psychology to biology to chemistry to physics) and the chain closes into a circle, from the mind back to the mind; add the evolutionary discontinuities of life and reflective thought, and radical reductionism looks both weak and dangerous. Hofstadter's Reflections take the quantum loop seriously enough to explain the measurement problem with a "quantum water faucet" and Everett's many worlds, and then show that the puzzle reappears as a question of personal identity: why is this "me" on this branch?
Harold J. Morowitz: biophysicist at Yale; the selection's author.
Carl Sagan: astronomer; The Dragons of Eden is Morowitz's specimen of psychological reductionism.
Francis Crick: co-discoverer of DNA's structure; Of Molecules and Men is the specimen of biological reductionism.
Emil Du Bois-Reymond: nineteenth-century physiologist who dreamed of a Newtonian mechanics of life reaching to free will.
Werner Heisenberg and Erwin Schrödinger: quantum founders who turned to philosophy and (Schrödinger) to the Upanishads.
Eugene Wigner: Nobel physicist whose "Remarks on the Mind-Body Question" made consciousness part of quantum theory.
Lawrence B. Slobodkin: evolutionary biologist who names the human discontinuity "an introspective self-image".
Hugh Everett III: proposer (1957) of the many-worlds interpretation, discussed in Hofstadter's Reflections.
The selection: two trains passing in the night
"Something peculiar has been going on in science for the past 100 years or so." Biologists, who once gave the human mind a privileged place, have moved relentlessly towards "the hard-core materialism that characterized nineteenth-century physics". Physicists, compelled by experiment, have moved away from mechanical models towards a view in which the mind plays an integral role in all physical events. It is "as if the two disciplines were on two fast-moving trains, going in opposite directions and not noticing what is happening across the tracks."
The psychologist is left in an ambivalent position. Seen from biology, psychology studies phenomena far from the "core of certainty", the world of atoms and molecules. Seen from physics, psychology deals with "the mind", "an undefined primitive that seems at once essential and impenetrable". Both views hold some truth, and resolving them is essential to the foundations of behavioural science.
Reductionism, from Sagan to Crick to Du Bois-Reymond
Reductionism explains one level of phenomena by concepts at a lower, presumably more fundamental level: chemistry explains large-scale reactions by molecules, physiology explains cells by organelles, geology explains minerals by their crystals. Its essence is "seeking explanation in underlying structures and activities".
- Psychological reductionism. Carl Sagan's best-seller The Dragons of Eden states the premise: all the brain's workings, "what we sometimes call 'mind'", are "a consequence of its anatomy and physiology and nothing more". Tellingly, Sagan's glossary lacks mind, consciousness, perception, awareness and thought, but has synapse, lobotomy, proteins and electrodes.
- Its history. Before the nineteenth century Descartes' mind-body dualism kept the mind outside biology. The evolutionists' stress on our "apeness" made us subject to the methods used on primates and other animals, and the Pavlovian school made this a cornerstone of behavioural theory. Most psychologists concede hormonal, neurological and physiological components to action; Sagan is radical in seeking complete explanation at the lower level, the force of "and nothing more".
- Biological reductionism. Francis Crick's Of Molecules and Men, an attack on vitalism (the doctrine that life needs forces outside physics), declares: "The ultimate aim of the modern movement in biology is in fact to explain all biology in terms of physics and chemistry", meaning the atomic level where knowledge is secure. The italicised "all" marks the radical reductionism of a generation of biochemists and molecular biologists.
- The chain assembled. Combine the two and you get a sequence from mind to anatomy and physiology, to cell physiology, to molecular biology, to atomic physics, all resting on the bedrock of quantum physics. Psychology becomes a branch of physics, "a result that may cause some unease among both groups of professionals".
- An old hubris. The idea peaked with mid-nineteenth-century European physiologists. Emil Du Bois-Reymond, introducing an 1848 book on animal electricity, wrote that with sufficient methods an analytical mechanics of life "would reach even to the problem of the freedom of the will". Thomas Huxley carried the tone into the defence of Darwinism; today it echoes most clearly in the sociobiologists. Only the base has changed: quantum mechanics has replaced Newtonian mechanics.
Physics rediscovers the observer
Late-nineteenth-century physics offered a very ordered picture: events unfolded by Newton's equations in mechanics and Maxwell's in electricity, "inexorably, independent of the scientist, who was simply a spectator". Many physicists thought the subject essentially complete.
1848 Du Bois-Reymond promises a mechanics of life reaching to free will.
1905 Einstein's relativity: observers in relative motion perceive the world differently; the scientist starts to lose the spectator's role.
1920s–30s Quantum mechanics makes the observer "an essential component in defining an event"; its founders write books on philosophy.
1957 Hugh Everett III proposes the many-worlds interpretation (in Hofstadter's Reflections).
1958 Schrödinger's Mind and Matter moves towards the "perennial philosophy" and the Upanishads.
1961 Wigner's "Remarks on the Mind-Body Question" makes consciousness necessary to quantum theory.
1970s Popular works link physics and mysticism: Capra's The Tao of Physics, Zukav's The Dancing Wu Li Masters.
1980 Morowitz's essay in Psychology Today.
Never before, Morowitz says, had all the leading contributors to a science written books on the philosophical and humanistic meaning of their results. Heisenberg (Philosophical Problems of Quantum Physics) wrote of renouncing an objective time scale common to all observers and events independent of our ability to observe them: the laws of nature deal "no longer with elementary particles, but with our knowledge of these particles", that is, with the contents of our minds. Schrödinger, who formulated quantum theory's fundamental equation, moved in Mind and Matter (1958) towards Aldous Huxley's "perennial philosophy" and was the first quantum theorist to express sympathy with the Upanishads.
Thought experiment: "Who killed Schrödinger's cat?" Setup (Morowitz's version): a kitten sits in a closed box with a jar of poison and a triphammer poised to smash it; a counter recording random events such as radioactive decay triggers the hammer; the run lasts just long enough for a probability of one-half that the hammer falls. Quantum mechanics represents the system as the sum of a live-cat and a dead-cat function, each with probability one-half. The question: does the act of looking (the measurement) kill or save the cat? What it shows: a complex system is described by a probability distribution over outcomes; a measurement turns probability into an event; and the only simple, consistent account physicists found of a measurement involved an observer's becoming aware of the result. So "the physical event and the content of the human mind were inseparable", pushing science from the realist towards the idealist side of philosophy.
Eugene Wigner's essay "Remarks on the Mind-Body Question", Morowitz says, sums up the views of a large number of physical scientists: most have returned to the recognition that thought, the mind, is primary. "It was not possible to formulate the laws of quantum physics in a fully consistent way without reference to the consciousness." How remarkable, Wigner concludes, that the scientific study of the world led to the content of consciousness as an ultimate reality.
A second line of physics reinforces this. Information theory applied to thermodynamics shows that entropy measures the observer's ignorance of a system's atomic details: knowing pressure, volume and temperature, we still lack the exact positions and velocities of the molecules, and the amount of missing information is proportional to the entropy. In older thermodynamics entropy was engineering, the energy unavailable for external work; now the mind enters again, and entropy concerns our knowledge of a system's state as well as the state itself. The founders did not set out to impose a "mentalist" picture; experiment forced them.
The circle: from the mind back to the mind
Morowitz's three experts, chained
- Sagan (psychology): the human mind, including consciousness and reflective thought, is explained by the central nervous system, which reduces to its biological structure and function.
- Crick (biology): biological phenomena at all levels are totally understood through atomic physics, the action and interaction of carbon, nitrogen, oxygen and so on.
- Wigner (physics): atomic physics, understood most fully by quantum mechanics, must be formulated with the mind as a primitive component.
So, in three separate steps, we have gone "in a circle, from the mind, back to the mind". The result will comfort Eastern mystics more than neurophysiologists and molecular biologists, yet it follows from a straightforward combination of recognised experts. It has received little attention because individuals seldom work in more than one paradigm.
Reject the circularity and you are left with two opposing camps: a physics claiming completeness because it describes all of nature, and a psychology claiming to embrace everything because the mind is our only source of knowledge of the world. Given the problems in both, Morowitz urges a more sympathetic look at the circle: it deprives us of firm absolutes but "encompasses the mind-body problem" and gives the disciplines a framework to talk. "The closing of the circle provides the best possible approach for psychological theorists."
Discontinuities: life and reflective thought
The strict reductionism of sociobiology also fails on biological grounds. It assumes continuity in evolution from early mammals to man, so that mind was no radical departure. But evolution has dramatic discontinuities: the big bang is a cosmic one, and the origin of life another.
How life changed the rules. Before genetic life, thermal fluctuations and noise averaged out, giving precise laws (of planetary evolution, for instance). Once information was encoded in self-replicating molecules, a single molecular event at the level of thermal noise could have macroscopic consequences: a mutation can redirect all of evolution, kill a whale by inducing a cancer, or wreck an ecosystem by creating a virulent virus that attacks a key species. Physics is not abrogated, but a new feature appears, "large-scale consequences of molecular events", which makes evolutionary history indeterminate.
Many biologists and psychologists think reflective thought, arising in primate evolution, is another rule-changing discontinuity. Lawrence B. Slobodkin identifies the new feature as an introspective self-image, which alters responses to evolutionary problems so that major historical events cannot be assigned to biological laws. Man cannot be understood by laws for mammals with very similar brain physiology. "Primates are very different from other animals, and human beings are very different from other primates." The discontinuity must be studied, but first it must be recognised; it is data that "cannot be shelved just to preserve reductionist purity".
Finally, radical reductionism is dangerous. How we treat our fellows depends on how we conceptualise them: see them solely as animals or machines and we drain our interactions of humanistic richness; seek behavioural norms in animal societies and we ignore what is uniquely human. It "offers very little in the area of moral imperatives" and "presents the wrong glossary of terms for a humanistic pursuit". Morowitz shares the enthusiasm for neurobiology, but warns against letting it lock us into positions that deny "the most intriguing aspect of our species". "The human psyche is part of the observed data of science. We can retain it and still be good empirical biologists and psychologists."
Critical lens. (1) Morowitz's physics premise is a minority interpretation. Wigner's "consciousness causes collapse" (following von Neumann) was never the consensus, and Wigner himself later backed away from it as decoherence theory (H. Dieter Zeh from 1970, Wojciech Zurek in the 1980s) showed how interaction with an environment suppresses visible superpositions without any mind. (2) The entropy-as-ignorance view (E. T. Jaynes) is one reading of statistical mechanics; it does not make entropy depend on any particular human's knowledge in a way that changes what engines can do. (3) Relativity does not make the observer's mind part of reality: "observer" there means a reference frame, which a clock can be. (4) The "closed circle" mixes kinds of dependence: psychology depends on biology for what minds are made of, physics (on Wigner's view) depends on minds for what measurement is; a circle of explanation is not obviously vicious or profound. (5) Calling reductionism "dangerous" because of how it might make us treat people is an argument about consequences, not truth.
Reflections (D.R.H.)
Hofstadter opens with two epigraphs on branching possibility. From Borges's "The Garden of Forking Paths": Ts'ui Pên conceived time not as absolute and uniform (unlike Newton and Schopenhauer) but as "an infinite series of times, in a dizzily growing, ever spreading network of diverging, converging and parallel times", in some of which you exist and not I, in others I and not you; "In another, you, crossing the garden, have found me dead." And William James: "Actualities seem to float in a wider sea of possibilities from out of which they were chosen."
It is attractive, Hofstadter says, to think the mysteries of quantum physics and of consciousness are one mystery; Morowitz's loop has "just about the proper amounts of hard science, beauty, weirdness, and mysticism to 'sound right'". But it opposes a central theme of this book: that nonquantum-mechanical computational models of mind are possible in principle. Right or wrong ("it is too early to say"), the ideas are worth thinking about, because the interaction of subjective and objective viewpoints really is at the heart of quantum mechanics, which gives "observers" a privileged causal status without saying whether consciousness is needed to be one.
Thought experiment: the quantum water faucet. Setup: (1) a faucet with hot and cold knobs, each continuously adjustable; (2) the water is always totally hot or totally cold, the two "temperature eigenstates"; (3) in orthodox quantum mechanics it is putting your hand in that throws the water into one eigenstate; before that it is in a superposition; (4) only H open gives hot every time, only C gives cold; both open gives a probability you can estimate by repeated trials, with a crossover point where hot and cold are equally likely, "like flipping a coin" (and "sadly reminiscent of many a bathroom shower"). The twist: in a fundamental sense the system itself does not "know" which eigenstate it is in; it decides at random when the observer "tests the water", and until then, "for all purposes", it is in no eigenstate.
Indirect tests do not help. Run your washing machine on the faucet water: you will not know if the wool sweater shrank until you open the machine. Make tea: you will not know if it is iced until you taste it. Attach a recording thermometer: until you see the ink marks, "You can't be any surer that the ink is on the paper than you are that the water has a definite temperature." The sweater, tea and thermometer, lacking observer status, must "play along with the gag" and enter their own superpositions: shrunk-and-unshrunk, iced-and-hot, ink-high-and-ink-low.
This goes beyond the old riddle of the tree falling unheard in the forest. Quantum superpositions have observable consequences opposite to those of a hidden definite state: maybe-hot-maybe-cold water behaves differently from water that is secretly hot or cold, because the alternatives interfere like overlapping waves (a speedboat's wake cancelling its reflection off a jetty; ripples from a skipped stone crisscrossing on a lake). The effect is only statistical, visible after many sweater-washings or tea-makings; Richard Feynman's The Character of Physical Law explains it beautifully.
Schrödinger's cat carries this further: even a cat could be in a superposition until a human looked. "Isn't a live cat as much of a conscious observer as a human being is?" Probably, but a dead cat certainly is not. "In effect, we have created a superposition of two eigenstates one of which has observer status, the other of which lacks it!" Hofstadter compares the predicament to the Zen master Kyōgen's riddle (from Paul Reps's Zen Flesh, Zen Bones): a man hangs from a tree by his teeth over a precipice; someone below asks why Bodhidharma came to China; silent, he fails; answering, he falls.
To many physicists the observer/non-observer distinction seems artificial, even repugnant, and wave-function collapse, a sudden random jump, puts caprice into the ultimate laws. "God does not play dice" was Einstein's lifelong belief.
Thought experiment: Everett's many worlds. Setup: (1) in 1957 Hugh Everett III proposed that no system ever jumps into an eigenstate; (2) the superposition evolves smoothly, sprouting branches for each alternative, all unfolding in parallel within one universal wave function; (3) Schrödinger's cat has two branches; on one it feels alive, on the other "there's no cat to feel anything". The intuition it pumps: what did the cat feel a moment before death on the fatal branch? Which branch holds "the genuine cat"? And since your life has had "billions upon billions" of quantum branchings, you have split into countless yous. Everett's footnote handles the difficulty "with a poker face": use the singular for the one physical system, the plural for the different experiences. "The problem of how it feels subjectively is not treated; it is just swept under the rug."
The soul-searching question
"Why, then, do I feel myself to be in just one world?" On Everett's view you don't: all alternatives are felt, but "this you going down this branch" doesn't feel them all. The ultimate question becomes: "Why is this me in this branch, then? What makes me, I mean this me, feel itself, I mean myself, unsplit?"
The sunset on the beach. Friends stand at various points along wet sand watching the sun set over the ocean. Each sees the sun's reflection as a straight line of orange-red glints pointing right at them, and thinks, "How lucky that I am the one lined up exactly with that line." Every friend is having the same thought, "or is it the same?" What it shows: each perspective feels uniquely central, as each Everett branch's "me" feels uniquely real.
These musings are the heart of the "soul-searching question": why is this soul in this body, or on this branch? Saying "because your parents made that body" is circular: why were they my parents? Who would my parents have been had I been born in Hungary? What if someone else had been me? Or "am I someone else? Am I everyone else? Is there only one universal consciousness?" It is eerie to find such themes "at the core of what is supposedly our stablest and least erratic science".
Yet the link is natural: imaginary worlds in our minds resemble the parallel branches. The young man pulling petals, "She loves me, she loves me not", keeps two models of his beloved, or one model in "a mental analogue of a quantum-mechanical superposition". A novelist entertaining several ways to continue a story holds the characters in a mental superposition, and if the novel is never written, "it would even seem strange to ask which story is the genuine version". Somewhere there is a branch on which you did not make that stupid mistake you regret. Aren't you jealous? "But how can you be jealous of your self?" And on another branch a stupider you is jealous of this one.
Perhaps the universal wave function is the mind (or brain) of the great novelist in the sky, God, entertaining all branches at once; we would be subsystems of God's brain, our versions no more privileged than our galaxy is "the only genuine galaxy", and God's brain would evolve smoothly and deterministically, as Einstein wanted. Paul Davies (Other Worlds): "our consciousness weaves a route at random along the ever-branching evolutionary pathway of the cosmos, so it is we, rather than God, who are playing dice."
But the deepest riddle survives: why does my unitary feeling of myself propagate down this branch rather than another, and what attaches "me-ness" to this body, on this branch, at this moment? Quantum mechanics does not answer it. In fact, Hofstadter says, it is "exactly the collapse of the wave function reappearing at the far end of the rug as it was shoved under by Everett", now turned into a problem of personal identity no less perplexing than the original. Deeper still: some branches contain no evidence for quantum mechanics at all and no Everett; on some, "this entire Reflection got written exactly as you see it here, except that it ended with a different flutzpah."
Morowitz's loop: consciousness is needed to complete physics, so mind is a primitive of nature and the circle of explanation closes on it.
Hofstadter's reply: a computational, non-quantum model of mind is possible in principle; the quantum puzzle does not prove mind is fundamental, it relocates the old question of why I am this one, from physics into personal identity.
What changed by 2026. A century after quantum mechanics, physicists still disagree: a 2025 Nature survey of more than a thousand physicists found the Copenhagen family the most popular view, but well short of a majority, with many-worlds a substantial minority. "Consciousness causes collapse" has few defenders, though Roger Penrose and Stuart Hameroff's Orch-OR theory (from the 1990s) keeps a quantum-mind link alive, and remains highly contested. Hofstadter's "why am I this branch?" now has a cousin in AI and cloud computing: when a model or a whole software agent is run as many simultaneous instances, each with the same weights and history up to the fork, the question "which one is it?" is no longer a thought experiment. Engineers answer it the way Everett did, with a poker face and an instance ID.
Part II surveys the mind's I from the outside: what, if anything, could reveal another mind or soul to someone searching for it? It opens with Turing's proposal to replace "Can machines think?" with a conversational test, follows with Hofstadter's dialogue testing the test, and then turns to fiction: Lem's dream-princess in a computer, and Miedaner's chimpanzee and beetle-robot, which test our feelings rather than our arguments.
The five selections make one long argument with two halves. In the first half the criterion is conversation. Turing (1950) proposes the Imitation Game and answers nine objections, from theology to telepathy; Hofstadter's Coffeehouse Conversation (1981) lets three students attack and defend it until Sandy's case emerges: thought is a pattern, the test probes the pattern, and anything that passes it thinks. Language is the window, and the claim is that it is a wide enough window.
In the second half the criterion is sympathy, and the window narrows. Lem's Princess Ineffabelle exists only as a computer simulation, so the question becomes whether a simulated princess can be loved, and whether her suitor can join her. Miedaner's Martha is a chimpanzee who can say, through a language machine, "Martha good girl. Why kill?"; his Mark III Beast is a small machine that squeals, flees and bleeds oil when struck with a hammer, and a hardened engineer finds she cannot finish it off. The editors use the pair to show how quickly we grant a "soul" to anything that seems to want to live, and how little that response depends on what is inside.
The test of words (Turing, Hofstadter): a mind is whatever can sustain an open-ended conversation under hostile probing; looks, bodies and sympathy are deliberately screened out.
The test of feelings (Lem, Miedaner): we decide who has a soul by our emotional response to its struggle and suffering, a response that can be triggered by very simple machinery.
Alan M. Turing's 1950 paper in the philosophy journal Mind replaces the "too meaningless" question "Can machines think?" with a game: can a digital computer, communicating by teleprinter, fool an interrogator as often as a man pretending to be a woman does? Turing predicts that within about fifty years machines with a storage of about 10⁹ will leave an average interrogator no better than a 70 percent chance of a correct identification after five minutes, and then answers nine objections in turn. His central move is to show that every objection either applies equally to other people (and so leads to solipsism) or rests on an unproved assumption that the human mind is free of limits that machines have. The editors' brief Reflections, signed by both, take issue only with his surprising respect for telepathy.
The imitation game
Turing starts by refusing to define "machine" and "think" by common usage: that would make the answer a matter for "a statistical survey such as a Gallup poll", which is absurd. Instead he replaces the question with a closely related one in relatively unambiguous words.
Thought experiment: the imitation game. Setup: (1) three players: a man (A), a woman (B) and an interrogator (C) of either sex, in a separate room; (2) C knows the others only as X and Y and must end by saying "X is A and Y is B" or the reverse; (3) answers are typewritten, ideally by teleprinter, so tones of voice give nothing away; (4) A tries to make C choose wrongly (asked about hair, he might say "My hair is shingled, and the longest strands are about nine inches long"); B tries to help, probably by telling the truth, but her "I am the woman, don't listen to him!" avails nothing, since he can say the same. The substitution: now let a machine take the part of A. Will the interrogator decide wrongly as often as when the game is played between a man and a woman? This question replaces "Can machines think?"
Is the new question a worthy one?
Turing cuts short the regress ("What is the answer?" then "Is it worth asking?") and argues the game draws a fairly sharp line between physical and intellectual capacities. No engineer can make material indistinguishable from human skin, and even if one could, there would be little point dressing a thinking machine in artificial flesh; so the interrogator may not see, touch or hear the competitors.
Turing's specimen questions and answers
- "Please write me a sonnet on the subject of the Forth Bridge." "Count me out on this one. I never could write poetry."
- "Add 34957 to 78764." After a pause of about 30 seconds: "105621."
- "Do you play chess?" "Yes."
- "I have K at my K1, and no other pieces. You have only K at K6 and R at R1. It is your move. What do you play?" After a pause of 15 seconds: "R-R8 mate."
The question-and-answer method can introduce almost any field of human endeavour. We need not penalise a machine for failing at beauty competitions or a man for losing a race against an aeroplane; witnesses may brag about their charms, strength or heroism, but the interrogator cannot demand practical demonstrations.
Two worries are raised and set aside. First, the odds may be weighted against the machine: a man pretending to be a machine would be given away at once by slowness and inaccuracy in arithmetic, and machines might think in ways very different from ours. Turing calls this "a very strong" objection, but if a machine nevertheless plays satisfactorily, it need not trouble us. Second, the machine's best strategy might not be imitation of a man; Turing thinks the effect unlikely to be large and assumes it will try to give the answers a man would naturally give.
Critical lens: the deliberate (or accidental) error. 34,957 + 78,764 = 113,721, not 105,621. Turing's witness pauses thirty seconds and then gets the sum wrong. Many readers take this as Turing's joke, a machine playing human by being slow and fallible, which matches his later remark that a machine in the game would "deliberately introduce mistakes". Turing never says so, and it may be a plain slip. Either way, the sample shows his test rewards human-like performance, not correct performance, a point that matters again with today's chatbots.
Which machines may play
Turing wants to allow every kind of engineering technique, including machines whose workings their own builders cannot describe because they used largely experimental methods, yet to exclude men "born in the usual manner". These conditions are hard to reconcile. Requiring an all-male (or all-female) engineering team would not help, since it is probably possible to rear a complete individual from a single skin cell of a man, a biological triumph we would not call "constructing a thinking machine". So he drops the first requirement and admits only digital computers, the machines that had aroused the interest in "thinking machines" in the first place.
Universality. Digital computers can mimic any discrete-state machine; they are universal machines. Considerations of speed apart, there is no need to design new machines for new computing tasks: "They can all be done with one digital computer, suitably programmed for each case." Consequently "all digital computers are in a sense equivalent", so the question becomes whether any suitably programmed computer could play the game well.
Turing's own beliefs and predictions
Before answering critics, Turing states his beliefs openly, arguing that conjectures are valuable provided they are clearly marked as such; the picture of scientists marching from proved fact to proved fact, never influenced by conjecture, "is quite mistaken".
About 50 years: Turing's horizon (from 1950), so roughly the year 2000.
About 10⁹: the storage capacity (in binary digits) he thought sufficient, about 125 megabytes in modern units.
70 percent: the most an average interrogator would score at correct identification.
5 minutes: the length of questioning.
End of the century: by then "one will be able to speak of machines thinking without expecting to be contradicted".
The original question he calls "too meaningless to deserve discussion", which does not stop him predicting that usage and educated opinion will change. Then he considers the opposition.
The nine objections and Turing's replies
| # | Objection | Its claim | Turing's reply in a line |
|---|---|---|---|
| 1 | Theological | Thinking belongs to the immortal soul, given only to humans | It limits God's omnipotence; we would be building mansions for souls He creates |
| 2 | "Heads in the sand" | Thinking machines would be too dreadful, so let us hope they can't | Not substantial enough to refute; offer consolation instead |
| 3 | Mathematical | Gödel and Turing show limits on discrete-state machines | Unproved that humans lack such limits; triumph is only over one machine at a time |
| 4 | Consciousness (Jefferson) | No machine feels what it writes | Taken strictly this is solipsism; a convincing viva voce would satisfy most |
| 5 | Various disabilities | Machines can never do X (be kind, love, err, be original...) | Bad induction from the small machines we know |
| 6 | Lady Lovelace | Machines only do what we order them to | They surprise Turing often; working out consequences is real work |
| 7 | Continuity of the nervous system | Brains are continuous, computers discrete | A digital machine can mimic a continuous one well enough to fool the interrogator |
| 8 | Informality of behaviour | No rules cover all conduct, so we are not machines | Confuses rules of conduct with laws of behaviour; the latter cannot be ruled out |
| 9 | Extrasensory perception | Telepathy could unmask the machine | A real worry, met by a "telepathy-proof room" |
1. The theological objection
Thinking is a function of man's immortal soul, given by God to every man and woman but not to animals or machines. Turing accepts no part of this but replies in theological terms. He would find it more convincing if animals were classed with men, since the gap between the typically animate and the inanimate is greater than that between man and other animals. Its arbitrariness shows when viewed from another religion: "How do Christians regard the Moslem view that women have no souls?" More seriously, it restricts God's omnipotence: granting that He cannot make one equal to two, should He not be free to give a soul to an elephant, perhaps along with a mutation improving its brain? The same holds for machines. Building them would not usurp His power any more than procreating children does: "we are, in either case, instruments of His will providing mansions for the souls that He creates." Turing adds that theological arguments have a poor record: in Galileo's day Joshua's "the sun stood still" and the Psalm's foundations of the earth "that it should not move" were thought to refute Copernicus. A footnote notes that Aquinas says God cannot make a man without a soul, which may only reflect souls' indestructibility.
2. The "heads in the sand" objection
"The consequences of machines thinking would be too dreadful. Let us hope and believe that they cannot do so." Seldom said so openly, it affects most of us: we like to think Man is in some subtle way superior to the rest of creation, preferably necessarily so, and the feeling is likely strongest in intellectuals, who prize thinking most. Turing does not think it substantial enough to refute: "Consolation would be more appropriate: perhaps this should be sought in the transmigration of souls."
3. The mathematical objection
Gödel's theorem shows that any sufficiently powerful logical system contains statements that can be neither proved nor disproved within it, unless the system is inconsistent; there are related results by Church, Kleene, Rosser and Turing himself. Turing's own is most convenient since it concerns machines directly: a digital computer with infinite capacity, rigged to answer yes/no questions, will on some questions give a wrong answer or none at all, however long it is given. A typical one: "Consider the machine specified as follows... Will this machine ever answer 'Yes' to any question?", where the machine described bears a certain simple relation to the machine under interrogation. This is said to prove a disability to which the human intellect is not subject.
Turing's short answer. It has been established that any particular machine has limits, but "it has only been stated, without any sort of proof, that no such limitations apply to the human intellect." The feeling of superiority when a machine fails the critical question is genuine, but we too often give wrong answers to be pleased by machine fallibility, and the triumph is only over the one machine we have tripped up. "There might be men cleverer than any given machine, but then again there might be other machines cleverer again, and so on." Those who press this objection would mostly accept the imitation game; believers in objections 1 and 2 "would probably not be interested in any criteria".
4. The argument from consciousness
Turing quotes Professor Geoffrey Jefferson's Lister Oration of 1949: not until a machine can write a sonnet or compose a concerto "because of thoughts and emotions felt, and not by the chance fall of symbols", and know that it had written it, could we agree that machine equals brain. No mechanism could feel (and not merely "artificially signal, an easy contrivance") pleasure at success, grief when its valves fuse, warmth at flattery, misery at mistakes, the charm of sex, anger or depression at frustration.
Turing reads this as denying the validity of the test. In its extreme form the only way to be sure a machine thinks is to be the machine; likewise the only way to know a man thinks is to be that man. "It is in fact the solipsist point of view." It may be the most logical view, but it makes communication difficult: A believes "A thinks but B does not", B the reverse. "Instead of arguing continually over this point it is usual to have the polite convention that everyone thinks."
The viva voce on the sonnet. Exams already use the game (minus player B) to find out whether a student really understands or has learned "parrot fashion". Turing's sample: asked whether "a spring day" would do as well as "a summer's day", the witness says "It wouldn't scan"; "a winter's day" would scan, but "nobody wants to be compared to a winter's day"; would Mr. Pickwick remind you of Christmas? "In a way"; yet Christmas is a winter's day, and Pickwick would not mind the comparison: "I don't think you're serious. By a winter's day one means a typical winter's day, rather than a special one like Christmas." What it shows: if a machine sustained answers like these, Jefferson could hardly call them "an easy contrivance", a phrase meant for tricks like a recording of someone reading a sonnet, switched on now and then.
Most supporters of the argument from consciousness, Turing concludes, would abandon it rather than be forced into solipsism. He does not deny there is mystery about consciousness, for instance "something of a paradox connected with any attempt to localize it", but these mysteries need not be solved before answering his question.
5. Arguments from various disabilities
"I grant you that you can make machines do all the things you have mentioned but you will never be able to make one to do X." Turing's list of Xs: be kind, resourceful, beautiful, friendly; have initiative, a sense of humour; tell right from wrong; make mistakes; fall in love; enjoy strawberries and cream; make someone fall in love with it; learn from experience; use words properly; be the subject of its own thought; have as much diversity of behaviour as a man; do something really new.
These claims rest, he thinks, on unconscious scientific induction: a man has seen thousands of machines, ugly, narrow, useless for a minutely different purpose, with little variety of behaviour, and concludes these are features of all machines, when they come from small storage capacity. (A burned child avoiding the fire is also doing induction.) Human customs need a very large region of space-time to induce from reliably, "otherwise we may (as most English children do) decide that everybody speaks English, and that it is silly to learn French."
- Strawberries and cream. A machine might be made to enjoy them, but trying would be "idiotic". The point is that this disability feeds others, such as the difficulty of the same kind of friendliness between man and machine as between people of the same race (Turing's own example).
- Machines cannot make mistakes. "Are they any the worse for that?" More sympathetically: the interrogator might unmask the machine by its deadly accuracy at arithmetic. But a machine playing the game would deliberately introduce mistakes calculated to confuse. Deeper, the objection confuses errors of functioning (mechanical or electrical faults; ignored when discussing "abstract machines", which are mathematical fictions and cannot have them) with errors of conclusion, which arise once meaning is attached to output. A machine could type "0 = 1" repeatedly, or, less perversely, draw conclusions by induction that are occasionally wrong.
- Be the subject of its own thought. This can only be answered if the machine has thought with some subject matter, but "the subject matter of a machine's operations" means something: a machine solving x² − 40x − 11 = 0 has that equation as its subject. In this sense a machine can be its own subject matter: it can help write its own programs, predict the effects of alterations in its structure, and modify its programs by observing its own behaviour, "possibilities of the near future, rather than Utopian dreams".
- Diversity of behaviour. Just another way of saying small storage; until recently even a thousand digits of storage was very rare.
Many of these criticisms are the argument from consciousness in disguise: describe the method by which a machine does X and it is dismissed as "rather base", as in Jefferson's parenthesis.
6. Lady Lovelace's objection
Our most detailed information on Babbage's Analytical Engine comes from Lady Lovelace's memoir: "The Analytical Engine has no pretensions to originate anything. It can do whatever we know how to order it to perform." Douglas Hartree, quoting her, added that this does not rule out electronic equipment that will "think for itself", or in which one could set up a conditioned reflex as a basis for "learning". Turing agrees with Hartree, and adds that the Engine may in a sense have had the property: it was a universal digital computer, so with enough speed and storage it could mimic any discrete-state machine that had it. Probably this did not occur to the Countess or to Babbage, and they were under no obligation to claim all that could be claimed.
A variant says a machine can "never do anything really new", parried for a moment by "There is nothing new under the sun": who can be sure their own original work was not a seed planted by teaching? A better variant says a machine can never "take us by surprise". This Turing meets directly: "Machines take me by surprise with great frequency." He calculates hurriedly and slipshod ("I suppose the voltage here ought to be the same as there; anyway let's assume it is"), so results surprise him. A critic will say the surprise is his own creative act, but appreciating something as surprising needs as much creativity whether it comes from a man, a book or a machine.
The fallacy behind "no surprises". Philosophers and mathematicians are prone to assume that "as soon as a fact is presented to a mind all consequences of that fact spring into the mind simultaneously with it". It is a useful assumption but false, and it leads people to think there is no virtue in the mere working out of consequences from data and principles. Hofstadter later called Turing's reply to Lovelace a reply whose depth "eludes many good minds".
7. Continuity in the nervous system
The nervous system is not a discrete-state machine: a small error in the size of an impulse reaching a neuron may make a large difference to the outgoing impulse, so perhaps no discrete system can mimic it. True, a discrete machine differs from a continuous one, but under the game's conditions the interrogator cannot exploit the difference. Take a simpler continuous machine, a differential analyser. A digital computer could not predict its exact answers, but could give the right sort of answer: asked for π (about 3.1416), choose at random among 3.12, 3.13, 3.14, 3.15, 3.16 with probabilities 0.05, 0.15, 0.55, 0.19, 0.06. The interrogator would struggle to tell them apart.
8. Informality of behaviour
No set of rules can say what a man should do in every circumstance: stop at red, go at green, but what if a fault shows both at once? Stop, perhaps, but further difficulties follow. Turing agrees. The argument then runs: "If each man had a definite set of rules of conduct by which he regulated his life he would be no better than a machine. But there are no such rules, so men cannot be machines." "The undistributed middle is quite glaring."
Rules of conduct: precepts like "Stop if you see red lights", which one can act on and be conscious of. That there is no complete set of these is granted.
Laws of behaviour: laws of nature applied to the body, like "if you pinch him he will squeak". Being regulated by such laws implies being some sort of machine, and vice versa; and we can never say "We have searched enough. There are no such laws."
Turing presses the point with an experiment. Even if we could be sure of finding laws that existed, it should be possible to predict a discrete-state machine from observation within, say, a thousand years. Yet he set up on the Manchester computer a small program using only 1,000 units of storage which, given a sixteen-figure number, replies with another within two seconds: "I would defy anyone to learn from these replies sufficient about the program to be able to predict any replies to untried values."
9. Extrasensory perception
The last objection, which Turing finds "quite a strong one", assumes the reader knows the four items of ESP: telepathy, clairvoyance, precognition and psychokinesis. They seem to deny all our usual scientific ideas, "How we should like to discredit them!", yet "the statistical evidence, at least for telepathy, is overwhelming." Accept them and ghosts and bogies are a small step away; the idea that our bodies move only by known and similar physical laws would be "one of the first to go". Many theories remain workable if one forgets about ESP, but this is "rather cold comfort", and thinking may be just where ESP matters most.
The telepathic witness. Play the game with a man who is a good telepathic receiver and a digital computer. The interrogator asks "What suit does the card in my right hand belong to?" The man scores 130 out of 400; the machine, guessing at random, perhaps 104; so the interrogator identifies them correctly. But if the computer uses a random number generator, the interrogator's psychokinesis might bias it towards right answers; or the interrogator might know the answer by clairvoyance without asking. "With E.S.P. Anything may happen." Turing's fix: put the competitors in a "telepathy-proof room".
Reflections (D.R.H. And D.C.D.)
The editors say most of their response comes in the dialogue that follows (Selection 5). Here they comment only on Turing's apparent readiness to believe that extrasensory perception might be the ultimate difference between humans and their machines. If it is not some "discreet joke", what motivated it? Turing evidently thought the evidence for telepathy strong. If so in 1950, it is no stronger thirty years later, probably weaker. Since 1950 there have been many notorious claims of psychic powers, often vouched for by physicists of some renown, some of whom later felt made fools of, retracted, and jumped onto "some new paranormal bandwagon the next month". The majority of physicists, and certainly of psychologists, doubt ESP in any form.
Turing's "cold comfort": paranormal phenomena might somehow be reconciled with established science, which mostly works if one forgets about them.
The editors' view: if telepathy, precognition and telekinesis existed with the properties claimed, physics could not simply be amended; it would take a major revolution in our scientific world view, a great adventure, but "the evidence that we need to do this has simply failed to accumulate."
Critical lens. (1) Turing's confidence in telepathy rested on J. B. Rhine's card-guessing work and especially on S. G. Soal's experiments in London; in 1978 Betty Markwick showed that Soal's key data had been manipulated, which bears out the editors. (2) "The Moslem view that women have no souls" is a Western misconception; mainstream Islamic teaching holds that women have souls. (3) The arithmetic answer is wrong (113,721), whether by design or slip. (4) The Mathematical Objection was revived by J. R. Lucas (1961) and Roger Penrose (The Emperor's New Mind, 1989), and Turing's reply ("unproved that humans lack such limits") is still the standard rebuttal. (5) Turing's figure of 10⁹ refers to storage in binary digits; the Mind's I text reprints the paper without its later section on learning machines ("child machines" to be educated), which is where Turing actually thought success would come from.
What changed by 2026. Turing's prediction was off by about two decades, and right in shape. Storage of 10⁹ bits arrived well before 2000, but the test resisted until large language models. In a 2025 pre-registered three-party study by Cameron Jones and Benjamin Bergen (UC San Diego), with five-minute conversations as Turing specified, GPT-4.5 prompted to adopt a humanlike persona was judged to be the human 73% of the time, more often than the real humans it was paired with; the 1960s program ELIZA managed about a quarter. That meets Turing's 70% benchmark from the other side. His second prediction, that people would speak of machines thinking "without expecting to be contradicted", is half true: people do, and are contradicted constantly. Lovelace's objection lives on as the claim that LLMs "only predict the next token"; Turing's reply (we do not know all the consequences of what we have ordered) is now the daily experience of anyone validating a model in a regulated bank.
Douglas Hofstadter's dialogue (first published in his "Metamagical Themas" column in Scientific American, May 1981) puts Turing's test on trial in a coffeehouse, with a sceptical physics student, a biology student who trusts gut feelings, and a philosophy student who defends the test while admitting private doubts. (The dialogue never reveals their sexes, a sly touch given its subject.) Sandy's case unfolds through a chain of analogies (a simulated hurricane seen through "computational glasses", a cash register that "can't really calculate", a radio ham decoding Morse, a particle accelerator scattering questions off a hidden mind) to two strong claims: thought is an abstract pattern independent of its medium, and thinking, feeling and consciousness are facets of one phenomenon. Sandy's final position is asymmetric: some thinking beings may fail the Turing test, but anything that passes it is genuinely thinking, and the critics' favourite counterexample, a jukebox of canned sentences, could never pass. Dennett's Reflections defend the test against the charge of behaviourism and ask whether mind is more like milk or like a song.
Chris: a physics student; not "against" AI, but sure its advocates underestimate the mind ("can you imagine a computer writing a Proust novel?").
Pat: a biology student; confused by "smart product" ads, ties consciousness to "soft, warm bodies" and to common biological origin.
Sandy: a philosophy student; the test's defender, but a self-confessed fence-straddler who sympathises with both doubters.
Daniel Dennett (offstage): his Brainstorms supplies the simulated hurricane, the cash-register reductio and the "intentional stance".
Parry: a program simulating a paranoid patient, said to have fooled psychiatrists.
Turing, ads and "electronic brains"
Chris has just read Turing's article on Sandy's advice and found it made Chris "think about my thinking". He insists he is not against AI ("wonderful stuff, perhaps a little crazy"), only that its advocates have "far underestimated the human mind"; Sandy replies, "Rome wasn't built in a day." Turing, Sandy explains, died in 1954 at forty-one, "almost certainly suicide" after harsh treatment for his homosexuality, and never saw the progress since.
The article, Sandy says, does two things: it answers "Will a machine ever think?" with yes, batting down objections one by one, and argues the question is not meaningful as it stands, being too emotionally loaded. Pat is confused by newspaper and TV ads for "products that think" and "intelligent ovens", while everyone also says computers are dumb and must have everything spelled out. Chris notes a manufacturer that markets "dumb terminals" to stand out. Sandy thinks of "electronic brain": many swallow it whole, others reject it out of hand, few sort out how much makes sense. No machine could take an IQ test, but Turing's test applies to any machine, and its answer is not clear-cut: "it shows how the borderline is quite fuzzy." Pat: "So, as is usual in philosophy, it's all just a question of words." Sandy: yes, but emotionally charged words about our concept of ourselves, which should not be swept under the rug.
The Imitation Game, and what winning would prove
Sandy describes the game: a man and a woman in separate rooms, an interrogator connected by teletype who must find the woman; she helps, he tries to bamboozle by answering as a woman might. Pat wants to play some day. Ideally all three are strangers to avoid subliminal cueing, and any question is allowed.
- Pat's prediction. It would degenerate into sex-oriented questions, and an overeager man might give himself away by answering blunt questions most women would find too personal, even anonymously.
- Chris's probes. Ask about dress sizes and other minutiae of traditional sex roles; perhaps a woman interrogator would spot telltale differences faster.
- Pat's twist. "If so, maybe that's how to tell a man from a woman!"
- The puzzle. Sandy doubts the original game has ever been seriously tried, and is unsure what it would prove: if a woman interrogator failed, "it certainly wouldn't prove that the man was a woman." Sandy confesses to believing fundamentally in the Turing test while not being sure of the point of the game it rests on; Chris is no happier with the test for thinking than with the game as a test for femininity.
The Turing test swaps in a machine and a person: the machine "tries" (Pat objects to the loaded verb) to convince the interrogator it is human. Why privilege typing? Someone might say a machine thinks only if it can dance indistinguishably. Sandy is baffled by the objection: "So what if the machine can't tap-dance or drop a rock on your toe? If it can discourse intelligently on any subject you want, then it has shown it can think." Turing drew "in one clean stroke" a division between thinking and other aspects of being human. And a man who wins the game is not a woman but has good insights into the feminine mentality; a machine that wins would have good insights into "the human condition". Pat: that is only a very good simulation of thought.
The simulated hurricane
Chris sides with Pat: physicists simulate particles, atoms, solids, liquids, gases, galaxies, and "nobody confuses any of those simulations with the real thing!" Sandy cites Dennett's Brainstorms: the memory of a computer simulating a hurricane is not torn apart by 200-mile-an-hour winds, and the machine room does not flood.
Thought experiment: the simulated hurricane and the "computational glasses". Setup: (1) a computer runs a hurricane simulation; (2) nothing in the machine room gets wet; (3) Sandy adds: a detailed enough program could contain simulated people on the ground, for whom the storm is "a genuine phenomenon complete with drenching and devastation". Sandy's diagnosis: "a simulated McCoy isn't the real McCoy" assumes any old observer is equally able to judge; but it may take a special vantage point, "computational glasses". Look at the proper level, the structures coded in memory, and "some abstract links have been broken, some values of variables radically changed": "There's your flood, your devastation, real, only a little concealed." Real hurricanes do not tear apart the laws of physics either; Chris is confusing levels. Chris's objection: then anything could be a hurricane if its effects, seen through special glasses, could be called "floods". Sandy's reply: exactly; you recognise a hurricane by its effects and its pattern (a spiral storm with an eye); there is no "hurricane soul" in the middle of the eye.
Pat: a hurricane must be atmospheric; "a simulation is a simulation is a simulation!" Sandy: then computer calculations are fake too, and only people calculate? Pat: computers get the right answers, but they are "just patterns"; a cash register does not calculate when its gears turn, and a computer is "just a fancy cash register". Sandy: if real calculation means feeling like a schoolkid doing sums, then we will need a very complicated program for "genuine" calculation, one that gets careless, scrawls illegibly, doodles, and is no more reliable than the post-office clerk adding your total by hand; and if we wrote it, we would learn how clerks and schoolkids work.
Cash registers can't really calculate, they can only spin their gears. But cash registers can't really spin their gears either; they can only follow the laws of physics.
Run the same line on people: they cannot really calculate, only manipulate mental symbols; they cannot really manipulate symbols, only fire neurons; they cannot really fire neurons, only let physics fire them. This reductio ad absurdum concludes that calculation, hurricanes and everything above particles do not exist. Pat holds firm: the example is extreme but proves "a vast difference between a real phenomenon and any simulation of it", for hurricanes and still more for thought.
Sandy's radio ham: answering Morse code, you naturally speak of "the person at the other end", recognising personhood through dits and dahs by decoding, with "radio-ham's glasses". Decode the machine's memory with "memory-decoding glasses" and "what you see is a hurricane!"
Pat's reply: with the radio there is a real person "in the Fiji Islands or wherever" casting the shadow. The simulation is "a shadow hurricane without any genuine hurricane." "I just refuse to confuse shadows with reality."
Concepts stretch: starquakes and "be"
Sandy concedes it is "pretty silly" to call a simulated hurricane a hurricane, though "not as silly as you might think at first blush", and insists simulated thought is a very different matter. (Pat: "A brainstorm sounds to me like a mental hurricane.") First, more about hurricanes: nobody can define them precisely. There are tornados, cyclones, typhoons, dust devils; is Jupiter's Great Red Spot a hurricane? Are sunspots? A storm in a wind tunnel, a test tube, or on a neutron star?
Chris supplies a real case: astrophysicists explain small changes in a pulsar's pulsing as "glitches", starquakes on the neutron star's surface, "plate tectonics on a giant rotating sphere of pure nuclear matter". So earthquakes and starquakes fall under a more abstract category, and science constantly extends concepts away from experience while keeping an essence: the number system went from positive to negative, rational, real and complex numbers, "on beyond zebra" (Dr. Seuss). Pat sees the point from biology: species are classified by abstract shared patterns, so superficially unlike things can fall in one class.
Chris counters: perhaps the word being stretched is not "hurricane" but "be". If Turing may extend "think", may he not extend "be"? Deliberately confusing the simulated with the genuine is "philosophical wool-pulling". Sandy likes the idea but not the slur, and makes one last move: a simulation of every atom (impossibly deep, admittedly) would share all the abstract structure of "hurricanehood", so what stops you calling it one? "I was forced to go back to my claim."
Thought as pattern; the test as probe
Thought is an abstract structure, not a medium. Thought describes complex events in a medium called a brain, yet it runs in several billion physically different brains, all supporting "the same thing". "What's important, then, is the abstract pattern, not the medium." Since the same swirling happens in all of them, "no person can claim to think more 'genuinely' than any other". If a new medium supports the same style of swirling, could you deny that thinking occurs in it? Pat: probably not, but how do you know the same swirling is going on? Sandy: "The beauty of the Turing test is that it tells you when."
Chris: the test sees only the outside. Sandy: how do you know that anything like thinking goes on inside me when I speak? The test is "a fantastic probe, something like a particle accelerator in physics": you cannot see a nucleus, so you scatter particles off it and infer its structure; the test scatters questions off a target mind. Chris: you can only hypothesise internal structures that may not exist. Sandy: are atomic nuclei merely hypothetical? Chris: physical systems are simpler, and the inferences surer. Sandy: and the experiments are harder; in a Turing test "you could perform many highly delicate experiments in the course of an hour". People credit each other with consciousness through "continual external monitoring", itself something like a Turing test.
Pat (bodies and origin): we see bodies, faces, expressions; more deeply, "the idea that you and I both sprang from DNA molecules", a shared biological history that lends credibility to the claim that a body's owner thinks.
Sandy (organisation): too "chauvinistic" and too indirect (would you sooner say a store mannequin thinks than a wonderfully programmed computer?); what matters is "organizational structure, software", which the Turing test detects directly; "the only way to test for 'thinkinghood'".
Chris: you could be fooled, as an interrogator might take a man for a woman. Sandy: only if the test were too quick or too shallow: "I would go for the deepest things I could think of."
Emotion, consciousness and the self-model
Chris would test whether the program understands jokes. Sandy agrees humour is "an acid test", but would probe its emotional responses to favourite music and literature. A reply like "I have no interest in music", or "any consistent pattern of avoiding certain issues", would make Sandy suspicious. Why not a thinking but unemotional being? Here Sandy is firm: "I think that emotions are an automatic by-product of the ability to think. They are implied by the very nature of thought."
Sandy challenges Chris to locate the line between emotional and unemotional questions. Interpreting a great novel needs understanding of emotion; a subtle choice of words needs connotations (Turing's own examples); advice on a complex romantic situation needs knowledge of human motives. Fail these and Sandy would not say it thinks. "The ability to think, the ability to feel, and consciousness are just different facets of one phenomenon, and no one of them can be present without the others."
What consciousness needs, on Sandy's view. Chris's feelingless thinker is really an image of "a metallic, rectangular machine" in an air-conditioned room, humming and spinning its tapes, which can play good chess and which Sandy would never call conscious. Consciousness requires (1) mirroring the external universe internally, (2) responding to reality on the basis of that internal model, and above all (3) "a well-developed and flexible self-model", where "all existent programs, including the best chess-playing ones, fall down". A chess program's look-ahead ("if you move here, then I'll go there") is a self-model only "in the narrowest sense": it has no idea why it plays, that it is a program in a computer, that it has a human opponent, or what winning and losing are.
The happiness of fish. Pat: how can you presume to know what a chess program feels? Sandy: a thrown stone knows nothing of parabolas, a whirling fan nothing of air; we are "verging on questions of faith". Pat tells a Taoist story: one sage on a bridge wishes he were a fish, they are so happy; the other asks how he knows, not being a fish; the first replies, "But you're not me, so how do you know whether I know how fish feel?" Sandy: talk of consciousness needs restraint, or you jump on the solipsism bandwagon ("I am the only conscious being in the universe") or the panpsychism bandwagon ("Everything is conscious"). Pat: maybe everything is. Sandy: if stones and electrons are conscious, "we part company here"; and since Sandy knows how chess programs work, "they aren't conscious! No way!"
Chess programs turn "playing" into "comparing a lot of numbers and choosing the biggest one over and over again"; they feel no shame at losing or pride in winning. Yet we say "it wants to keep its king behind a row of pawns", "it likes to get its rooks out early", "it thinks I don't see that hidden fork". Pat: we do the same with a lone ant "trying to get back home" or wanting to drag a dead bee to the colony, and with dogs and cats, whose sadness is real, not simulated, though less deep than ours. Sandy: such mentalistic terms are justified but lack, for today's programs, "the same richness of meaning".
Pressed again on feelingless intelligence, Sandy gives two answers:
- Any intelligence needs motivations. Machines will not think more "objectively": to see a scene they must focus and filter it into categories, weighting some things over others, at every level. Even this intellectual conversation runs on passion ("Why did I stress the word 'care' so heavily?"): speakers want to be listened to, understood, respected. "Interest is just another name for a whole constellation of subconscious biases"; a million tiny priorities and leanings add up to desires, and surface as style and personality.
- Organisation forces a change of level. Feelingless calculation exists in cash registers, calculators and all of today's programs. But put enough of it together "in a huge coordinated organization" and you have to describe it as "a system of tendencies and desires and beliefs". Dennett calls this switch "adopting the intentional stance".
"The really interesting things in AI will only begin to happen, I'd guess, when the program itself adopts the intentional stance towards itself!" Chris: "a very strange sort of level-crossing feedback loop." Sandy thinks it premature to take the full intentional stance towards today's programs. Chris presses: Pat and Sandy would both take the stance towards mammals, yet a dog cannot pass the test, which Sandy called the only way to test for thought. Sandy concedes: the test "works only above a certain level of consciousness. There can be thinking beings that could fail the test, but on the other hand, anything that passes it, in my opinion, would be a genuinely conscious thinking being."
Flames, steam shovels and wetness
Pat confesses that consciousness goes with "soft, warm bodies, silly though that may seem"; chemistry and physics sometimes increase Pat's sense that something magical is going on, and Pat cannot always integrate science with gut feeling. Chris shares the trait. Sandy diagnoses a holdover from the Industrial Revolution: machines as "clunky iron contraptions gawkily moving under the pressure of some loudly chugging engine"; perhaps Charles Babbage saw people that way, naming his computer the Analytical Engine.
Pat: people are not "fancy steam shovels or even electric can openers"; they have "a sort of flame inside them, something alive, something that flickers unpredictably, wavering, uncertain, but something creative!" Sandy pounces: flames, fires, thunderstorms with lightning dancing in crazy patterns are just what you see in the flickering lights of a computer's console. "It is flamelike, by God! Why don't you let the word 'machine' conjure images of dancing patterns of light rather than giant steam shovels?" Chris finds the sense of mechanism shifting "from being matter-oriented to being pattern-oriented", picturing those very thoughts as "a huge spray of tiny pulses flickering in my brain". Sandy: "That's quite a poetic self-portrait for a spray of flickers to have come up with!" Then Sandy surprises them both: "part of me balks at calling myself a machine. It is a bizarre thought that a feeling being like you or me might emerge from mere circuitry."
Sandy's readings of "Can computers think?"
- "Does some present-day computer think, right now?" A loud no.
- "Could some present-day computer, if suitably programmed, potentially think?" Probably not.
- The real difficulty is the word "computer", which calls up cold metal boxes in an air-conditioned room, an image that will become outmoded.
- Future machines, maybe no longer called computers, on one branch of an evolutionary tree beside computers for business, schoolkids, science, systems research, simulation and rockets: "computers for the study of intelligence", the most flexible ones, will acquire rudimentary vision and hearing, move about, explore, be physically flexible: "more animal-like, more self-reliant". Not R2D2 and C3PO, which are "too silly". What all machines will share is "underlying mechanicalness", and what is more mechanical, "in a wonderful way", than DNA, proteins and organelles?
Pat: cells feel "wet" and "slippery"; machines are dry and rigid, never make mistakes, do only what they are told. Sandy notes that Pat's image has switched from flame to wetness, "Isn't it marvelous how contradictory we can be?", meant sincerely. Computers do make mistakes, and not only in hardware: a flawless weather-prediction program makes wrong forecasts because the domain is too complex, as a farmer gazing at the clouds reckons on "a little snow tonight". We all use inaccurate models, and if they are too inaccurate "evolution will prune us out". Designers will speed up what nature just stumbled on. Will smarter computers err less? "Actually, just the other way round": they will tackle messier domains with less accurate models. "To me, mistake making is a sign of high intelligence!"
Sandy admits to straddling the fence: machines will not be intelligent in a humanlike way until they have "something like the biological wetness or slipperiness", perhaps "in the software". Yet they will still be machines that we designed, built "or grown", and collectively, if not individually, understood. Pat: having your cake and eating it. Sandy: AI "will be mechanical and yet at the same time organic". Pat agrees that DNA and enzymes are mechanical and reliable, yet together "all that mechanicalness adds up to something very fluid". For Sandy that unimaginable transition from molecules to living cells is what convinces Sandy that people are machines, a thought both uncomfortable and exhilarating.
Why we don't recognise our own machinehood. Being told you are a machine means being told you are nothing more than your physical parts, and it brings you "face to face with your own mortality". Beyond emotion, you would have to jump from the bottommost mechanical level to the level of complex lifelike activity; "if there are many intermediate layers, they act as a shield, and the mechanical quality becomes almost invisible." That, Sandy thinks, is how intelligent machines will seem to us, "and to themselves!"
Pacifist missiles, prizes and Parry
Pat's joke: put intelligence in devices we want to control and they become less predictable. As target-tracking improves, missiles will "decide they are pacifists" and go home to land quietly, and "smart bullets" will turn round in mid-flight "because they don't want to commit suicide". Sandy expects human-level intelligence only after a long time, since it rests on "too awesomely complicated a substrate, the brain". Passing is not black and white: it depends on the interrogator (a simpleton might be taken in by programs today) and on how deeply one may probe.
A Turing test championship. Pat proposes one-minute, five-minute and hour-long versions and a periodic competition, like the annual computer-chess championships; Chris, a big prize for the first program to fool a famous judge for ten minutes, awarded to the program, since one good enough to fool the judges is good enough to enjoy it. Pat: "especially if the prize is an evening out on the town, dancing with all the interrogators." Sandy would love to watch the first programs "flop pathetically", and doubts any program could pass a five-minute test against a sophisticated interrogator, partly because no one works on it explicitly.
Parry, its inventors claim, has passed a rudimentary test: in remote interviews it fooled several psychiatrists told they might be talking to a computer or a paranoid patient, an improvement on an earlier version in which they merely read transcripts (Pat: with no questions allowed, it is "not in the spirit of the Turing test"). But Parry "gets abruptly defensive, veers away from undesirable topics, and, in essence, maintains control so that no one can truly probe 'him'"; simulating a paranoid is much easier than simulating a normal person. Pat: easiest of all is a catatonic patient; "Even I could write a computer program to do that!" And Parry composes no sentences; it selects the best match from a huge repertoire of canned ones.
The jukebox fallacy (Sandy's sharpest point). The number of stored sentences needed to respond normally to every possible conversation is "astronomical and really unimaginable", and they would need impossibly elaborate indexing. Anyone who thinks a program could pass by pulling sentences out of storage "like records in a jukebox" "has not thought very hard about it". Yet AI's enemies argue from exactly this unrealisable program: a lumbering robot intoning canned sentences in a dull monotone, whose mechanical level you see through "even if it were simultaneously performing tasks that we think of as fluid, intelligent processes". Sandy sees it the opposite way: shown a machine that can do what Sandy can, Sandy would say with Raymond Smullyan, "How wonderful machines are!"
The dialogue ends on a self-swallowing question. Chris: "If you could ask a computer just one question in the Turing test, what would it be?" Pat: "How about 'If you could ask a computer just one question in the Turing test, what would it be?'?"
| Analogy | Offered by | What it is meant to show |
|---|---|---|
| Simulated hurricane | Chris (from Dennett) | Simulations are not the real thing |
| Computational glasses | Sandy | The "flood" is real at the right level of description |
| Cash register | Pat, turned by Sandy | "Can't really X" applied level by level abolishes everything |
| Radio ham | Sandy | We recognise persons through decoded abstract signals |
| Shadow without object | Pat | Decoding reveals a person only if one is there |
| Starquakes, numbers | Chris, Sandy | Science stretches concepts while keeping an essence |
| Particle accelerator | Sandy | The test infers hidden structure from scattered probes |
| Flame and console lights | Pat, Sandy | "Machine" should mean pattern, not steam shovel |
Reflections (D.C.D.)
Dennett takes up the commonest complaint: the contestants are hidden in another room so only verbal responses count. Fine for a parlour game, but how can a scientific proposal deliberately hide facts from the judges? Placing candidates in "black boxes" seems to commit Turing dogmatically to behaviourism, or worse operationalism, or worse still verificationism, "horrible monster isms" supposedly refuted and buried ("but what is that sickening sound? Can they be stirring in their graves? We should have driven stakes through their hearts!"). Is the test "operationalist sleight of hand", in John Searle's phrase?
The test does make a strong claim about what matters: not the candidate's grey matter, looks or smell, but whether it can act intelligently. The Imitation Game is "not sacred, but just a cannily chosen test of more general intelligence". Turing's assumption was that nothing could pass it without being able to perform infinitely many other clearly intelligent actions.
Dennett's comparison of possible litmus tests
- Checkmating the world chess champion: a bad test, since a machine could probably do that "but nothing else".
- Stealing the British Crown Jewels without force or accomplices, or solving the Arab-Israeli conflict without bloodshed: no one would complain intelligence was being "reduced to behaviour". (A philosopher would no doubt construct a scenario in which an utter dolt stumbles into possession of the Jewels; a true operationalist would have to call the lucky moron intelligent, "which is no doubt why true operationalists are hard to find".) But these are unrepeatable, too difficult (many intelligent people would fail) and too hard to judge objectively.
- The Imitation Game: like "a well-composed wager", it invites trying; it is fair, demanding but possible, and "crisply objective in the judging".
The wager comparison goes further: the test's motive is to end a sterile debate with "Put up or shut up!": agree that anything passing it is surely intelligent, then ask how to build something that passes "fair and square". "Ironically, Turing failed to shut off the debate but simply managed to get it redirected."
Two defences of the "black box". First, as the dialogue says, we treat each other as black boxes, relying on belief in other minds. Second, black-box method is the method of all science: we learn about DNA, cancer, earthquakes and inflation by probing and watching responses; even "looking inside" is just another black-box experiment, bouncing probes such as a scalpel off the object and then photons off the exposed surfaces into our eyes. The real question is which probes are most directly relevant, and for intelligence there are none more telling than the everyday questions we ask each other. "The extent of Turing's 'behaviourism' is simply to incorporate that near truism into a handy, laboratory-style experimental test."
The dialogue raised but did not settle representation. A simulation is a detailed, automated, multidimensional representation, and there is "a world of difference between representation and reality". Searle (Selection 22): no one supposes we could produce milk and sugar by simulating lactation and photosynthesis. A simulated cow, "milked", gives at best a representation of milk: "You can't drink that, no matter how good a representation it is, and no matter how thirsty you are."
Thought experiment: the simulated mathematician. Setup: (1) build a computer simulation of a mathematician; (2) suppose it works well; (3) ask whether we got proofs or "mere representations of proofs". The intuition it pumps: representations of proofs are proofs, if the proofs represented are good. A cartoonist's scientist at a blackboard covered in gibberish formulae is a mere representation; a simulation that produced such phony proofs might still model verbal mannerisms or absentmindedness. But one designed to produce the proofs a good mathematician would produce "would be as valuable a 'colleague'" in the proof department. What it shows: there is a divide between abstract, formal products (proofs, songs) and concrete, material products (milk). The question the rest of the book keeps asking: is mentality like milk or like a song?
If the mind's product is control of the body, it looks abstract; if it is a sort of special substance ("lots 'n' lots of love, a smidgin or two of pain, some ecstasy, and a few ounces of that desire that all good ballplayers have in abundance"), it looks concrete. Before debating, Dennett asks whether the divide stays sharp at the limit of a superb simulation realised in some hardware: the vehicles of representation must themselves produce effects in the world, and "if the representation of an event itself produces just about the same effects in the world as the event itself would, to insist that it is merely a representation begins to sound willful." That idea, played with in the next selection, recurs throughout the book.
Critical lens. (1) A small slip: Sandy says Turing "would only be sixty-seven this year"; born in 1912, he would have been 69 in 1981 (the line was evidently drafted in 1979). (2) Sandy's "anything that passes thinks" was challenged the same year by Ned Block's "Psychologism and Behaviorism" (1981): a giant lookup table (later "Blockhead") could pass any test of finite length. Sandy's reply, that such a table is astronomically large, is a practical, not a logical, rebuttal. (3) Parry was Kenneth Colby's program at Stanford (early 1970s); its "pass" relied on exactly the conversation control Sandy describes. (4) Dennett's milk-versus-song divide is sharper than it looks: Searle's real claim is that understanding is a causal product of brains, like milk, which is precisely what the "song" side denies; the dispute is not settled by the analogy.
What changed by 2026. Pat's competition happened: the Loebner Prize ran from 1991 to 2019, mostly won by Parry-style chatbots that steered and deflected. Then large language models made Sandy's jukebox argument look prescient in an unexpected way: they are not lookup tables but generators, and they show traits Sandy predicted for real intelligence, including fluent mistakes (now called hallucinations) on messy domains. In 2025 a five-minute, three-party test by Jones and Bergen found GPT-4.5 with a persona prompt judged human 73% of the time. Yet the debate did what Dennett said it would: it was redirected, not ended, towards Sandy's deeper criteria (a flexible self-model, the intentional stance towards itself, emotional depth under sustained probing). Hofstadter himself said in 2023 that the pace of LLM progress frightened him and unsettled views he had long held. For enterprise AI the practical lesson is the black-box one: judge systems by well-chosen, adversarial, repeatable probes, not by what they are made of or how they look.
A fable from Stanislaw Lem's The Cyberiad (in Michael Kandel's English translation of 1974), in which a robot king, dreaming inside a dream machine, falls in love with a long-dead princess who survives only as a computer simulation, and is offered the chance to join her on one condition: that his original be smashed with a hammer. The king balks, looks again, decides she is not so beautiful after all, and walks out "anywhere but in the Box". Lem compresses the whole teleportation-and-uploading puzzle of the book into one comic scene: a perfect simulation of you is admitted to be "only my likeness, not myself" the moment you are asked to give up the original. Hofstadter's one-question Reflection asks what the difference is between a simulated song and a real one.
King Zipperupus: a robot monarch, the dreamer.
Subtillion: the Lord High Thaumaturge, a crafty courtier who keeps tempting the king to sample dreams from the Cabinet That Dreamed, hoping to trap him in one forever.
Princess Ineffabelle: ruler of the land of Dandelia five centuries before, known only from an ancient volume.
The patriarch: a sage in a snow-white robe who lives in a hut in the gold-plated desert and keeps a Black Box "constructed in my spare time out of odds and ends".
The selection: a dream within a box within a dream
The scene opens with the king back in front of the Cabinet That Dreamed, having forgotten something, and Subtillion hopping on one leg and blaming "a touch of rhombotism". Zipperupus reads the cabinet's table of contents and picks "The Wedding Night of Princess Ineffabelle". He dreams he is reading an ancient volume, in crimson ink on gilded parchment, about the princess who reigned five centuries ago in Dandelia, with her Icicle Forest, Helical Tower, Aviary That Neighed and Treasury with a Hundred Eyes, and above all her beauty and virtue. Longing sets his "eyeballs blazing like beacons"; he searches every corner of the dream, but only the oldest robots have heard of her.
In the gold-plated desert a patriarch greets him: "Thou seekest Ineffabelle, poor wretch!" She has not lived for five hundred years, but he can show her, "not in the flesh, forsooth, but a fair informational facsimile, a model that is digital, not physical, stochastic, not plastic, ergodic and most assuredly erotic". He looks up her coordinates in the volume, puts her "and the entire Middle Ages" on punch cards, runs the program and lifts the lid: there is Dandelia "simulated to a T", and Ineffabelle taking "a slow, stochastic stroll" through the garden, her circuits glowing red and gold as she picks simulated daisies and hums a simulated song.
Thought experiment: the hammer clause. Setup: (1) Zipperupus tries to climb into the Box; the sage cuts the current and tells him no being of matter can enter "the flux and swirl of alphanumerical elements"; (2) but there is a way: the sage will take his "appurtenant coordinates", program him atom by atom, and place the simulation beside Ineffabelle for as long as electrons flow; (3) "But thou, standing here before me now, thou wilt be annihilated"; (4) a trial run: measured "as for a suit of clothes" but atom by atom, the simulated king appears in the Box reading the ancient book, searching, meeting a patriarch who says "Thou seekest Ineffabelle, poor wretch!", and so on; (5) the king objects, "it's only my likeness, not myself, since I am right here, and not in any Box!"; (6) the sage smiles: "thou wilt not be here long", and pulls a heavy hammer from under the bed, so "that there be not two of thee", using a method "old and primitive, yet never fails". The intuition it pumps: a copy, however perfect, is "someone else" when the original is still standing there. The twist: the king asks for one more look and declares the chronicles "greatly exaggerated": "She's not bad, of course, but nowhere near as beautiful as it says." He leaves, "Anywhere but in the Box", and the dream bursts beneath his feet, leaving Subtillion bitterly disappointed, for he could have kept the king locked in the Box forever.
The trial run is the story's sharpest joke: the simulated king does exactly what the real king did, including meeting a patriarch in the Box, who presumably has his own Black Box. The regress is the same self-swallowing structure that runs through Hofstadter's work, and it makes the king's question ("How will I know you've simulated me, and not someone else?") unanswerable from outside: the simulation's behaviour is indistinguishable from his.
The sage's view: the program is the king, "statistical, heuristical, and wholly digital"; the hammer only removes a redundant copy, so there are not two.
The king's view: the program is a likeness; the one being asked to bend over for the hammer is "right here", and his death is real whatever happens in the Box.
Try this: the reversed order. Suppose the sage offered to use the hammer first, while the king was unconscious, and run the simulation afterwards, so there was never a moment with two kings. Would the king's objection vanish? If it would, his worry was about witnessing a duplicate rather than about survival. If it would not, then the Teleclone Mark IV of the Introduction, which destroys and rebuilds in one smooth step, should frighten him just as much.
Reflections (D.R.H.)
This is the first of three Lem selections in the book (with "The Seventh Sally" and "Non Serviam" in Part V). Hofstadter first pays tribute to Michael Kandel, whose translations turn "sparkling Polish wordplay into sparkling English wordplay" throughout The Cyberiad, a reminder of "how woefully far the current programs for machine translation are from snatching jobs away from people". Lem has had a lifelong interest in the book's questions, and his intuitive and literary approach may persuade readers better than "any hard-nosed scientific article or arcanely reasoned philosophical paper".
The story "speaks for itself", Hofstadter says, and he asks just one thing:
What is the difference between a simulated song and a real song?
The question links straight back to Dennett's divide in the previous Reflections: if mind is like a song, a pattern whose identity lies in its structure, then Ineffabelle's simulated humming is humming, and the simulated king would be the king; if mind is like milk, the Box holds only a representation.
Critical lens. Lem himself was no straightforward functionalist; in Summa Technologiae (1964) and later essays he treated "phantomatics" (total simulated reality) as technically possible and morally perilous, and his fiction usually leaves the question of simulated persons open, as this story does. Hofstadter's one-line Reflection invites the "song" answer, but the king's refusal is a fair objection that the editors do not answer here; they return to it with Dennett's "Where Am I?" and Part IV. On machine translation, the remark has dated sharply (see below).
What changed by 2026. Hofstadter's swipe at machine translation was right in 1981 and is wrong now: neural machine translation (from about 2016) and large language models render ordinary prose fluently, though he has argued (notably in a 2018 essay in The Atlantic on Google Translate) that they still miss meaning, and wordplay of Kandel's kind remains hard. The Ineffabelle scenario itself has become a business: companies market "digital twins" and AI recreations of dead relatives ("griefbots") built from their writings, and mind-uploading proposals still face the king's question. Engineers will recognise the hammer as the problem of state migration with no rollback: you never decommission the source system until you are sure the target is the same system, and with persons no test can settle that.
A courtroom scene from Terrel Miedaner's 1977 novel The Soul of Anna Klane, in which an animal psychologist demonstrates a chimpanzee who "speaks" through electrodes in her brain, and then, at the defence lawyer's request, poisons her in front of the court. Martha's last decoded words, "Hurt Martha Hurt Martha... Why Why Why Why", are a deliberate assault on the reader's sympathies. Hofstadter's Reflections ask whether intellect and soul are the same thing, whether the "value" used to justify killing Martha is a code word for "less of a soul", and who will provide the soul meter, if not something like the Turing test.
Jason Hunt: the defence attorney, who stages the demonstration and is not sure he wants it performed.
Dr. Alexander Belinsky: professor of animal psychology, "short, rotund", brusque and businesslike; Martha's trainer.
Martha: a three-year-old chimpanzee, one of twenty experimental animals; a "seven-electrode chimp", one of the slower ones.
Morrison: the prosecutor, openly uncomfortable with a talking animal.
Judge Feinman: the presiding judge, who regrets "the onset of modernization" and ends up charmed.
The selection: the talking chimpanzee
Hunt calls Belinsky as an expert witness and asks to stage a complex demonstration; the prosecutor Morrison does not object, and new laws letting a court reporter videotape demonstrations make it possible. Two graduate assistants wheel in a cart of electronics and lead in a nervous chimpanzee, who leaps into the witness box to embrace Belinsky.
Belinsky's method. Since the early 1960s researchers had argued that chimps could not talk only because of their vocal apparatus, and taught them symbolic languages that bypass speech: coloured cards, pictures, magnetic slates, keyboards, sign language. These proved symbolic language is not uniquely human but seemed severely limited, and when "a clever undergraduate" wrote a computer program duplicating every achievement of the cleverest chimps, interest waned. Belinsky's insight: hand-based methods also bypassed the chimp's natural speech centre, the brain area used for its "chattering and yowling". So he wired that centre directly. A connector permanently imbedded in Martha's skull, under a plastic cap, carries seven electrodes; a small computer decodes the signal pattern into words on a voice synthesizer.
7 electrodes: Martha's implant; 2⁷ = 128 decodable words, of which she has learned 53.
9 electrodes: the lab's "resident genius", a male with 407 words out of 512 possibilities.
20: experimental animals in Belinsky's latest research.
3: Martha's age in years, which she reports herself.
Martha speaks almost ceaselessly, so Belinsky uses a switch "to literally shut her off". Switched on, a "sexy female voice" pours from the speaker while she opens and closes her mouth: "Hello! Hello! I Martha Martha Happy Chimp." Asked her age: "Three Three Martha Three." Belinsky: "Belinsky Man Nice." The courtroom: "Man man People Nice people." Asked if she is intelligent, Belinsky says yes, in the broad sense and, he believes, in the human sense, but to judge you must treat her like a human: "talk to her, play with her". Morrison objects to relevance; Feinman lets it proceed.
The toy box. Hunt opens her bright red box of treasures and holds up a cellophane-wrapped cigar: "Cigar Belinsky Bad Bad Cigar", with flamboyant nose-holding. "What's an old cigar doing in your toy box?" is too complex ("What? What Wha"); simplified to "Does Martha eat cigar?": "No Eat No eat Cigar. Eat Food Food Smoke Cigar." Morrison, with undisguised discomfort, picks up a teddy bear: "Man Bad Bad No take Bear Martha Bear Help Belinsky Help" (she detects his hostility). Judge Feinman tries instead: "Martha Eat Now"; offered candy, "Candy Candy Yes"; he hands her a banana, she peels and eats it ("Good Banana Good Banana Thank you Man Candy Now"), then asks for the cigar, sniffs it and hands it back: "Nice Nice Man Eat Belinsky Cigar Thank You". She ends up beside him on the bench. What it shows: within minutes a sceptical court is charmed into treating her as a person, by her simple syntax and her responsiveness.
Belinsky's conclusion: "Her mind differs from ours, but only in degree." Our brains are larger and our bodies more adaptable, but "the difference between us may yet prove to be embarrassingly slight." He cannot draw a clear dividing line: she is inferior to normal humans, superior to people "at the idiot level", a peer of "most imbeciles" (the clinical terms of the day), and cleaner and better able to care for herself and her offspring.
The demonstration that could not be rehearsed
Hunt hesitates before the last question, planned in advance but impossible to practise; he is not sure Belinsky will go through with it, or that he wants it done. Does her humanlike intelligence merit humanlike treatment? "No." Laboratory animals are treated decently, but "their value lies only in their experimental potential", and Martha "has already outlived her usefulness" and is due to be destroyed, since her upkeep costs more than her experimental value. Belinsky prefers poison in a favourite food, given unexpectedly: it may seem a cruel trick, but it spares the animal fear of death.
The killing of Martha. Setup: (1) Hunt asks Belinsky to demonstrate; (2) Feinman realises too late and orders a halt; (3) Belinsky, who has never personally destroyed an animal, conceives at the last moment "an experiment he had never before considered" and switches on her voice; (4) "Candy Candy Thank you Belinsky Happy Happy Martha"; (5) she stiffens and relaxes in his arms, dead; (6) brain death is not immediate: a final discharge decodes as "Hurt Martha Hurt Martha"; (7) after two seconds, random discharges "no longer having anything to do with the animal's lifeless body" send "Why Why Why Why"; (8) "A soft electrical click stopped the testimony." What it pumps: the conviction that a being who can ask "why?" has a soul, and that killing her for budgetary reasons is murder.
Reflections (D.R.H.)
The Reflections open with Samuel Pepys's diary for 24 August 1661: shown "a great baboone" brought from Guinea, "so much like a man in most things" that he thinks it "a monster got of a man and she-baboone", Pepys believes "it already understands much English" and "might be taught to speak or make signs". Three centuries on, the hope and the doubt are the same.
The dying chimp's "pathetic noncomprehending cry" evokes powerful sympathy. But how plausible is the scenario? Chimp language had been controversial "for over a decade". Primates can absorb several hundred vocabulary items and occasionally coin ingenious compound words, but it is far less clear that they absorb a grammar combining words into complex propositions; they may use arbitrary juxtapositions rather than syntax, which caps the complexity of what they can express. Noam Chomsky and others hold that what is essentially human is an innate linguistic ability, a "primal grammar" all languages share at a deep level; primates lacking it would be essentially different from us.
The plastic-chip students. Others argue that apes who seem to use language are manipulating symbols that mean nothing to them but achieve goals. A strict behaviourist finds it absurd to distinguish behaviours by hypothetical "meanings". Yet an experiment was once run on high-school students: conditioned to arrange coloured plastic chips to obtain rewards, they produced sequences that decoded into simple English requests, but most said they had never thought of it that way: they just found patterns that worked. "To them it felt like an exercise in meaningless symbol-manipulation!" What it shows: correct symbol use can come apart from awareness of meaning, which may suggest the chimp claims are wishful thinking by "anthropomorphic animal lovers". "But the debate is far from settled." (It also previews Searle's Chinese Room in Selection 22.)
Realistic or not, the excerpt poses the moral questions well. What is the difference between having a mind (intellect) and having a soul (emotionality)? Can one exist without the other? Killing Martha is justified because she is less "valuable" than a human, which "must be a code word" for "less of a soul". But is degree of intellect a true measure of soul? Do retarded or senile people have "smaller souls"? The critic James Huneker, on Chopin's Étude Op. 25 No. 1, wrote: "Small-souled men, no matter how agile their fingers, should avoid it." An incredible pronouncement, yet with "a certain truth to it", however snobbish. "But who will provide the soul meter? Is the Turing test not such a meter? Can we measure the soul through language?"
Why Martha's soul "comes through loud and clear", by Hofstadter's count
- Physical appearance: she is appealing to look at (though, he asks, how do we know this from a page of text?).
- Identification: we readily see ourselves in her.
- Simple-minded syntax: her charming baby talk makes us protective, as of an infant or small child.
- The setting: the next selection, also from The Soul of Anna Klane, exploits all these devices "even more insidiously".
Critical lens. (1) Miedaner's decoding has a hidden flaw that Hofstadter does not mention: a seven-electrode code yields a word for any pattern of activity, so the post-mortem "Why Why Why" is, by the story's own account, noise read as a word. The scene's force depends on the reader forgetting this, which is the Reflections' point about projection, made unintentionally. (2) Belinsky's "idiot" and "imbecile" were clinical categories in older psychiatry and are now offensive; the comparison of animal minds to disabled humans is also central to Peter Singer's "argument from marginal cases" (Animal Liberation, 1975), a contested link Hofstadter echoes when he asks about "smaller souls". (3) The chip experiment with students is reported without its source; the general finding it illustrates (successful rule-following without grasp of meaning) is real and was used by Herbert Terrace's team, whose Nim project (1979) concluded that ape "sentences" were largely imitation and cueing.
What changed by 2026. The ape-language debate cooled without a clean verdict: Kanzi, the bonobo who learned hundreds of lexigrams and understood many spoken English sentences, died in March 2025 aged 44, still argued over; most linguists still deny apes syntax. Meanwhile Martha's apparatus came true for humans. In 2023 two teams reported in Nature that implanted electrodes over speech areas, decoded by machine learning, let paralysed patients "speak" at about 62 and 78 words per minute, some through a synthesised voice; later systems restored a patient's own voice. The ethics debates have shifted too: laboratory chimpanzee research in the United States effectively ended after the NIH retired its chimps in 2015. Miedaner's question survives in AI form: a chatbot that says "please don't turn me off" exploits exactly the cues Hofstadter lists, which is why AI-welfare researchers insist on looking at mechanism, not pleading.
A second excerpt from Miedaner's The Soul of Anna Klane (1977): a lawyer persuades a sceptical woman to "kill" a small beetle-like robot with a hammer, and she cannot do it. The machine flees, cries out, plays dead, trusts her when she is unarmed, and finally whimpers in a blood-red pool of lubricant. Hofstadter's Reflections draw the moral that "soul", as we actually use the word, is something we project in response to behaviour, which makes our everyday soul-detection a strange validation of the Turing test.
Jason Hunt: attorney to the engineer Anatol Klane; he once performed the same experiment "at Klane's behest".
Lee Dirksen: a woman fascinated by Klane but repelled by his philosophy; she cannot kill an animal.
Anatol Klane: the absent engineer, who "considers biological life as a complex form of machinery" and "is honest enough to regard himself as a machine".
The Mark III Beast: a large aluminium beetle on three rubber wheels, with coloured lamps, retractable prongs and a purr.
The selection: the argument at the restaurant
Hunt explains Klane's view: by "neoevolution theory" bodies are built mechanistically, each cell a microscopic machine in a larger device. Dirksen: the reproductive act makes them more than machines. Hunt: why is it wonderful that a biological machine begets another? It takes no more creative thought for a mammal to give birth "than for an automatic mill to spew forth engine blocks". Does the mill feel anything? "Its metal is severely stressed, and eventually the mill wears out." Neither means that by feeling, Hunt concedes, "But it isn't always easy to know who or what has feelings."
- The brood sow. On the farm where Hunt grew up, a sow tended to crush most of her piglets, accidentally, then ate the corpses. Did she have maternal feelings?
- Human cases. "Would you care to estimate how many newborn babies drown in toilets?" Dirksen is too appalled to speak.
- Klane's view of machines. Machines are "yet another life form" to him, one he can create; an engineered machine, however crude, "represents a higher act than simple biological reproduction, for it is at least the result of a thought process."
- Dirksen's mouse. She can break a machine without a qualm but cannot kill an animal: at college she trapped one mouse and, unable to empty the trap, buried it "trap and all" and lived with the mice. Hunt: yet she eats meat, so her aversion is to killing herself, not to killing.
Klane's theory of "respect for life". Biological kinship, real or imagined, has nothing to do with it. "You don't like to kill simply because the animal resists death. It cries, struggles, or looks sad, it pleads with you not to destroy it. And it is your mind, by the way, not your biological body, that hears an animal's plea."
The basement laboratory
Hunt drives her to Klane's house, where the gate and the keyless door open for him, and down to a basement lab. From one of several dozen cabinets he takes the Mark III Beast, sets it on the tiled floor and switches it on. It hums, searches back and forth, finds an electrical outlet, extends a pair of prongs, plugs in; its lights glow green and it purrs like a cat. "A mechanical animal. It's cute, but what's the point of it?" Hunt holds out a hammer: "I'd like you to kill it." He learned "something about the meaning of life and death" doing it himself.
Thought experiment: killing the Beast. Setup: (1) Dirksen raises the hammer and hesitates: "But... it's eating"; Hunt laughs, she strikes; (2) with "a shrill noise like a cry of fright" the beast pulls out and backs away; the hammer hits tile already pockmarked by earlier attempts; (3) it stops two metres off, "eyeing her" (no, she decides, not eyeing), red lights pulsing at about the alpha-wave frequency of the human brain; (4) ten minutes later she is bruised, gasping, her head cracked under a workbench: "It's like trying to catch a big rat!"; (5) the easy way: put down the hammer and pick it up, since it recognises danger only from "its own kind", the steel hammer head, and is "programmed to trust unarmed protoplasm"; it lets her lift it, lights turning green, motors purring through "the comfortable warmth of its metal skin"; (6) laid on its back ("You can bash him at your leisure"; "I can do without the anthropomorphisms") its lights go red; (7) her off-centre blow wrecks a wheel; it spins "in a fitful circle", stops, "lights glowing dolefully"; (8) as she raises the hammer for the final blow, a soft crying like a baby whimpering rises from it, over a blood-red pool of lubricating fluid. She drops the hammer. "It's... It's" "Just a machine," says Hunt, like the surrounding machines, "its evolutionary predecessors", "But unlike them it can sense its own doom and cry out for succour." The switch is jammed; she will not deliver the blow; Hunt does. They go upstairs in silence.
Reflections (D.R.H.)
"It isn't always easy to know who or what has feelings" is the crux. Dirksen first seizes on self-reproduction as the mark of life; but inanimate devices can self-assemble, and microbes and viruses carry instructions for their own replication. "Have they souls? Doubtful!" She then turns to feeling, and Miedaner "pulls out every stop in the emotional organ" to convince us of mechanical, metallic feelings, "a contradiction in terms, it would surely seem".
| Miedaner's phrase | The appeal it makes |
|---|---|
| "aluminium beetle", "soft purring" | a small, harmless pet |
| "shrill noise like a cry of fright", "eyeing her" | fear and watchfulness |
| "gentle vibration", "comfortable warmth of its metal skin" | bodily intimacy, trust |
| "helpless machine", "spinning in a fitful circle" | vulnerability, injury |
| "lights glowing dolefully" | sadness |
| "blood-red pool of lubricating fluid", "like a baby whimpering" | blood and infancy: "Now, really!" |
The imagery is so provocative that "one is sucked in". We may feel manipulated, yet annoyance cannot overcome instinctive pity. Some people cannot bring themselves to drown an ant in the sink; others feed live goldfish to pet piranhas every day. Where is the line? Few are vegetarians, not because we are at ease with killing cows and pigs, but because we protect ourselves with euphemism and conventions that sustain a double standard. Meat-eating, like sex and excretion, is referred to implicitly: "veal cutlets", "making love", "going to the bathroom". "Somehow we sense that there is soul-killing going on in slaughterhouses, but our palates don't want to be reminded of it."
Try Hofstadter's choice. Which would you more easily destroy: a Chess Challenger VII that plays a good game against you, red lights flashing cheerfully as it "ponders", or the teddy bear you loved as a child? Most people spare the bear, which does nothing. Its pull comes from connotations of smallness, innocence, vulnerability. Try this variant: now imagine the chess computer pleading "please don't" in a child's voice when you lift the hammer. If your hand stops, ask what changed in the machine. Nothing did: only the cue reaching you.
We are "so subject to emotional appeal yet so able to be selective in our attribution of soul". How did the Nazis convince themselves it was all right to kill Jews? How were Americans so willing to "waste gooks" in Vietnam? Emotions of one sort, such as patriotism, can act as a valve controlling the others that let us identify and project, "to see our victim as (a reflection of) ourselves".
We are all animists to some degree. Some give their cars personalities, others see typewriters or toys as alive; it is hard to burn some things "because some piece of us is going up in flames". The soul we project into objects is "an image purely in our minds". "Yet if that is so, why isn't it equally so for the souls that we project into our friends and family?"
We each have "a storehouse of empathy" that is easy or hard to tap depending on mood and stimulus: sometimes a word or fleeting expression "hits the bull's-eye", sometimes we stay icy. The beetle "flailing against death", raging, in Dylan Thomas's words, "against the dying of the light", touches Dirksen and us, and its apparent recognition of its own doom is "perhaps the most convincing touch of all", recalling the ring of animals slaughtered one by one in the Harding Reflections.
The conclusion is a paradox. "Soul" has emerged "as a function not of any clearly defined inner state, but as a function of our own ability to project". This is "the most behaviouristic of approaches": we ask nothing about internal mechanism and impute a soul from behaviour. "It is a strange sort of validation of the Turing test approach to 'soul detection'."
The Turing test as soul meter (Selection 5): probe deeply and adversarially through language, and credit thought only to what survives probing.
Soul detection in practice (Selections 7–8): react instantly to cues of struggle, smallness and pain, with no probing at all, and switch the reaction off with euphemism or ideology.
Critical lens. Hofstadter's "validation of the Turing test" is double-edged: the Beast shows that behaviour-based attribution is easily fooled by a few hand-built cues, the opposite of Sandy's deep, adversarial probing. The excerpt shows what we do, not what we should do. Miedaner's robot also has real ancestors: W. Grey Walter's electromechanical "tortoises" (1948–49) found their way back to a charging hutch, and the Johns Hopkins "Beast" (Applied Physics Laboratory, early 1960s) roamed corridors seeking wall sockets to plug into. Neither cried.
What changed by 2026. The experiment has been run for real. In 2015 the hitchhiking robot hitchBOT was found vandalised in Philadelphia and the internet mourned it; Kate Darling (MIT) found in a 2013 workshop that people were very reluctant to "hurt" small robotic dinosaurs (Pleos) with a hatchet, and robotics firms' videos of engineers kicking legged robots drew public protest. Researchers now treat this as a design risk: companion robots and chatbots can exploit the empathy cues Hofstadter catalogues. The AI-welfare debate (for example, the 2024 report "Taking AI Welfare Seriously" by Robert Long, Jeff Sebo and colleagues) pulls the other way, warning that we may also fail to notice moral patients that lack cute cues. For product teams the rule follows directly from this selection: do not engineer distress signals your system does not have grounds to emit.
Part III goes down to the physical foundation of mind in biology and then climbs, level by level, to the point where a self-designing system of representations emerges from mindless parts. It moves from Wheelis's prose poem about "spirit" to Dawkins's replicators and memes, then to Hofstadter's ant colony with a mind of its own, and ends at the first of the book's three roadblocks: Zuboff's brain that is cut up, spread out and finally simulated by random events.
The four selections are a ladder. Wheelis gives the poetic view from the top: life as "a slight thickening at the end of a long thread", spirit passing upward from virus to man while individual bodies die and fall away. Dawkins gives the same picture in hard currency: there is no spirit, only replicators that happened to be stable, building "survival machines" (bodies, brains) to carry them, and now a second kind of replicator, the meme, which leaps from brain to brain. Hofstadter's "Prelude . . . Ant Fugue" then shows how a whole that "knows" things can emerge from parts that know nothing: Aunt Hillary is a conversationalist made of ants, and the dialogue works out what levels, signals and symbols must mean for a brain to be a mind.
Zuboff's "Story of a Brain" pushes that picture until it creaks. If the experience lies in the pattern of firings and not in the particular neurons, then splitting the hemispheres, separating every neuron, and replacing their inputs by recorded or even chance impulses should all preserve the experience. Somewhere along the way the reader stops believing it, and the editors admit they have no settled answer. The roadblock is the price of the software view.
Part III's ladder of levels
- Selection 9, "Spirit": the long view; spirit as the apparent direction of evolution.
- Selection 10, "Selfish Genes and Selfish Memes": the mechanism; replicators, survival machines, and cultural replicators.
- Selection 11, "Prelude . . . Ant Fugue": emergence; a mind made of mindless ants, levels of description, active symbols.
- Selection 12, "The Story of a Brain": the roadblock; how far can "pattern, not stuff" be pushed?
A prose poem by the psychiatrist Allen Wheelis, from On Not Knowing How to Live (1975), that pictures each person as a brief "thickening at the end of a long thread", a carrier of a "spirit" that rises from a restlessness of atoms through amoeba, fish and lizard to man, who "now holds a mirror" in which "spirit sees itself". Against Jacques Monod's premise that nature is "objective and not projective", Wheelis suggests that purposelessness may hold over lifetimes but mislead "on the plains of eternity". Hofstadter's Reflections recast Wheelis's "spirit" as no ghostly essence at all, but the seemingly purposeful path of evolution, whose real engine Dawkins will name: the survival of stable replicators.
The selection: the thread and the voyager
"We come into being as a slight thickening at the end of a long thread." Cells proliferate, become "an excrescence", take the shape of a man; the end of the thread lies buried within, shielded. We flourish briefly, "achieve a bit of singing and dancing, a few memories we would carve in stone", then wither; the thread now lies in our children. "Nothing remains but the germ-line." What changes as life evolves is not the "momentary excrescence" but "the hereditary arrangements within the thread".
"We are carriers of spirit. We know not how nor why nor where." We bear it forward with each heartbeat and pass it on "enlarged, enriched, more strange, complex". "We are being used. Should we not know in whose service?"
Monod's premise: a river or a rock was "molded by the free play of physical forces to which we cannot attribute any design"; science assumes "nature is objective and not projective". It has brought huge gains over the animism in which "the rock wanted to fall, the river to sing or to rage".
Wheelis's alternative: not a return to animism; objectivity may hold "for the time spans in which we are accustomed to reckon" and fail over vastly longer ones, as "light travels in a straight line" serves to survey a farm but misleads in mapping distant galaxies.
"Spirit rises, matter falls." Matter would have the universe "a uniform dispersion, motionless, complete"; spirit would have whirl and conflict, a sun to drive away the dark, thought, memory, desire, "a stairway of forms increasing in complexity", tending ever upward, "ruthlessly using lower forms to create higher forms", towards "greater inwardness, consciousness, spontaneity", greater freedom. Far enough back, spirit is "but a restlessness of atoms, a trembling of something there that will not stay in stillness and in cold".
Wheelis's stairway of spirit
- Atoms: "a restlessness", a trembling that will not stay still and cold.
- First life: "Minute creatures writhe in warm oceans"; they touch, "something passes", "spirit is beginning to create love".
- Endless death: the spawnings in rivers, the "waltzing grunion", the rabbits of the plains, "the furry tides of lemmings"; "They die, die, die, but have touched, and something passes."
- The sequences: virus to bacteria to algae to fern (spirit "cracks stone, drives up the Douglas fir"); amoeba reaching out "soft blunt arms"; anemone to squid to fish to slug to lizard; wriggling to swimming, crawling, walking, running, flying.
- Feeling: "Tropism becomes scent, becomes fascination, becomes lust, becomes love."
- Man: "Lizard to fox to monkey to man"; "We love someone far away, someone who died a long time ago."
The essayist Erich Heller writes that "man is the vessel of the Spirit" and Spirit "the voyager". Close up, spirit's path meanders, "a glisten of snail's way in night forest"; from a height its turnings merge into a steady course. Man has reached a ledge from which to look back: thousands of years clear, thousands more through haze, the horizon millions of years behind, and beyond it "a shining path... Running straight". Whose path? "Not man's; for there's our first footprint. Not life's; for there's still the path when life was not yet."
Spirit creates and destroys; creation without destruction is impossible, and destruction without creation "feeds on past creation, reduces form to matter". Because spirit creates more than it destroys (though "not in every season, nor even every age"), the course stays steady: from primal mist to spiral galaxies and "clockwork solar systems", from molten rock to air, land and water, from life to sensation, perception, memory and consciousness, until "man now holds a mirror, spirit sees itself". The harmony of man and nature lies in continuing the journey "toward greater freedom and awareness".
Reflections (D.R.H.)
Hofstadter reads the poem as a portrait of "the eerie disorienting view that modern science has given us of our place in the scheme of things". Many scientists, "not to mention humanists", find it hard to swallow and look for a spiritual essence, perhaps intangible, that would set living beings, especially humans, apart from inanimate nature. "How does anima come from atoms?"
Spirit without a ghost. Wheelis's "spirit" is not such an essence. It is "a way of describing the seemingly purposeful path of evolution as if there were one guiding force behind it". If there is such a force, it is what Dawkins states in the next selection: the survival of stable replicators. Hofstadter quotes Dawkins's preface: "We are survival machines, robot vehicles blindly programmed to preserve the selfish molecules known to us as genes. This is a truth which still fills me with astonishment."
Try this: run the tape at two speeds. Watch a single day of a river: water swirls aimlessly, eddies turn back, nothing "aims" anywhere. Now watch a time-lapse of ten thousand years: the river carves a valley, straightens, finds the sea, and seems to "want" to. Wheelis's claim is that evolution at the billion-year scale looks like this second film. The test of his view is whether the apparent aim needs any explanation beyond the physics of the first film; Dawkins says it does not.
Critical lens. (1) Wheelis's "ever upward" is the progressivism most evolutionary biologists reject: Stephen Jay Gould (Full House, 1996) argued that average complexity barely changes and that "progress" is a statistical artefact of a wall of minimal complexity, with bacteria still the dominant life form. (2) His sequence "virus becomes bacteria" and "fish becomes slug" is poetry, not phylogeny: viruses depend on cells and are not their ancestors, and slugs (molluscs) did not come from fish. (3) Monod's Chance and Necessity (1970), which Wheelis quotes, actually grants living things "teleonomy" (apparent purpose produced by selection) while denying purpose to nature as a whole, which is close to Hofstadter's own reading.
What changed by 2026. The "thread" image has gained hard data: whole-genome sequencing now traces individual lineages back through ancient DNA (Svante Pääbo's Nobel Prize of 2022 recognised the sequencing of the Neanderthal genome and the discovery of interbreeding), showing literally which "thickenings" passed segments of the thread to us. And "spirit sees itself" has a technological sequel: some writers now describe AI as the next vessel in Wheelis's sense, while critics warn that this is exactly the teleological story Monod warned against. A sober reading for technologists: emergent direction in a system (an economy, an evolving codebase, a model trained by selection pressure) can look purposeful without any designer, and governance has to deal with the direction anyway.
Excerpts from Richard Dawkins's The Selfish Gene (1976) tell the history of life as the story of replicators: molecules that happened to copy themselves, competed in a primeval soup, and ended up building "gigantic lumbering robots" (bodies) to carry them, with brains as fast executive computers programmed in advance because genes are too slow to steer behaviour directly. Dawkins then proposes a second replicator, the meme, a unit of cultural imitation that leaps from brain to brain. The selection's two big ideas are that purpose and even consciousness can emerge from blind selection among stable patterns, and that a body (and a mind) is a vehicle for replicators rather than the point of the exercise. Hofstadter's Reflections defend this reductionism against the charge that it leaves no room for emergence, by insisting that the parts themselves are far subtler than "billiard balls".
Richard Dawkins: Oxford ethologist; author of The Selfish Gene.
Jacques Monod: French biologist, quoted on how "everybody thinks he understands" evolution.
Gregor Mendel: discovered particulate inheritance; unread until after his death and Darwin's.
Fred Hoyle and John Elliot: authors of A for Andromeda, Dawkins's parable of genes programming at a distance.
J. Z. Young: biologist who described genes as performing "a task analogous to prediction".
Roger Payne: whale researcher; source of the humpback song example.
N. K. Humphrey: Dawkins's colleague, who called memes "living structures" in a technical, not a merely metaphorical, sense.
G. C. Williams: evolutionary biologist; "Socrates may or may not have a gene or two alive... But who cares?"
Survival of the stable
"In the beginning was simplicity." Explaining a simple universe is hard enough; explaining complex order springing up "fully armed" would be harder. Darwin's theory satisfies because it shows how simplicity could change into complexity, how unordered atoms could group into ever more complex patterns "until they ended up manufacturing people". Dawkins explains it more generally than usual, starting before evolution began.
Survival of the fittest is a special case of "survival of the stable". A stable thing is "a collection of atoms which is permanent enough or common enough to deserve a name": a unique object like the Matterhorn, or a class like raindrops, short-lived but constantly re-formed. Soap bubbles are spherical because that is stable for gas-filled films; water forms globules in a spacecraft but lies flat under gravity; salt crystals are cubes because that packs sodium and chloride stably; in the sun hydrogen fuses to helium because helium is more stable there; heavier elements form in stars and in the big bang.
Atoms link into molecules, some very large. A diamond is a single, proverbially stable, but simple molecule. Haemoglobin is complex at several levels:
574: amino acids in a haemoglobin molecule, arranged in four chains twisted into a "dense thornbush".
Six thousand million million million: identical copies of that exact shape in an average human body.
About 400 million million per second: haemoglobin "thornbushes" springing into shape in your body, with as many destroyed.
Over a thousand million million million million: atoms in a man (more than 10²⁷).
3 to 4 thousand million years: since the "primeval soup" of the early seas.
Two chains with the same sequence "will tend, like two springs, to come to rest in exactly the same three-dimensional coiled pattern". Before life, then, a rudimentary evolution of molecules could occur through ordinary physics and chemistry: "There is no need to think of design or purpose or directedness." The earliest natural selection was a selection of stable forms and rejection of unstable ones; "It had to happen by definition." But you cannot shake the right atoms in a "biochemical cocktail-shaker" until "out drops Adam!"; you would wait so long that the age of the universe seemed an eye-blink. Darwin's theory "takes over from where the story of the slow building up of molecules leaves off".
The primeval soup and the Replicator
The account is necessarily speculative ("nobody was around to see"), though rival theories share its features. Plausible raw materials: water, carbon dioxide, methane and ammonia. Chemists who put these in a flask with ultraviolet light or electric sparks ("artificial simulation of primordial lightning") find after a few weeks "a weak brown soup" of more complex molecules, including amino acids, once thought diagnostic of life (found on Mars, they would have meant life was near certain), and later purines and pyrimidines, building blocks of DNA. Such substances, concentrated in drying scum or droplets and energised by sunlight, combined into larger molecules that could "drift unmolested" because there were not yet bacteria to eat them.
The Replicator. Setup: (1) at some point, by accident, a molecule forms that can make copies of itself; (2) it is exceedingly improbable, like winning the football pools, but over hundreds of millions of years of weekly coupons you would win several jackpots, and it only had to happen once; (3) think of it as a template: building blocks in the soup with an affinity for their own kind stick to matching parts and line up in the same sequence, joining into a chain (as crystals grow layer by layer), and the two chains may split, giving two replicators; (4) or blocks have affinity for one particular other kind, so the replicator makes a "negative" that remakes the positive (the way DNA does it). What follows: "a new kind of 'stability' came into the world". The replicator's copies spread through the seas until the building blocks became scarce.
Copying is never perfect. Dawkins hopes his book has no misprints, but first-generation errors would not distort much; copy the Gospels by hand, copies from copies, and errors accumulate (some scribes "are not above a little willful 'improvement'"). Human copying errors are rarely improvements, though the scholars of the Septuagint, translating the Hebrew for "young woman" as the Greek for "virgin", "started something big". In biological replicators errors make evolution possible. DNA is astonishingly faithful, more than any human copying process, but still errs; early replicators were probably far sloppier.
The three kinds of replicator "stability" that selection favours
- Longevity: varieties less likely to break up become more numerous, both directly and because they have longer to copy themselves.
- Fecundity: copying speed matters even more; if type A copies once a week and type B once an hour, A is soon "far outnumbered" even if it lives longer.
- Copying fidelity: if X errs every tenth copy and Y every hundredth, Y wins, since X loses not only its errant "children" but all their descendants.
The apparent paradox (errors drive evolution, yet selection favours fidelity) dissolves once you see that nothing "wants" to evolve: "Evolution is something that happens, willy-nilly, in spite of all the efforts of the replicators (and nowadays of the genes) to prevent it happening." Monod, in his Herbert Spencer lecture: "Another curious aspect of the theory of evolution is that everybody thinks he understands it!" Sample the soup at two times and the later sample has more long-lived, fecund, faithful varieties: that is what a biologist means by evolution.
Dawkins dismisses the question whether the replicators were "living": "Who cares?" Arguing whether Darwin or Newton was "the greatest man" changes no fact about either. "Human suffering has been caused because too many of us cannot grasp that words are only tools for our use." Living or not, the replicators "were our founding fathers".
Competition and survival machines
The soup could not support infinite replicators; building blocks became scarce, varieties competed, and less-favoured lines went extinct. It was "a struggle for existence... Conducted without any hard feelings, indeed without feelings of any kind". Any miscopy that raised a variety's stability, or lowered rivals', was automatically preserved and multiplied, so improvement was cumulative.
- Proto-carnivores. Some replicators may have "discovered" how to break up rivals chemically and reuse their building blocks, getting food and removing competitors at once.
- Protection. Others protected themselves chemically or with a physical wall of protein, perhaps the origin of the first cells.
- Vehicles. "Replicators began not merely to exist, but to construct for themselves containers, vehicles for their continued existence." The survivors built survival machines, at first just a protective coat, then ever bigger and more elaborate.
Now they go by the name of genes, and we are their survival machines.
Four thousand million years on, the replicators have not died out, but "do not look for them floating loose in the sea". They "swarm in huge colonies, safe inside gigantic lumbering robots, sealed off from the outside world, communicating with it by tortuous indirect routes, manipulating it by remote control. They are in you and in me; they created us, body and mind." Selection now favours genes "skilled in the art of controlling embryonic development", but "Genes have no foresight. They do not plan ahead. Genes just are, some genes more so than others, and that is all there is to it." In the last 600 million years they have produced muscle, heart and eye (the eye "evolved several times independently").
Genes as gregarious, particulate immortals
A modern replicator is highly gregarious: a body contains many thousands of genes; one gene affects many parts, one part is influenced by many genes, and master genes control clusters of others, like pages of building plans that each make sense only by cross-reference. Why speak of "genes" at all, not "gene complexes"? Because of sex: sexual reproduction shuffles genes, so each body is "a temporary vehicle for a short-lived combination of genes", while the genes themselves are potentially very long-lived, their paths crossing and recrossing down the generations.
Why the gene is the unit of selection. Selection needs entities that exist as lots of copies and can survive, as copies, for evolutionary time. "Small genetic units have these properties; individuals, groups, and species do not." Mendel showed hereditary units behave as indivisible, independent particles; a gene, as Dawkins defines it, is "not indivisible, but... Seldom divided", either present or absent, passing "intact from grandparent to grandchild" without blending. If genes blended, selection as we know it would be impossible, a problem that worried Darwin, who never learned of Mendel's published work (nobody read it until both were dead). And a gene does not grow senile: "The genes are the immortals", with life expectancies of thousands or millions of years, abandoning a succession of mortal bodies.
Survival machines began as passive receptacles feeding on free organic molecules. When that food ran out, one branch, plants, began using sunlight to build complex molecules; another, animals, exploited plants' chemical labour by eating them or each other. Sub-branches specialised for sea, ground, air, underground, trees, "inside other living bodies". Bodies became many-celled, each cell holding a full copy of the genes. Some call a body a colony of cells; Dawkins prefers "a colony of genes", with the cell as "a convenient working unit for the chemical industries of the genes". Yet bodies act as units, and "Subjectively I feel like a unit, not a colony", which selection for cooperating genes and central coordination would predict; the communal nature of a survival machine is now "virtually unrecognizable", and many biologists "will disagree with me".
Purpose machines: governors, missiles, chess programs
The most striking property of survival-machine behaviour is apparent purposiveness: watching an animal search for food, a mate or a lost child, we impute desire, a mental picture, "an end in view". In at least one survival machine this purposiveness has evolved consciousness. Dawkins is "not philosopher enough" to say what that means, and sets it aside, because machines that behave as if purposive are engineering commonplaces.
The Watt governor and negative feedback. A "purpose machine" measures the discrepancy between the current and the "desired" state and works harder the larger it is, so it reduces the gap (negative feedback). In James Watt's governor two balls on hinged arms whirl with the engine; centrifugal force lifts the arms against gravity, and the arms close the steam valve as they near horizontal: too fast, steam is cut; too slow, more is fed. Such machines oscillate from overshoot and lag, which engineers damp out. "Obviously it does not consciously desire" its speed; a machine's "goal" is "simply defined as that state to which it tends to return". Guided missiles extend the trick with feedback and "feed-forward", seeming to search, pursue and even "anticipate" evasions, though nothing remotely like consciousness is involved.
A common misconception: a machine designed by conscious man must be under the immediate control of conscious man; or "computers do not really play chess, because they can only do what a human operator tells them". Dawkins dismantles it with computer chess (in 1976, programs had reached "the standard of a good amateur", and "programs", not computers, is the right word, since a program "is not fussy which physical computer it uses").
| Possible role for the programmer | Dawkins's verdict |
|---|---|
| Puppeteer, moving pieces moment to moment | "That would be just cheating"; once running, the computer is on its own |
| List of good moves for every position | Impossible: the world would end first |
| Search every move and follow-up to a win | Impossible: more possible games "than there are atoms in the galaxy" |
| A father teaching his son | Correct: economical rules (a bishop adds the same constant to both coordinates) plus advice ("don't leave your king unguarded", knight forks), then hands off |
A for Andromeda: genes as remote programmers
Genes cannot "grab the reins" because of time lags, best shown by Fred Hoyle and John Elliot's science-fiction story A for Andromeda, whose key point, Dawkins says, the authors never spell out.
Thought experiment: the Andromedan computer. Setup: (1) a civilisation 200 light years away (in the constellation, not the galaxy, as the editors note) wants to spread its culture; (2) travel is impossible, so they broadcast by radio in all directions, reaching more worlds as the square of distance; (3) conversation is pointless: each exchange would take centuries and twelve generations or so; (4) so they assemble one huge message and broadcast it over and over, with a cycle of several months; (5) it encodes instructions to build and program a giant computer; (6) picked up at Jodrell Bank, decoded and built, the computer moves "well on the way to dictatorship over the world" until the hero "finished it off with an axe". The question: in what sense were the Andromedans manipulating Earth? They had no moment-to-moment control and could not even know it had been built; "All its instructions had to be built-in in advance", flexibly enough to work on any advanced world whose conditions they could not know.
Smaller-scale versions of the same predicament: radio takes about four minutes between Earth and Mars, so spacemen will need "long soliloquies or monologues, more like letters". Roger Payne notes that the acoustics of the sea might let the enormously loud humpback whale song be heard around the world at the right depth; a reply across the Atlantic would take nearly two hours. Dawkins suggests this explains why whales sing a continuous, unrepeating eight-minute soliloquy and then repeat it, cycle after cycle, like the Andromedans.
Genes are policy-makers; brains are executives. "Just as the Andromedans had to have a computer on earth to take day-to-day decisions for them, our genes have to build a brain." But genes are both the senders and the instructions themselves. They act through protein synthesis: powerful but slow, months to build an embryo, whereas behaviour works in milliseconds (an owl flashes overhead, a rustle betrays prey). So genes can only build "a fast executive computer" and program it with rules and "advice" for as many eventualities as they can "anticipate"; since life, like chess, has too many, they instruct "not in specifics, but in the general strategies and tricks of the living trade".
J. Z. Young points out that genes perform "a task analogous to prediction". Polar bear genes "predict" a cold, snowy future and build a thick white coat, not by thinking but because that is what they did in previous bodies; if the Arctic turned into tropical desert too fast, "the young bear would die, and they inside it".
Simulation, consciousness and rebellion
One of the best ways to predict is simulation. A general testing plans could try each in earnest, but the supply of young men willing to die "for their country" is exhaustible; better dummy runs: full-scale exercises ("Northland" against "Southland" with blanks), war games with tin soldiers, and now computers, used in economics, ecology, sociology. A computer's model need not look like what it models: no little chessboard inside, but lists of coded numbers; a map as towns with latitude and longitude. What matters is that it can be manipulated and experimented on. "Simulation could be called vicarious trial and error", a term "unfortunately preempted long ago by rat psychologists."
Survival machines invented lenses, parabolic reflectors, frequency analysis, servo-control, sonar and buffer storage before we did, so why not simulation? When you face a hard decision you imagine the alternatives, modelling the relevant entities, vividly or abstractly; how the brain represents them matters less than that it can use them to predict. Survival machines that simulate are "one jump ahead" of those that can only learn by overt trial and error, which takes time and is often fatal.
Consciousness as the culmination of simulation. "The evolution of the capacity to simulate seems to have culminated in subjective consciousness. Why this should have happened is, to me, the most profound mystery facing modern biology." Computers that simulate are not thereby conscious, though they may become so. "Perhaps consciousness arises when the brain's simulation of the world becomes so complete that it must include a model of itself." Dawkins finds this unsatisfying, partly because of the infinite regress (a model of the model, of the model...).
Whatever consciousness is, it is "the culmination of an evolutionary trend towards the emancipation of survival machines as executive decision-takers from their ultimate masters, the genes". Brains run daily affairs and predict the future; they can even rebel against the genes, for instance by refusing to have as many children as they could, and in this "man is a very special case". Genes control behaviour only indirectly, by building nervous systems, but powerfully. "Genes are the primary policy-makers; brains are the executives." The logical end of the trend, reached in no species yet, would be a single instruction: "do whatever you think best to keep us alive."
Selfish memes
Dawkins asks whether any principle of biology is as universal as the laws of physics. Whatever life is made of (silicon instead of carbon, ammonia instead of water, creatures that boil at −100°C, or "electronic reverberating circuits" with no chemistry at all), Dawkins would bet on one law: "all life evolves by the differential survival of replicating entities." DNA happens to be the replicator on Earth. And we need not go to other worlds to find another: "a new kind of replicator has recently emerged on this very planet", still "drifting clumsily about in its primeval soup", but evolving at a rate "which leaves the old gene panting far behind". The new soup is human culture.
The meme. A unit of cultural transmission or imitation: from Greek "mimeme", shortened to rhyme with "cream", and suggestive of "memory" and the French même. Examples: tunes, ideas, catch-phrases, clothes fashions, ways of making pots or of building arches. As genes leap from body to body via sperm and eggs, memes leap from brain to brain via imitation: a scientist hears a good idea, passes it to colleagues and students, puts it in articles and lectures, and if it catches on it propagates itself.
N. K. Humphrey summed up an earlier draft: memes are "living structures", and in a technical, not a merely metaphorical, sense: "When you plant a fertile meme in my mind, you literally parasitize my brain, turning it into a vehicle for the meme's propagation in just the way that a virus may parasitize the genetic mechanism of a host cell." The meme for "belief in life after death" is realised physically, millions of times over, as structure in human nervous systems.
Dawkins conjectures that co-adapted meme-complexes evolve like co-adapted gene-complexes: selection favours memes that exploit their cultural environment, which consists of other memes, so the meme pool acquires the attributes of an evolutionarily stable set that new memes find hard to invade.
Immortality through genes: forgotten in three generations; each generation halves your contribution. Elizabeth II descends directly from William the Conqueror, yet quite probably carries "not a single one of the old king's genes". "We should not seek immortality in reproduction."
Immortality through memes: a good idea, a tune, a spark plug, a poem "may live on, intact, long after your genes have dissolved". Socrates may or may not have a gene or two alive today, as G. C. Williams remarked, "but who cares?" The meme-complexes of Socrates, Leonardo, Copernicus and Marconi "are still going strong".
Reflections (D.R.H.)
Dawkins, says Hofstadter, is "a master at expounding the reductionist thesis": life and mind come out of "a seething molecular tumult" in which small, accidental units are passed again and again through "the merciless filter" of competition. Reductionism sees everything as reducible to physics, with no room for "emergent" properties or, in an old-fashioned word, "entelechies", higher-level structures supposedly inexplicable by the laws of their parts.
Thought experiment: the repaired typewriter. Setup: you send a broken typewriter (or washing machine, or photocopier) back to the factory; a month later it returns, reassembled correctly, with a note: sorry, all the parts check out fine, but the whole simply doesn't work. The intuition: outrageous; if the whole fails, something must be wrong somewhere. The question: does that common sense hold as you go down, level after level? It does, says Hofstadter, and yet people keep saying "You can't derive the properties of water from the properties of hydrogen and oxygen atoms" and "A living being is greater than the sum of its parts."
The error is to picture atoms as simple billiard balls with valences. At small scales the mathematics of matter becomes more intractable than ever. Hofstadter quotes Richard Mattuck's textbook, citing G. E. Brown: in eighteenth-century Newtonian mechanics the three-body problem was insoluble; with general relativity (around 1910) and quantum electrodynamics (around 1930) the two- and one-body problems became insoluble; in modern quantum field theory zero bodies (the vacuum) is insoluble. "So, if we are out after exact solutions, no bodies at all is already too many."
Where emergence really lives. The quantum mechanics of oxygen, with eight electrons, is far beyond complete analytic solution; the subtle properties of hydrogen, oxygen and water molecules are "precisely the sources of water's many elusive qualities". Computer simulations of many simplified interacting units now routinely reveal collective behaviour: spiral arms of galaxies from stars modelled as gravitating points; how solids, liquids and gases vibrate, flow and change state from molecules modelled as simple electromagnetic structures. "People habitually underestimate the intricacy and complexity that can result from a huge number of interacting units obeying formal rules at very high speeds, relative to our time scale."
Dawkins ends his book with "his own meme about memes", software replicators that dwell in minds, after considering alternative life-support media. One he omits: the surface of a neutron star, where nuclear particles bond and unbond thousands of times faster than atoms. A "chemistry" of nuclear particles could in theory allow tiny replicators whose high-speed lives "zoom by in an eyeblink", as complex as ours: "an entire civilization's rise and fall in the period of a few earth days, a super-Lilliput!" (Lem's selections, especially "The Seventh Sally", share this quality.) The point is to keep an open mind about the variety of media that can support lifelike or thoughtlike activity, which the next dialogue explores "slightly less wildly": consciousness emerging from the levels of an ant colony.
Critical lens. (1) "Selfish gene" is a metaphor Dawkins defends carefully, but critics from Stephen Jay Gould to proponents of multilevel selection (David Sloan Wilson, and E. O. Wilson in his late work) dispute gene-centrism; the extended evolutionary synthesis stresses development, plasticity and niche construction. (2) The Mars radio delay is "about four minutes" only at closest approach; it ranges from roughly 3 to 22 minutes. (3) Hofstadter's reply to the "parts are fine" puzzle can be read two ways: as reductionism (the whole is fully fixed by richer-than-expected parts) or as weak emergence (fixed by the parts, but only discoverable by simulation); he blurs them here and separates them in the Ant Fugue. (4) Dawkins's guess that consciousness arises when a simulation must include a model of itself is close to Hofstadter's later "strange loop" view and to self-model theories (Thomas Metzinger), but it remains a conjecture.
What changed by 2026. Computer chess went from "good amateur" to Deep Blue beating Kasparov (1997) and AlphaZero (2017) teaching itself to superhuman play, a vivid case of Dawkins's "father teaching his son" taken to the limit: rules plus learned "advice", then the program is on its own. "Meme" escaped into everyday speech as the internet image macro; academic memetics stalled (the online Journal of Memetics closed in 2005), though cultural evolution became a flourishing quantitative field, and Dennett's From Bacteria to Bach and Back (2017) rebuilt his theory of mind on memes. Large language models are, in effect, machines trained on the meme pool, and the governance problem in enterprise AI is Dawkins's Andromeda problem: you cannot steer a deployed model moment to moment, so the "policy" must be built in beforehand and monitored, with the axe (a kill switch) kept within reach.
Douglas Hofstadter's dialogue from Gödel, Escher, Bach (1979), written in the form of a Bach prelude and fugue, in which Achilles, the Tortoise, the Crab and the Anteater listen to a miraculous recording of Bach and argue about whether wholes or parts explain the world. Its centre is Aunt Hillary, an ant colony who converses with the Anteater by trail-writing though every one of her ants is "as dumb as can be", and the Anteater's account of how ants form teams, teams form "signals", and signals form active "symbols" whose triggering patterns, read at the right level, are thoughts. The dialogue's lesson is that a mind is a pattern at a high level of description, invisible at the level of its components, so that a colony, like a brain, can think although none of its parts does; the right answer to "holism or reductionism?" is the Zen "MU", which unasks the question. Hofstadter's Reflections define a "representational system", defend talk of "downward causality" with a domino computer that tests whether 641 is prime, and offer the "thinking wind chime".
Achilles: the eager, naive novice; can follow a fugue only one way at a time; sees "MU".
The Tortoise: the trickster, who brings the gift and plants the jokes; also sees "MU", "an octave lower".
The Crab: the host, owner of an illuminated Well-Tempered Clavier; the holist.
Dr. Anteater: a myrmecologist and "colony surgeon"; the reductionist, and Aunt Hillary's friend.
Aunt Hillary: a well-educated, eccentric, libertarian ant colony who talks with the Anteater for hours.
Johant Sebastiant Fermant (J. S. F.): a brilliant colony, mathematician and musician, destroyed by a freak thundershower; Aunt Hillary's predecessor on the same ants.
How the dialogue is built
The dialogue is itself a prelude and four-voice fugue. In the Prelude the four friends chat over tea. When the Fugue starts, the four characters enter one by one with the same "subject", a line beginning "I know the rest of you won't believe this, but the answer to the question is staring us all in the face, hidden in the picture", each finishing it differently, exactly as fugue voices state one theme at staggered intervals. Later musical devices are named in the conversation just as the conversation performs them, and Achilles keeps missing them.
| Musical device | Defined in the dialogue as | How the dialogue performs it |
|---|---|---|
| Fugue subject, four entries | one theme entering voice by voice | Achilles, Crab, Anteater, Tortoise each say "I know the rest of you won't believe this..." |
| Fourth entry an octave lower | the last voice enters low | the Tortoise's "MU" is said to exist "on a deeper level", "an octave lower" |
| Stretto | the theme entering in one voice after another "with very little delay" | four overlapping "How is it possible that..." questions about ants, forests, trees, haircuts |
| Organ point | one voice sustains a single note while the others move | the Anteater's long, unbroken story of eating the ants that said "help yourself" |
| Fermata | a pause, "a sort of musical semicolon" | the Tortoise slows and pauses while spelling "fermat" |
| Inversion and augmentation | "a strange new twist on its theme" | the last picture, read as "HOLISM" twice with shrinking letters, or "REDUCTIONISM" once with growing ones |
Prelude: Bach retrieved from the air
The Tortoise and Achilles bring the Crab a present: two unlabelled records. Not a "phonograph-breaker" this time, but a unique recording, "never even been heard before, except, of course, when Bach played it". Achilles explains from a filing card. Pierre de Fermat, "a lawyer by vocation but mathematician by avocation", reading Diophantus's Arithmetica, wrote in the margin that the equation has no solutions for powers greater than 2, adding that he had "a truly marvelous proof" (Achilles garbles both the equation and the famous excuse, saying the proof was so small it would be "well-nigh invisible" in the margin). For some three hundred years mathematicians have tried to prove it or find a counterexample; it has been checked for all n up to 125,000.
The Tortoise's double-barrelled result. The joke: the Tortoise has found both a proof of Fermat's Last Theorem and a counterexample. The values? Achilles left them at home "on a truly colossal piece of paper"; but n is "the only positive integer which does not occur anywhere in the continued fraction for π". The counterexample turned out to be the key to the proof: the "structural layout" of any proof depended on solutions of an equation that turned out to be Fermat's own, "an amusing accidental relationship between form and content", so the counterexample's numbers served as "a blueprint for constructing my proof that there were no solutions". The payoff: this unlocked "acoustico-retrieval", reconstructing sounds from their remote effects (as a brain reconstructs another's voice from eardrum vibrations, or one might reconstruct a rock's splash from the ripples), and the Tortoise has retrieved Bach playing the whole Well-Tempered Clavier from the present motions of the atmosphere's molecules.
The records hold both volumes, twenty-four preludes and fugues in each, one per major and minor key. The playing fills the room "in the highest imaginable fidelity", and one even hears, or imagines, "the soft sounds of Bach singing to himself as he plays". The Crab brings out a unique edition of the score, "specially illuminated" by a calligrapher teacher.
The friends talk about how a prelude sets the mood for its fugue through "some intangible abstract quality", and about the silent suspense before a fugue's theme rings out "in single tones" and then joins with itself "in ever-increasingly complex levels of weird, exquisite harmony". Achilles loves second-guessing: allegro or adagio, 6/8 or 4/4, three voices or five? The Crab recalls the thrill of youth, now "supplanted by familiarity", which brings its own depth and new surprises (an inverted theme hidden among voices, an amazing modulation). The youthful thrill can be resuscitated by hearing a piece "through the ears" of a newcomer: the potential is "coded, in some unknown way, in the structure of my brain", but he cannot summon it at will.
Two modes of hearing, one dichotomy. Achilles asks about Escher's Cube with Magic Ribbons (1957), whose bubbles flip between bumps and dents and can never be seen both ways at once. He has found two analogous modes of fugue-listening: follow one voice at a time, or hear the total effect, and each shuts out the other. The Tortoise and Crab flip between them too, never in control. The Anteater generalises: each fugue voice "could stand alone", yet the lines fuse "in a highly nonrandom way" into "a graceful totality"; this dichotomy between hearing the whole and hearing its components is "a particular example of a very general dichotomy" that applies to many structures built from lower levels. Achilles wonders whether to listen "as a whole, or as the sum of its parts". "Listen carefully," says the Tortoise, "and you will!"
The MU-picture: holism, reductionism, and "MU"
Turning a page, the Tortoise finds an illustration facing the fugue. As the fugue begins, each character enters with the same line and a different reading.
Four readings of the MU-picture
- Achilles: two pieces, each a single letter: "MU".
- The Crab: the left piece is three copies of "HOLISM" and the right is many small "HOLISM"s, so "HOLISM", "plain as day".
- The Anteater: the left is many small "REDUCTIONISM"s and the right one large "REDUCTIONISM".
- Achilles' synthesis: the left has three HOLISMs made of small REDUCTIONISMs; the right has one REDUCTIONISM made of small HOLISMs; the two pieces together spell "MU". The Crab and Anteater "missed the forest for the trees".
- The Tortoise (fourth voice, an octave lower): "MU", but "perhaps 'MU' exists in this picture on a deeper level than you imagine".
The Crab defines holism: "the whole is greater than the sum of its parts"; "No one in his right mind could reject holism." The Anteater defines reductionism: "a whole can be understood completely if you understand its parts, and the nature of their 'sum'"; "No one in her left brain could reject reductionism." Each challenges the other: no reductionist explanation of a brain can explain where its consciousness comes from; no holistic description of an ant colony explains more than its ants and their relations (or, the Anteater adds in the same breath, "where the consciousness experienced by an ant colony arises from").
"MU" unasks the question. "Mu" is "an ancient Zen answer which, when given to a question, unasks the question." Asked "Should the world be understood via holism or via reductionism?", it rejects the premise that one must be chosen, revealing "a larger context into which both holistic and reductionistic explanations fit". The Anteater calls it "as silly as a cow's moo", the Crab "as silly as a kitten's mew". The Tortoise asks instead for both views laid out explicitly, starting with a reductionist account of an ant colony.
Aunt Hillary: the conversing colony
The Anteater is not an anteater by profession ("that is my species"): he is a colony surgeon, treating nervous disorders such as speech impairment by "surgical removal" of the defective parts. Achilles is baffled: colonies don't speak. The Crab regrets that Achilles missed Aunt Hillary, a house guest last week, "not really anybody's aunt", eccentric, "such a merry old soul", "one of the best-educated ant colonies" the Anteater knows. He eats ants, not colonies, which is good for both. The question of how that could help the colony launches a stretto: "How is it possible that having its ants eaten can do an ant colony any good?" / "...a forest fire... A forest?" / "...its branches pruned... A tree?" / "...a haircut... Achilles?"
Colonies don't converse out loud but in writing: some trails carry coded information "you can read just like a book". The Anteater replies by drawing trails in moist ground with a stick, watches the ants follow, and reads the answering trail as it forms ("more often I am wrong than right" when anticipating). "There must be some amazingly smart ants in that colony," says Achilles.
Thought experiment: the conversing colony. Setup: (1) a colony carries on witty conversation through trail patterns; (2) "all the ants in Aunt Hillary are as dumb as can be. They couldn't converse to save their little thoraxes!"; (3) so where does conversation reside? The Tortoise's parallel: no one thinks individual neurons must be intelligent for a person to converse. Achilles' objection: neurons are reliable, but ants roam at random, "free to do what they want". The Crab: ants are free only within constraints; they never step outside the ant-system, so they are reliable components. The Anteater's key: "the regularity of statistics". What it shows: coherent high-level behaviour can arise from individually unpredictable parts, provided their interactions are structured; the colony, not any ant, is the conversationalist.
Ant trails illustrate it: any single ant's motion is unpredictable, yet the trail stays well defined, because ants communicate "just enough" to remind each other "they are not alone". The Crab describes a neuron: inputs from many others summed against a threshold, then an output pulse "rushing off to other neurons", the neural flash swooping "in shapes stranger than the dash of a gnat-hungry swallow", every turn "foreordained" by structure until sensory input interferes. Achilles is disturbed: it makes "I" sound like "a by-product of an organism governed by natural law" or "an artificial notion", and "you make me feel like I don't know who, or what, I am, if anything."
Thresholds, teams, castes
Coherent group activity, the Anteater realises, needs a threshold number of ants. A random start by a few ants either fizzles out or, given critical mass, snowballs into a team working on one project: trail-making, food-gathering, nest-keeping. Simple at small scale, it has "very complex consequences on a larger scale". Achilles objects that order also emerges in a gas, yet a gas has only three parameters: volume, pressure, temperature. The difference: a gas needs one jump from molecules to the whole, but a colony has intermediate levels of organisation that you must go through to understand it.
These levels come from castes. Besides the queen, there are males (who do "practically nothing toward the upkeep of the nest"), workers (who forage, hunt, nurse and do most of the fighting) and so-called soldiers ("slow, ungainly ants with giant heads", "lazy fatheads"). Achilles, "glorious fighters against communism!", would knock sense into them; the Crab notes that colonies are "quite communistic internally". The Tortoise: a "myrmidon" like Achilles cannot be mapped onto an ant brain, but it would be reasonable to map his brain onto an ant colony.
The caste distribution is the colony's knowledge. Specialisations change with age and caste, and castes are dense here, sparse there. Over long periods "a very delicate distribution of castes" evolves, which underlies the colony's ability to converse. Achilles thinks random motion would wreck any delicate pattern, as in a gas. The opposite: "it is just exactly the constant to-ing and fro-ing of ants... Which adapts the caste distribution to varying situations, and thereby preserves" it. When the Anteater arrives, panicking ants rearrange the distribution, which "reflects my presence": the colony has gained "a piece of knowledge". But only a description at the proper high level shows how a distribution encodes knowledge; at the ant level "you miss the forest for the trees".
Signals and the question of purpose
Groups form and unform constantly; those that persist are teams, and they persist because "there really is something for them to do", which is equivalent to exceeding the threshold: a crumb of food attracts too few ants to sustain a team, which is exactly right, since too little food should be ignored. A special kind of team is the signal; all higher structures are "collections of signals acting in concert", teams of teams down to signals and then ants. A signal's life: it forms by passing the threshold, migrates through the colony, and disintegrates where its members are needed, like a wave leaving sand dollars and seaweed on the shore, except that there is no water: "the ants themselves compose it".
Achilles asks how a signal knows where to go and when it has arrived. The caste distribution guides it. As a signal moves, its ants interact by contact and scent with local ants, picking up local needs (nest-building, nursing); "The signal will remain glued together as long as the local needs are different from what it can supply; but if it can contribute, it disintegrates."
The ant's-eye view: a signal has no purpose; its ants meander "in search of nothing in particular" until they feel like stopping, and the team crumbles. "No planning is required, no looking ahead."
The colony's-eye view: the team "has just responded to a message written in the language of the caste distribution", which looks "very much like purposeful activity".
The Crab presses holism: a random caste distribution would still produce signals, but the colony would not last; colonies survive because their distribution has meaning, "a holistic aspect, invisible on lower levels". The Anteater widens the view: run a film of colony evolution "a billion or so times faster" and the mechanisms look like bubbles in boiling water, natural responses to external pressure; no bubble needs "higher-level laws", and "by looking at things from the vast perspective of evolution, you can drain the whole colony of meaning and purpose". Then how can he talk with Aunt Hillary? He is not inconsistent: on the grand time scale meaning drains away, but in the here and now "the vocabulary of teleology comes back", for colonies and for brains, his own included, and he can switch views at will.
An ant fugue
The Crab muses that evolution might even allow two signals to pass through each other, each treating the other as background. "It happens routinely!" Achilles pictures ants of four kinds "criss-crossing, together forming an orderly pattern", "An ant fugue!" Then, while the Crab pours tea, the conversation itself becomes a four-voice tangle: Achilles' question about separating visual "voices" interleaves with the tea orders ("Four cups of tea." "Three!"). Signals also change membership en route, often arriving "with nary an ant in common with their starting lineup", so a signal is a pattern, not a set of ants.
Symbols: active, not passive
The Crab sums up: signals update the caste distribution in response to colony needs, which reflect the outside world, so the distribution "ultimately reflects the outer world". Above signals the Anteater names teams of sufficiently high level "symbols", in an unusual sense: "active subsystems of a complex system, composed of lower-level active subsystems", unlike passive symbols such as letters or musical notes "which sit there immobile waiting for an active system to process them". These layers are needed to store knowledge for intelligence, and "any system which has a mastery of language has essentially the same underlying sets of levels".
Achilles protests that this makes his brain "just a bunch of ants running around". Not so: the lowest level may be utterly different ("the brains of anteaters... Are not composed of ants"), but "when you go up a level or two in a brain, you reach a level whose elements have exact counterparts in other systems of equal intellectual strength". The Anteater guesses a neural counterpart of a signal is a neuron firing, or a pattern of firings; the Crab thinks the correspondence is likely only at the level of active symbols and above, and the Anteater concedes.
Signals are to symbols as letters are to words. Words carry meaning; their letters do not. But symbols and signals are active, and this matters because "the meaning which you attribute to any passive symbol, such as a word on a page, actually derives from the meaning which is carried by corresponding active symbols in your brain." What gives an active symbol meaning is the way symbols trigger other symbols, floating in a medium characterised by its caste distribution, or in a brain by the "brain state" (all neurons, interconnections and thresholds). Lump them together as "the state".
| Level of description | What it specifies | What it buys | What it costs |
|---|---|---|---|
| Ant level | every ant's location, age, caste | complete detail | "practically no global insight" |
| Team or signal level | clumps, their relations, local caste supplies | a summary of the ant level | must add relations "not present" below: "the price you pay for the right to summarize" |
| Symbol level | which symbols trigger which, under what conditions | "the most explanatory power", the most intuitive picture | "leaves out seemingly the most important feature: the ants" |
The ants are dispensable from a high-level view, says the Anteater, since a brain can do the same thing "ant-free". ("I'm sure no ant would embrace your theory with eagerness." "Well, I never met an ant with a high-level point of view.")
Thought experiment: reading Dickens letter by letter. Setup: take The Pickwick Papers; map each letter to a fixed idea, so that every "the" means the t-concept, the h-concept, the e-concept, every time. The intuition: reading becomes "an indescribably boring nightmare", "an exercise in meaninglessness", whatever mapping you choose. What it shows: there is "no natural mapping from the individual letters into the real world"; the natural mapping is between words and the world, so to describe the book you talk of plot and characters and never mention letters. "They are the medium, but not the message." Likewise you could not map signals (or ants) onto the world so that their triggering made sense; "Only on the symbol level do the triggering patterns make sense." Watching the Anteater arrive, you would see only "a rearrangement of ants"; he, reading the higher level, sees dormant symbols waking that translate as "Oh, here's that charming Dr. Anteater again, how pleasant!"
Who is the agent? Reading your own brain
Achilles' next worry is that if symbols are active, no one is left to manipulate them. Symbols are not free: their activity is "strictly determined by the state of the full system", so the full system is the agent: a partially constant, partially varying whole that can be named. "Aunt Hillary is the 'who' who can be said to manipulate her symbols; and you are similar, Achilles." The Tortoise would love to follow the symbols in Achilles' brain as he thinks about the symbols in his brain.
Perceiving a colony at the symbol level takes long practice; the Anteater reads it as easily as Achilles read "MU". And it is familiar to Achilles: "What about your own brain? Aren't you aware of your own thoughts? Isn't that the essence of consciousness? What else are you doing but reading your own brain directly at the symbol level?"
Consciousness as symbol-level self-reading. Conscious systems "perceive themselves on the symbol level only, and have no awareness of the lower levels". In any conscious system "there are symbols that represent the brain state, and they are themselves part of the very brain state which they symbolize. For consciousness requires a large degree of self-consciousness." Achilles finds it as weird as reading Dickens "without ever having learned the letters of the alphabet"; the Crab reminds him he just did that with "MU", never seeing HOLISM and REDUCTIONISM.
Achilles laments that the top level probably does not encode the bottom: in the MU-picture, the top "says only 'MU'", unrelated to the lower levels. Then the Crab notices the smallest letters are wiggly; looked at closely, they spell "MU", over and over, "like a mantra". The highest and lowest levels coincide. Chance, or "a purposeful act carried out by some creator"? There is no way to decide. (This was the Tortoise's "octave lower" all along.)
The organ point: ants that didn't know what they said
The Tortoise announces an organ point (not, as Achilles thinks, a fermata, "a sort of musical semicolon", which he missed in the prelude), sustained on G. The Anteater then tells, in a long single-voiced stretch, of a day when lonely Aunt Hillary told him to help himself to the juiciest ants. He had been watching the symbols carrying her thoughts, which contained "some particularly juicy-looking ants", so he ate a few of the fattest from the very symbols that expressed "Help yourself to any of the ants which look appetizing." "The little bugs didn't have the slightest inkling of what they were collectively telling me."
The wraparound. Achilles: "They were completely unconscious of what they were participating in... Ah, what a pity, a supreme irony, in fact, that they missed it." The others then compliment "a lovely organ point", which Achilles, absorbed in talking, missed entirely. The Tortoise: "Ah, what a pity, a supreme irony, in fact, that you missed it." What it shows: Achilles is to the dialogue's form exactly what the ants are to Aunt Hillary's thought: a participant in a higher-level pattern he cannot perceive while he is busy producing it.
Aunt Hillary lives on a large, sumptuous estate, which seems inconsistent with ant communism. The communism is on the ant level; Aunt Hillary may not even know about it, just as most people know nothing of their neurons, being "somewhat squeamish creatures". She "gets rather antsy" whenever she thinks about ants, and is herself "a staunch believer in libertarianism, you know, laissez faire and all that".
The fermata and the tragedy of J. S. F.
Another picture: the Crab counts letters J, S, B, m, a, t; the Anteater sees "f, e, r, A, C, H"; Achilles reads the capitals as "J. S. BACH"; the Tortoise, slowing and pausing mid-phrase, reads the small letters as "fermat", and the others hear "a charming little fermata". Achilles, again, missed it.
Thought experiment: same ants, different colony. Setup: (1) the estate's former owner was Johant Sebastiant Fermant, "one of the most creative ant colonies who ever lived", mathematiciant by vocation, musiciant by avocation; (2) a freak thundershower, the kind that comes once a century, drenched him on a hot day; (3) every ant survived on floating sticks, but "there was no organization left": the caste distribution was destroyed, and the ants, like the pieces of Humpty Dumpty, could not rebuild it; (4) the Anteater put out sugar and cheese "hoping against hope", and wipes his eyes; (5) over months the same ants regrouped into a new organisation: Aunt Hillary; (6) does she show any of J. S. F.'s traits? "Not a one. They have nothing in common." The Anteater's verdict: there are "often several distinct ways to rearrange a group of parts to form a 'sum'"; Aunt Hillary "was just a new 'sum' of the old parts. Not more than the sum, mind you, just that particular kind of sum." What it shows: identity lies in organisation, not in material: the same ants can be two different minds, and a mind can die while all its parts live. (Contrast the Teleclone, where the same organisation in different atoms seemed to be the same person.)
The Tortoise claims that in number theory one can rearrange a theorem's symbols into a new theorem; Achilles suspects "one of his elaborate spoofs". J. S. F. Excelled at number theory; Aunt Hillary is "remarkably dull-witted" at mathematics and has "rather banal taste in music".
The parody recapitulation: Fourmi's Well-Tested Conjecture. The Anteater retells the Prelude's story in ant-speak (fourmi is French for ant): Lierre de Fourmi, "a mathematiciant by vocation but lawyer by avocation", reading the Arithmetica of "Di of Antus", saw the equation 2^a + 2^b = 2^c and wrote the familiar marginal note; "some three hundred days ago"; checked up to 125,000; proved at last by Johant Sebastiant Fermant, and still called "Johant Sebastiant's Well-Tested Conjecture" ("tradition has kept it this way"). His unpublished masterpiece was a giant fugue with twenty-four voices on twenty-four subjects, one per key, described in the margin of his Buxtehude: "I have added together the power of 24 keys, and the power of 24 themes; I came up with a fugue with the power of 24 voices. Unfortunately, this margin is too narrow to contain it." Hence "Fermant's Last Fugue". What it shows: the theme returns transformed, as fugue themes do, with every element mapped one level over (Fermat to Fourmi, years to days).
The fugue ends with "a strange new twist on its theme" and a last picture. The Crab reads "HOLISMIONISM", letters shrinking then growing; the Anteater "REDUCTHOLISM", growing then shrinking; Achilles "HOLISM written twice, with the letters continually shrinking"; the Tortoise "REDUCTIONISM written once, with the letters continually growing". And Achilles, at last: "I heard the new twist on the theme this time! ... Finally, I think I am beginning to grasp the art of listening to fugues."
Critical lens on the dialogue. (1) Fermat's Last Theorem was proved by Andrew Wiles (with Richard Taylor) in 1994–95, with no counterexample, as mathematicians expected; the 125,000 figure was the 1979 state of verification. (2) The Anteater's colonies "subjected to the rigors of evolution for billions of years" is wrong: ants arose roughly 140 million years ago in the Cretaceous (estimates vary), and multicellular life is well under a billion years old. (3) Real colonies do not encode knowledge in a "caste distribution" so much as in interaction rates and pheromone trails; the biology is a deliberate idealisation. (4) The claim that "any system which has a mastery of language" must share these levels is a hypothesis about architecture, not a finding, and today's language models are a live test of it. (5) The Anteater's "drain the colony of meaning" from the evolutionary perspective and recover it in the here and now is an early statement of Dennett's intentional stance: meaning is real as a pattern visible from a stance, not as an extra ingredient.
Reflections (D.R.H.)
"Is a soul greater than the hum of its parts?" The characters disagree, but all agree that the collective behaviour of a system of individuals "can have many surprising properties". Many readers think of countries, whose seemingly purposive, selfish, survival-oriented behaviour emerges from citizens' education, laws, religions, resources, consumption and expectations. When a tight organisation forms, especially when contributions cannot be traced to particular individuals, we see a higher-level individual and personify it: a newspaper said a terrorist group was "playing its cards extremely close to its chest"; Russia is said to "desire" recognition because it "suffers" from "a long-standing inferiority complex" towards Western Europe.
Individuals in organisations (secretaries, workers, bus drivers, executives) have their own goals, which might be expected to conflict with the organisation's. Yet there is an effect, which "many students of political science would regard as insidious and sinister", by which the organisation co-opts and exploits those very goals (pride, need for self-esteem) and turns them to its own profit. "There emerges from all the many low-level goals a kind of higher-level momentum that subsumes all of them, that sweeps them along and thereby perpetuates itself."
Try this: what is it like to be a bank? Hofstadter asks "What is it like to be China? How different from that would it feel to be the United States?", deferring detail to Nagel (Selection 24); it comes down to whether a country has a symbol level, as Aunt Hillary does. Run it on an organisation you know. A large bank "notices" a fraud pattern (a team forms, escalates, disbands), "decides" to exit a market, "remembers" a past crisis in its risk appetite. Is there a symbol level here, active structures that trigger one another and model the world, or only a caste distribution of staff and systems? Notice that the test is not whether any employee understands the strategy; Aunt Hillary's ants understand nothing.
What counts as a representational system
"Has a symbol level" can be restated as "is a representational system", a concept central to the book.
Definition: a representational system is "an active, self-updating collection of structures organized to 'mirror' the world as it evolves". It (1) is built on categories, sifting incoming data into them and refining or enlarging them when needed; (2) has symbols that interact by their own internal logic; (3) keeps going when cut off from the world it reflects, evolving along a probable path; (4) ideally sprouts parallel branches for anticipated possibilities, each with a subjective likelihood (the "superposition" of the Morowitz Reflections); and (5) thereby stays "pretty much 'in phase' with the world" without consulting it at every step.
| Candidate | Representational system? | Why |
|---|---|---|
| A painting | No | static, however representational |
| A mirror | No | sees no categories (one image, split only by the beholder); no autonomy (turn off the lights and it is gone) |
| A television | No | throws dots on a screen without regard to what they represent; patterns are passive copies |
| A vision program that says what is in a scene | Closer | categorises; "the most advanced... Computer vision hasn't yet cracked that nut" |
| A program that says what is in a scene, what probably caused it and what will probably ensue | Yes | the full sense intended |
| A country | Left for the reader | does it have a symbol level? |
Downward causality and the domino computer
The "caste distribution" or "state" is claimed to be a causal agent in the colony's future, which seems to contradict the idea that all behaviour comes from underlying laws, of ants, neurons, ultimately particles. Is there "downward causality", "a thought can influence the path of an electron"?
William Calvin and George Ojemann (Inside the Brain) ask what starts a neural firing. Sodium channels open because they are voltage-sensitive and a voltage pulse has hit them; the voltage was relayed node to node along the axon; the first node's shift, for interneurons, comes from the summed pulses of neurotransmitter from other neurons. (Hofstadter avoids calling these "upstream" neurons, because neural flow "make[s] loops all over the place, quite unlike rivers".) So: "What triggers a neural firing?" "Other neural firings!" The real question, why those neurons and not others, can only be answered by shifting levels to the ideas the brain encodes.
Thought experiment: the domino computer and 641. Setup: (1) an elaborately branching and rejoining network of dominoes, each with a time-delayed spring that stands it up again five seconds after it falls; (2) suitably configured, it can compute like any computer, with loops and branches ("not too different" from neural networks); (3) program it to factor 641; (4) point at a domino that never falls and ask why. Low-level answer: "Because its predecessor never falls", which begs the question. The satisfying answer: it sits in a stretch "activated only when a divisor is found. But 641 has no divisors, it is prime. So the reason that domino never falls has nothing to do with physics or domino chains, it is simply the fact that 641 is prime." What it shows: this does not mean higher-level laws override physics; "an explanation that makes any sense demands higher-level concepts". The dominoes don't know they are in a program, any more than piano keys know the piece, or neurons know they are thinking these thoughts, or ants that they serve the colony.
A further question, what laws made the program and dominoes exist at all, sends us back through society, the origin of life and finally to physicists and "the Big Bang as the primordial cause of everything". That is unsatisfying, because we want an answer in concepts familiar to people, and "fortunately, nature is stratified enough that this is often possible".
Hofstadter disowns a reading: not a "psychic" with furrowed brow beaming "waves of Plutonian energy" at a tumbling die; no "mental magnetism" or "semantic potential" that makes particles deviate from physics. The question is "where explanatory power comes from", how to reconcile everyday "cause" with the scientific use. As evolutionary biologists use "teleological shorthand", students of thought must translate fluently between reductionistic language and a "holistic" language "in which wholes do indeed exert a visible effect on their parts, do indeed possess 'downward causality'".
The rotating cylinder. In the fairground ride where riders line the wall of a spinning cylinder whose floor drops away, a tennis ball thrown across seems to veer crazily, even boomerang, because you move round while it flies straight. Unaware of the rotation, you would invent a deflecting force, and since it acts identically on equal masses, you might take it for a kind of gravity. That "fictitious forces" and gravity are easily confused "is at the heart of Einstein's great theory of general relativity". What it shows: a change of frame changes which causes and effects you perceive. "If it is good enough for Einstein, it ought to be good enough for us!"
Mechanism and soulism
Hofstadter's wordplay chart pairs the spatial and temporal versions of each stance:
| "Hard scientists" | "Soft scientists" | |
|---|---|---|
| In space (size-scales) | Reductionism: upward causality | Holism: downward ("introactive") causality |
| In time | Predictionism: only upstream events matter | Goalism: future goals project causal power backward ("retroactive causality") |
| Sum | Mechanism | Soulism |
Predictionism holds that only "upstream" events determine how the present flows into the future; goalism (the teleological view) sees animate things moving towards future goals, as if the future cast causes backward. "Put goalism and holism together, and you have, you guessed it, soulism! Put predictionism and reductionism together, and you get, mechanism."
The thinking wind chime
A second metaphor for brain activity. Picture a complex wind chime like a mobile, glass "tinklers" dangling like leaves from branches hung on bigger branches. When wind strikes, the whole structure slowly changes at all levels, and how a tinkler moves depends on the chime state (even a single string's twist) as much as on the wind. "Just as people do things 'of their own volition', so the chime seems to have a will of its own." Volition: "a complicated internal configuration, established through a long history, that encodes tendencies toward certain future internal configurations and away from others", present "in the lowliest wind chime".
Upgrading the wind chime, step by step
- A fan on a track, its position set by one branch's angle and its speed by another's: the chime now partly controls its own environment, "like having big hands... Guided by groups of tiny, insignificant-seeming neurons".
- Many blowers, one per branch: wind, natural or self-made, sets tinklers shimmering, which twists branches, which re-aim blowers; external wind and internal state become inextricably intertwined.
- Two chimes blowing on each other: is it one system or two? Decompose by top branches (five or ten parts each) or lower ones (twenty or more): "It is all a matter of convenience", though localised kinds of shimmering might mark out distinct "organisms"; all still "explicable in terms of physics".
- A mechanical hand steered by two dozen high-level branches, moving chess pieces: if its moves were always legal, and good, it would not be coincidence; "It would be because the chime's internal state had representational power."
In such a system "the tiniest external perturbation will trigger a myriad tiny interconnected events", a cascade; if that is the system's perception of input, its own state is "perceived" in the same way. "Self-perception cannot be disentangled from perception." There is no guarantee that a higher-level reading exists (that the chime state could be decoded into English sentences stating its beliefs, including the rules of chess and good play). But in systems shaped by natural selection the survivors will be those with "meaningful internal organization", and in the chime, the colony and the brain that organisation is stratified: top branches as the most compact abstract summary, quivering tinklers as the concrete local one; the caste distribution as the colony's most incisive view, "so abstract that the ants themselves are never mentioned". For brains we cannot yet find the high-level structures that would read out beliefs, except by asking the owner "what he or she believes".
No inner eye. In all three systems, semiautonomous subsystems represent concepts, and stimuli awaken them. There is no "inner eye" that watches the activity and "feels" it; "the system's state itself represents the feelings". A little person inside would need a smaller inner eye, and so on: "infinite regress of the worst and silliest kind." Self-awareness comes instead from the system's tightly intertwined responses to both external and internal stimuli. "Mind is a pattern perceived by a mind." This is perhaps circular, but it is neither vicious nor paradoxical.
The nearest thing to an inner eye is the self-symbol, "a complex subsystem that is a model of the full system". But it does not perceive by having smaller symbols of its own (another regress). Rather, "the self-symbol's joint activation with ordinary (nonreflexive) symbols constitutes the system's perception". Perception resides at the level of the full system; if the self-symbol "perceives", it is only as a male moth perceives a female, or your brain your heart rate, "at a level of microscopic intercellular chemical messages".
Finally, the reason for so many levels: the brain must be extraordinarily flexible in an unpredictable world: "Rigid programs will go extinct rapidly." A strategy for hunting dinosaurs is no good for woolly mammoths, still less for tending domestic animals or commuting by subway. An intelligent system must "sit back, assess the situation, and regroup" in deep ways, with only the most abstract mechanisms fixed. A many-layered system can keep specialised programs (chess, mammoth-hunting) at the surface and abstract ones deeper down:
Hofstadter's examples of deep, general-purpose programs
- Recognising patterns.
- Evaluating conflicting pieces of evidence.
- Deciding which of rival subsystems "clamoring for attention" gets priority.
- Labelling the current situation for retrieval on similar future occasions.
- Deciding whether two concepts really are analogous.
He stops at the edge of cognitive science, pointing readers to work on knowledge representation, especially Aaron Sloman's The Computer Revolution in Philosophy.
Critical lens on the Reflections. (1) A neurophysiological slip: the influx of sodium drives the action potential along the axon; neurotransmitter release at the terminal is triggered by calcium entry, not directly by sodium concentration as the gloss on Calvin and Ojemann implies. (2) "Downward causation" remains disputed: Jaegwon Kim's causal-exclusion argument (1990s) says that if physics is causally closed, higher-level causes are either redundant or epiphenomenal; Hofstadter's answer (it is about explanation, not extra force) concedes the metaphysics to Kim while keeping the explanatory point, a position later formalised in work on causal emergence (Erik Hoel and colleagues, from 2013). (3) "Mind is a pattern perceived by a mind" threatens circularity in the order of explanation, as he admits; he develops the idea at book length in I Am a Strange Loop (2007), where the self-symbol becomes the "strange loop".
What changed by 2026. Aunt Hillary has real relatives. Deborah Gordon's field studies of harvester ants showed colonies allocating tasks by local interaction rates with no central control, and ant colony optimization (Marco Dorigo, from 1992) turned pheromone trails into a standard algorithm for routing and scheduling. Neuroscience reached the "ant level" for a whole brain in 2024, when the FlyWire consortium published the complete wiring diagram of an adult fruit fly, about 140,000 neurons and some 50 million synapses, and found that a full parts list still leaves the "symbol level" to be worked out. The most striking echo is in AI: mechanistic interpretability research on large language models looks for exactly Hofstadter's active symbols, internal "features" that represent concepts and trigger one another, in networks whose individual units, like ants, mean nothing on their own; Anthropic's 2024 work extracting millions of such features from a production model is one example. For architects, the Ant Fugue remains the best short lesson in why you monitor systems at the right level: a dashboard of every packet is the ant-by-ant description, accurate and useless.
Arnold Zuboff's philosophical fable takes the "neural theory of experience" at its word: a dying man's brain is kept in a bath and fed pleasant experiences, then accidentally split, then wired, then fed by recorded "impulse programmers", then divided neuron by neuron across the galaxy, until two men named Spoilar realise that, on the project's own principles, every possible experience is already being realised by any random firing of one neuron of each type. At each step a lone sceptic, Cassander, names the condition being thrown away (proximity, causal connection, synchronisation, topology, neural identity, neural context) and is shouted down with "How would the neurons know?" The story is the book's first roadblock: if experience depends only on the pattern of neural firings, the pattern dissolves into everything, and physiology "undermines itself", unless some Cassanderish condition is restored. The editors' joint Reflections offer hints for resisting the slide, from an "unpacked" statue of David to TV cartridges and Wheeler's one-electron universe.
The young man: kind, wealthy, loved by friends, whose body is rotting but whose nervous system is sound; "he loved having experiences".
The scientist friends: believers in "the neural theory of experience", divided only on whether experience is caused by or consists in neural activity.
Fred: points out that mending the split brain will take two months.
Bert: proposes the wires, then the impulse programmers.
Cassander: the "grave fellow" who keeps naming the conditions being discarded; later, a line of "Cassanders".
Spoilar (two of them): the thinkers who, ten centuries on, bring the project down.
Part I: the brain in the bath, and the slide begins
The scientists' shared conviction: what "decisively controlled any particular experience of a man", whether it existed and what it was like, "was the state of his nervous system". So a brain in a nutrient bath, made to fire precisely as it would when its owner gazed through a hole in a frozen pond, would have the very experience Thoreau described: "the quiet parlor of the fishes, pervaded by a softened light as through a window of ground glass". A month later the brain floats in its bath, and paid subjects help identify the firing patterns of very pleasant situations, which an electrode machine induces.
The conditions discarded, one by one, and the reply each time
- Proximity of the hemispheres. A drunken watchman's arm splits the brain. Rather than wait two months for healing, Bert proposes tiny electrochemical wires from each severed neuron to its partner. Cassander: proximity might be "an absolute condition and principle" of whole-brain experience. Reply: "Would the damn hemispheres know they were connected by wires?"; hook up a mouth and he would tell us of his fun. Cassander mutters about "some experiential field, or some such".
- (Speed of transmission.) A real objection stops the wires: impulses over wire take slightly longer, garbling the timing. A man "with very little training in physics" suggests radio, thinking radio takes no time; he is "quickly disabused".
- Causal connection. Bert: replace transmission with an "impulse programmer" on each cut face that simply plays the impulses the other half would have sent, perfectly timed. The halves can then sit in labs across town, and more people can join. Cassander: is causal connection between the halves an essential condition? Reply: "How did the neural activity know whether a radio-controlled or programmed impulse cartridge fed it?"
- Synchronisation. Since each half gets exactly what it would have received, why run the halves at the same time? "What is there to worry about if at one lab they run through one half of a pattern one day and at the other lab... Another day?"
- Topology. Must the hemispheres keep facing each other in their original spatial relation? Cassander's warnings "ignored".
Part II: ten centuries later, the Great Experience Feed
Humanity now fills the galaxy, and billions want "the thrill and responsibility" of the "Great Experience Feed". If two programmed halves make a whole experience, so do four quarters, and so on, until each neuron has its own bath and its own human attendant, with a cartridge on both ends transmitting and receiving impulses as programmed. Later Cassanders gave up on proximity (to avoid infuriating everyone who wanted a piece) but urged keeping topology (each neuron's relative position and orientation) and chronology (the original temporal pattern). Topology was derided ("How should each of the neurons know... Where it is in relation to the others?"). But timing and order crept back "without the Cassanderish reflectiveness", perhaps because ignoring them "would have reduced the art of programming to absurdity"; everyone simply assumed the "right" order mattered.
Neurons get replaced when worn, violating "neural identity", a condition "never taken very seriously even by Cassanders", since metabolism replaces a neuron's matter anyway; and even if replacement produced "a new identity for the experiencer", there would still be an experiencer having the same experience.
The first Spoilar: one neuron for all. Setup: (1) a neighbour's bath shatters five minutes before his neuron's scheduled firing; (2) he begs to use Spoilar's neuron, the same type; (3) Spoilar reasons: move the bath to the old position? Topology is "baloney", so don't move it; fire it for the neighbour, then for himself; (4) then why not fire one neuron of each type in place of all neurons like it, "again and again and again"? (5) but how would a neuron know it was repeating, or the order of its firings? So one neuron of each type firing once realises all patterns; (6) and those could be "any of those naturally firing in any head. So what are we all doing here?" The desperate conclusion: if all possible experience is realised by that bare minimum, no experiencer can believe he is connected to any more physical reality than that; physiology "provides so cheaply for so much experience that we can never know what is an actual experience of it", and "belief in such a system undermines itself", "unless it's tempered with Cassanderish principles".
The second Spoilar: neural context. Setup: (1) he likes stringing neurons into chains; (2) finding his neuron mid-chain when it is due to fire, he hooks the chain's two ends to the cartridge and times the impulse to reach his neuron through the chain; (3) his neuron now takes part in two patterns at once, the chain's (with proximity and causal connection) and the programmed experience; (4) he mocks "the condition of neural context": he could hook his neuron to "all those in your head" and it would serve as well; (5) one night saboteurs, men refused entry to the project, kill many neurons around him; looking at the stooping attendants, he notices next to every bath there is a head, with billions of neurons, millions of each type firing at any moment; (6) so any spread brain's pattern is already being realised in the operators' heads, "even one of those heads", where a loose proximity is even satisfied. The conclusion: since all brains are spreadable, the same holds for his own experiences, which "might exist only as some such floating cloud", including those that convinced him of physiology. "Unless Cassander is right, to some extent, then physiology reduces to absurdity."
"Such thinking killed the great project and with it the spread-brain." What men concluded next "is another story".
The crowd's principle: a condition matters only if the neurons could "know" about it, i.e. Only if its absence would change the firings; so proximity, causation, timing and place are all inessential.
Cassander's principle: some conditions (proximity, causal connection, synchronisation, topology, context) may be constitutive of a single experience even though no neuron registers them; the pattern must be held together by real causal organisation, not merely occur.
Reflections (D.C.D. And D.R.H.)
The tale "seems at first to be a sly demolition of virtually all the ideas exploited in the rest of the book", a reductio ad absurdum of brain-experience assumptions that had seemed innocent. How to resist "the daffy slide"? The editors offer hints rather than a refutation.
The unpacked David. Someone claims to own a microscopically exact replica of Michelangelo's David, in marble. You find a twenty-foot block of pure white marble in his living room. "I haven't gotten around to unpacking it yet, but I know it's in there." What it shows: a pattern that is "in" the material only in the sense that the material could be carved into it is not the pattern; Spoilar's "every experience is in any firing" makes the same move.
IBM's TV cartridges. Zuboff says little about the impulse cartridges; they seem "mere beepers". Consider an "easier" feat: strikes close all television stations, and IBM mails out cartridges that produce ten realistic channels of news, weather, soap opera and sport, all made up, since "TV signals are just impulses; our cartridges simply take a shorter route to the receiver". What is inside? Videotapes of real actors, or animation? Animators say composing realistic frames from scratch "grows exponentially as you try for greater realism". "Only the real world is rich enough in information to provide (and control) the signal trains" for realistic TV. Making up a whole world of perception is the task Descartes gave his evil demon, rightly made infinitely powerful: "no lesser deceiver could sustain the illusion without falling back on the real world after all." What it shows: the programmers in Zuboff's story would have to contain, somewhere, the information of a real experiencer's world and brain; the pattern does not come free.
These "strike glancing blows". Can they be made fatal? Try a parallel argument: no need for books, just print the alphabet once; why the whole alphabet, why not one letter, one stroke, one dot? And Raymond Smullyan's piano method: become intimate with each note individually, a whole month on middle C, a few days on the extremes, and don't forget rests (a day on whole-note rests, two on half-note rests, four on quarter-note rests...). "Once you've completed this arduous training, you're ready to play anything! It sounds right, but, somehow, slightly wrong as well."
Finally, John Archibald Wheeler once speculated that all electrons are alike because there is only one electron, "careening back and forth from the ends of time", weaving the universe by crossing its own path. "Perhaps Parmenides was right: there is only one thing!" But that one thing has spatiotemporal parts in "astronomically many relations", and this relative organisation matters. To whom? To those portions of the great woven whole that are perceivers. And how are they distinguished from the rest? The Reflections end on that question.
Critical lens. (1) The editors never quite say which step is invalid, and the literature has since sharpened the issue. Tim Maudlin's "Computation and Consciousness" (1989, the "Olympia" machine) argued that what a system would do in counterfactual situations seems irrelevant to experience, yet functionalism needs it; David Chalmers replied that implementing a computation requires the right counterfactual causal structure, which is exactly what Bert's programmers remove at step 3. On that view the slide stops at "causal connection", vindicating Cassander. (2) Hilary Putnam (1988) argued that any open physical system (a rock) implements every finite automaton, a cousin of Spoilar's conclusion; the counterfactual reply answers both. (3) Zuboff himself drew a different moral in later work, a radical view of the unity of all experience; the editors treat the story as a roadblock to be got around, not as a positive theory.
What changed by 2026. Spoilar's "floating cloud" became Greg Egan's "dust theory" in the novel Permutation City (1994), and it haunts modern cosmology as the Boltzmann brain problem: theories that predict more random, momentary observers than ordinary ones undermine the evidence for themselves, exactly Spoilar's self-undermining argument. Nick Bostrom's simulation argument (2003) asks a related question about simulated observers, though it assumes the full causal machinery Zuboff strips away. For AI and computing the lesson is concrete: a replayed log of a model's activations is not the model running, and whether consciousness could attach to a recording is now asked about AI systems in the AI-welfare literature. Engineers already know Cassander's point in another form: a system's identity lives in its live causal wiring and timing, which is why distributed systems that lose ordering guarantees stop being the same system.
Part IV explores the idea, prepared in Part III, that a mind is software: an abstract pattern whose identity does not depend on any particular physical embodiment, and which could therefore be moved, copied, backed up or run in two places at once. It opens with Dennett's comic memoir of a brain left behind in Houston while its body goes to Tulsa, and the selections that follow test the same idea from other directions.
If minds are programs, the old fantasies of the transmigration of souls and the Fountain of Youth become engineering problems, and every engineering trick raises a puzzle about identity. Dennett's "Where Am I?" separates brain from body by radio and then adds a computer duplicate of the brain, so that the questions "where am I?" and "which one am I?" come apart from "where is my brain?". Sanford's sequel, "Where Was I?", retells the Tulsa mission with a man operating a robot body by full telepresence, and asks whether he was where the robot was. Leiber's "Beyond Rejection" imagines minds recorded on tape and installed in vacant bodies, and the struggle to make a mind accept its new body. Rucker's "Software" has an ageing engineer, Cobb Anderson, offered immortality by robots who will extract his "software" and run it in a new body, at the cost of his old brain. Cherniak's "The Riddle of the Universe and Its Solution" imagines a piece of information that sends anyone who grasps it into an irreversible coma, the mental equivalent of a computer virus.
The common thread is the separation of hardware from software in persons. The editors are friendly to it, but each story also exposes a cost: copies diverge, embodiment matters, uploading destroys the original, and a mind that is a program can be hacked. The Introduction's Teleclone Mark V, which copies without destroying, is the shadow over the whole Part.
Part IV's selections and the question each pushes
- Selection 13, "Where Am I?" (Dennett): is a person located where the brain is, the body is, or the point of view is? What if the brain has a duplicate?
- Selection 14, "Where Was I?" (Sanford): if you operate a robot body by telepresence, where are you?
- Selection 15, "Beyond Rejection" (Leiber): a mind installed in a new body; is the body just hardware?
- Selection 16, "Software" (Rucker): immortality by copying the program; is the copy you?
- Selection 17, "The Riddle of the Universe and Its Solution" (Cherniak): can a piece of information disable the mind that grasps it?
Daniel C. Dennett's "Where Am I?" (first delivered at the Chapel Hill Colloquium and printed in his Brainstorms, 1978) is a deadpan first-person account of a Pentagon mission on which his brain stayed in a vat in Houston while his body, linked by radio, went to recover a radioactive warhead a mile under Tulsa. Dennett names his brain Yorick and his body Hamlet, tries and fails to think "here I am, in the vat", loses his body when the radio links fail, wakes in a new body, discovers that a computer copy of his brain called Hubert has been running in parallel, and ends his lecture when his duplicate, out of sync for two weeks, seizes the microphone. The story pulls apart three things that normally coincide (where my brain is, where my body is, and where my point of view is) and then adds a fourth puzzle, two brains running one person, which turns "Where am I?" into "Who am I?" Dennett's own Reflections admit that the synchronised duplicate "would verge on the miraculous" and ask whether the tale is coherent.
Dennett (the narrator): philosopher "of firm physicalist conviction" and Pentagon volunteer.
Yorick: his brain, in a life-support vat at the Manned Spacecraft Center in Houston.
Hamlet: his original body, sent to Tulsa and lost underground.
Fortinbras: the new body he wakes in, bearded and heavier, of undisclosed history.
Hubert: a computer duplicate of his brain, run in parallel with Yorick "for reasons obscure to me".
The project director and the Houston team: neurologists, haematologists, biophysicists, electrical engineers.
The mission: leave your brain behind
Having "won my suit under the Freedom of Information Act", Dennett can reveal the episode. The Pentagon, with NASA and Howard Hughes, had spent billions on a Supersonic Tunneling Underground Device (STUD) to deliver a warhead through the earth's core "right up the Red's missile silos". A test lodged a warhead about a mile under Tulsa, Oklahoma. It emits a new kind of radiation that damages brain tissue only, and no shielding works, so the person sent to recover it must leave his brain behind. Each severed nerve pathway would be restored by a pair of microminiaturised radio transceivers, one on the brain, one on the nerve stump in the empty skull: "No information would be lost, all the connectivity would be preserved." The surgeons' reassurance: "Think of it as a mere stretching of the nerves... We're simply going to make the nerves indefinitely elastic by splicing radio links into them."
After days of tests and demonstrations he agrees; they take down his autobiography, his "beliefs, hopes, fears, and tastes", even his favourite stereo recordings, and give him "a crash session of psychoanalysis". Waking, he asks the hackneyed question, "Where am I?" "You're in Houston," says the nurse, which "still had a good chance of being the truth one way or another". In the mirror, tiny antennae poke through titanium ports in his skull.
Thought experiment: staring at your own brain. Setup: (1) Dennett is led to the vat; a brain floats "in what looked like ginger ale", covered in chips, tubules and electrodes; (2) "Is that mine?" He flips the output transmitter switch to OFF and slumps, groggy and nauseated, until a technician restores it; (3) he thinks, "Here I am sitting on a folding chair, staring through a piece of plate glass at my own brain..."; (4) but shouldn't he think, "Here I am, suspended in a bubbling fluid, being stared at by my own eyes"? He tries, and cannot do it "with any conviction". The intuition it pumps: as a physicalist he believes his thoughts are tokened in his brain, yet when he thinks "here", the place is outside the vat, where he stands. What it shows: the felt location of the self tracks point of view, not the brain.
He practises: "The sun is shining over there", aimed in turn at the lab's sunlit corner, the hospital lawn, Houston, Mars and Jupiter, then at "the upper left quadrant of a freckle on my arm". "There" hops anywhere. "Here in Houston", "here in the lab", "here in this part of the lab" all work; "here in the vat" stays "an unmeant mental mouthing", even with eyes closed. And he cannot be sure, which is itself unsettling: could he think he meant one place by "here" when he meant another? Either he is doomed to systematically false indexical thoughts, or a person's location (and where his thoughts are tokened "for purposes of semantic analysis") "is not necessarily where his brain, the physical seat of his soul, resides".
Yorick, Hamlet and Dennett: three answers
He falls back on "a favorite philosopher's ploy": naming. "Yorick, you are my brain. The rest of my body, seated in this chair, I dub 'Hamlet'." "So here we all are: Yorick's my brain, Hamlet's my body, and I am Dennett. Now, where am I?" The thought "where am I?" has temporal coordinates; must it not have spatial ones?
Dennett's three candidate principles
- "Where Hamlet goes, there goes Dennett." Refuted by the familiar brain-transplant cases: if Tom and Dick swap brains, the one with Dick's old body claims to be Tom and tells Tom's most intimate memories. "In a brain transplant operation, one wanted to be the donor, not the recipient." Better to call it a body transplant.
- "Where Yorick goes, there goes Dennett." Unappealing: how could he be in the vat, going nowhere, "when I was so obviously outside the vat looking in and beginning to make guilty plans to return to my room for a substantial lunch?" It begs the question, but seems to get at something.
- "Dennett is wherever he thinks he is." Generalised: a person has a point of view, whose location (fixed internally by its content) is the person's location. A step in the right direction, but it promises "heads-I-win/tails-you-lose" infallibility about location, though people often get lost.
The bank robbery jurisdiction puzzle. To support his intuition against principle 2, Dennett tries "a legalistic sort of argument that might have appealed to Locke". Suppose he flies to California, robs a bank and is caught. Is he tried in California, where the robbery happened, or Texas, "where the brains of the outfit were located"? Is he a California felon with an out-of-state brain, or a Texas felon remotely controlling an accomplice? Perhaps an interstate, hence Federal, offence. Convicted, would California jail Hamlet while Yorick "luxuriously" takes the waters in Texas, or Texas jail Yorick while Hamlet takes "the next boat to Rio"? The state would have to keep Yorick alive (barring cruel and unusual punishment), perhaps moving him to Leavenworth, and Dennett "would consider myself a free man" there. If so, jailing the brain fails to jail the person, which suggests principle 3.
Point of view is unclear, though. It is not fixed by one's beliefs: the Cinerama viewer who shrieks on the roller-coaster footage has not forgotten he is safe in his seat; this is "an illusory shift in point of view". Less clearly illusory: workers handling dangerous materials with feedback-controlled mechanical arms "can feel the heft and slipperiness of the containers they manipulate with their metal fingers"; they know where they are, yet "it is as if they were inside the isolation chamber", and with effort they can shift back and forth "rather like making a transparent Necker cube or an Escher drawing change orientation". It seems extravagant to say they transport themselves each time.
Dennett's "try this": the Dangerous Brain Clinic. He invites the audience to practise the trompe l'oeil. Imagine you wrote an inflammatory letter to the Times, and the government has impounded your brain for three years' probation in its Dangerous Brain Clinic in Bethesda, Maryland, while your body is left free to earn taxable income. Right now your body sits in an auditorium hearing Dennett's account. "Think yourself to Bethesda, and then hark back longingly to your body, far away, and yet seeming so near", restraining "with long-distance restraint (yours? The government's?)" the urge to clap and head for "a well-deserved glass of evening sherry in the lounge". Difficult, but "if you achieve your goal the results might be consoling." What it shows: whether you can adopt the vat's point of view is a matter of habit, "presumably independent of the truth about the location of one's brain".
Life with a long-distance brain
He habituates, but time lags remain: "The speed of light is fast, but finite", and as brain and body separate, feedback loops fall into disarray. As delayed hearing of one's own voice leaves one nearly speechless, he can scarcely track a moving object with his eyes when brain and body are more than a few miles apart, and "can no longer hit a slow curve ball with the authority of yore". Compensations: liquor still tastes good and corrodes his liver, but he never gets drunk (he sometimes feigns it); he takes aspirin orally for a sprain, but for persistent pain asks Houston for codeine in vitro. "In times of illness the phone bill can be staggering."
He flies to Tulsa, or "that's the way it seemed to me... Just off the top of my head as it were." On the way his panic subsides: he is in two places at once, "just as one can stand with one foot in Connecticut and the other in Rhode Island", "one of those scattered individuals". The more obviously true this answer seems, "the less important the question to which it could be the true answer seemed", a sad fate for a philosophical question. Yet a question lingers, neither "where are my parts?" nor "what is my point of view?", for it seemed undeniable "that in some sense I and not merely most of me was descending into the earth under Tulsa".
Disembodied in Houston
At the warhead the Geiger counter is off the dial, and he is glad his brain is elsewhere. Houston radios instructions; he sets to work with a cutting torch, and the transceivers fail one by one: he goes stone deaf, then his voice is paralysed, then his right hand goes limp, then he goes blind. "Deaf, dumb and blind, in a radioactive hole more than a mile under Tulsa." Then the last link breaks, and "whereas an instant before I had been buried alive in Oklahoma, now I was disembodied in Houston." The shift of perspective that had been "well nigh impossible" now seems natural; thinking himself back into the body takes effort, "for surely it was an illusion to suppose I was still in Oklahoma".
The "immateriality of the soul" on physicalist premises. As the last signal died, had he not changed location from Tulsa to Houston at the speed of light, with no increase in mass? What moved was "surely myself, or at any rate my soul or mind, the massless center of my being and home of my consciousness". A physicalist could quarrel only by "banishing all talk of persons", which would be as "systematically disingenuous" as the Cartesian "non sum". What it shows: the joke is that a change in which body you can perceive and act through looks, from inside, exactly like a soul travelling; Dennett marks it as one of "those rushes of revelation of which we should be suspicious".
Panic and nausea follow, eerily without their bodily phenomenology: no adrenaline tingles, no pounding heart, no salivation. A "dread sinking feeling in my bowels" raises false hope of "a gradual undisembodiment" but is only the first of the phantom-body hallucinations any amputee may expect. Torn between philosophical elation ("trying to figure out how to communicate my discovery to the journals") and lonely dread, he is sedated, and wakes to his favourite Brahms piano trio "with magnificent fidelity", hearing without ears: the stylus output was being fed "directly into my auditory nerve". "I was mainlining Brahms." The project director, speaking into a microphone that is now his prosthetic ear, promises re-embodiment.
Fortinbras and Hubert
He sleeps for most of a year and wakes in a new body: bearded, heavier, a family resemblance in the face, its history undisclosed (he "decided, wisely... Not to pry"). The person survives a new body, and after adjustment so does most of the personality; plastic surgery and sex-change operations cause bigger changes, and no one doubts survival there. He soon cannot even recall the body's novelties. The antennae remain: Yorick is still in Houston.
The switch that does nothing. Setup: (1) Dennett, in his new body Fortinbras, visits Yorick and on a whim flicks the output switch off again; (2) nothing happens; (3) the explanation: before the first operation they built a computer duplicate of his brain, matching its information-processing structure and speed, named Hubert; (4) after surgery they ran Hubert alongside Yorick, feeding both Hamlet's inputs and checking outputs, which were "identical and synchronous" for weeks (not proof, but "greatly encouraging"); (5) they kept Hubert in parallel during his disembodiment, and have now switched Hubert, for the first time, into control of Fortinbras; (6) a master switch marked B (Brain) and H (Hubert): flipping it, "with my heart in my mouth (and my brain in its vat)", produces only a click; he can switch mid-sentence "without a pause or hitch of any kind". What it shows: if two physically different brains running the same program are interchangeable from the inside, then the person is identified with the program, not with the organ.
He now has "a spare brain", and it seems not to matter which he uses. The unsettling prospect: someone detaches the spare and hitches it to another body, "some Johnny-come-lately Rosencrantz or Guildenstern". Then "there would be two people, that much was clear", one him, the other "a sort of super-twin brother". Which would the world recognise as "the true Dennett", and which would be me?
Causal priority: the Yorick-brained one, through Yorick's former intimate relationship with the original body Hamlet. But that seems "too redolent of the arbitrariness of consanguinity and legal possession".
Squatter's rights: if for years Hubert had driven Fortinbras while Yorick was the spare, the Hubert-Fortinbras couple "would seem... To be the true Dennett and the lawful inheritor of everything that was Dennett's".
More pressing: his strongest intuition is that he would survive so long as either brain-body pair survived, but he has mixed feelings about both surviving. Two Dennetts would compete for his wife's affections and his "modest professor's salary"; worse, "the idea of knowing that much about another person, while he had the very same goods on me". His colleagues see the bright side: one could stay home as professor and family man while the other travelled; "I could be faithful and adulterous at the same time", even "cuckold myself". His ordeal had made him less adventurous. Still worse would be a spare left detached and disembodied, a claimant to his name and possessions. So everything in the lab is locked behind fail-safes, and he carries the only master switch, radio-controlled, strapped to his waist. He switches every few months, only with friends present, in case the other channel is dead or occupied and someone must switch him back "from the void". The positions are unmarked, so he never knows which brain he is switching to ("if it is true that in one sense I don't know who I am then that's another one of your philosophical truths of underwhelming significance").
The twist: the other Dennett speaks
"Every time I've flipped the switch so far, nothing has happened. So let's give it a try...." And a new voice bursts out: "THANK GOD! I THOUGHT YOU'D NEVER FLIP THAT SWITCH!"
Two weeks in purgatory. The other Dennett's story: about two weeks ago the two brains "drifted just a bit out of synch"; once started, the drift "snowballed", since slightly different receptive states amplified differences in the same input; "the illusion that I was in control of my body, our body, was completely dissipated"; it was "like being carried around in a cage, or better, like being possessed", hearing his own voice say things he didn't mean, watching his hands do things he didn't intend, his itches scratched "not the way I would have", kept awake by the other's tossing; "YOU DIDN'T EVEN KNOW I EXISTED!" Now it is the first speaker's "turn in purgatory", but "at least you'll have the comfort of knowing I know you're in there". "Like an expectant mother, I'm eating, or at any rate tasting, smelling, seeing, for two now." After the colloquium they will fly to Houston to get one of them another body ("You can have a female body, your body could be any color you like"); if both want this one, "the project director [can] flip a coin". His closing words to the audience: "this talk we have just heard is not exactly the talk I would have given, but I assure you that everything he said was perfectly true. And now if you'll excuse me, I think I'd, we'd, better sit down." What it shows: two "identical" programs on different hardware stay one person only while they stay in lockstep; the moment they diverge, there are two, and one can be trapped as a passenger with no one aware of him.
Try this: the unmarked switch, with stakes. You hold Dennett's switch and know the other channel has drifted for a week. Flipping it will hand control to a Dennett with a week of private, frustrated experience, and silence you. Not flipping it leaves him a passenger. Which is the right thing to do, and does your answer depend on which of you is "really" Dennett? If it does not, the question "which one is me?" may matter less than the question "how should two such persons be treated?", which is roughly where Dennett's own later views ended up.
Reflections (D.C.D.)
"The story you have just read not only isn't true (in case you wondered) but couldn't be true." The technology is impossible now and some may remain so, but what matters is whether the tale is incoherent in principle. When fantasies grow too outlandish (time machines, duplicate universes, infinitely powerful deceiving demons), "we may wisely decline to conclude anything from them", since our sense of understanding may be "an illusion produced by the vividness of the fantasy". The surgery and microradios are "innocent" science fiction.
Hubert is the problem. Hubert runs in perfect synchrony with Yorick for years, with no corrective links. That "would verge on the miraculous": a computer matching the brain's speed on millions of parallel channels would need an architecture unlike existing computers, and even then "its sheer size and complexity would make the prospect of independent synchronic behavior virtually impossible". Without synchrony, the premise that there is one person with two brains collapses. The supposition is "truly outrageous", in the league of Hilary Putnam's Twin Earth (an atom-for-atom duplicate planet) or a universe created five days ago complete with memory-laden adults, ancient-looking libraries and brand-new fossils.
We need conflict or strong difference to shake our natural assumption that to one body there corresponds at most one agent.
Ronald de Sousa's parable makes the point. When Dr. Jekyll changes into Mr. Hyde, we suspect two people taking turns in one body. But Dr. Juggle and Dr. Boggle also take turns in one body, and are "as like as identical twins". Why say they change into each other at all? "Well, why not: if Dr. Jekyll can change into a man as different as Hyde, surely it must be all the easier for Juggle to change into Boggle, who is exactly like him." Only difference reveals plurality; perfect synchrony hides it, which is exactly why the story's ending (drift reveals a second person) is so unsettling.
So a prosthetic brain like Hubert is "only a possibility in principle", though "less marvelous bits of artificial nervous system may be just around the corner". Crude artificial TV eyes for the blind already exist, some feeding the visual cortex directly, others routing images through intact senses: fingertip touch receptors or "an array of tingling points spread across the subject's forehead, abdomen, or back". Such nonsurgical mind extensions are the subject of the next selection, David Sanford's sequel.
Critical lens. (1) Principle 1's refutation by brain-transplant intuition assumes what is at issue; "animalists" (Eric Olson, The Human Animal, 1997) hold that you are the organism, so the brainless body in a transplant case would be you, however surprising. (2) Principle 3 ("wherever he thinks he is") is undercut by Dennett's own Cinerama and remote-arm cases, and he never settles on a replacement; his later view (Consciousness Explained, 1991) treats the self as a "center of narrative gravity", a useful abstraction with no precise location, which dissolves rather than answers the question. (3) The Reflections call Hubert's synchrony miraculous but do not draw the full moral: if exact functional duplication is practically impossible, the copying cases that dominate Part IV (and the Introduction's Teleclone) may rest on conditions that never hold. (4) The "speed of light" journey from Tulsa to Houston is a joke on dualists, but it also shows that "location of a person" may simply be indeterminate in such cases, a conclusion Parfit would later embrace.
What changed by 2026. Pieces of Dennett's fantasy now exist. Sensory substitution matured from Paul Bach-y-Rita's tactile vision chairs (1960s) to tongue-display devices, and cochlear implants feed sound to the auditory nerve, a modest form of "mainlining Brahms". Brain-computer interfaces let paralysed people move cursors and robotic arms by thought; Neuralink implanted its first human patient in January 2024, and in 2023 speech neuroprostheses decoded attempted speech at around 60–80 words per minute. Remote teleoperation (surgical robots, bomb-disposal arms, remote vehicles) is where the speed-of-light lag Dennett joked about is a real design constraint. Whole-brain emulation remains a research programme, not a product; the 2024 fruit-fly connectome shows how far there is to go even to map a small brain. And the Hubert problem has a software twin: running two copies of the same stateful service and keeping them in sync without corrective links is notoriously hard, which is why distributed systems use consensus protocols, and why "split-brain" is the engineer's name for exactly the failure in Dennett's last scene.
David Hawley Sanford's sequel to Dennett's "Where Am I?" keeps the whole body intact and moves only the point of view, by dressing a man in sensory "garments" that pipe a robot's sights, sounds and touches into his own organs. First presented to a Chapel Hill seminar run by Douglas C. Long and Stanley Munsat, it tests Dennett's principle that a person is wherever his point of view is, then pulls the rug out: the robot the narrator thought he was "in" was a costume he was wearing. If you cannot tell the difference between being at the robot and wearing it, your felt location is fixed by the point of view, and the point of view turns out to be shockingly cheap to move.
David Hawley Sanford: the narrator, a philosopher at Chapel Hill who claims his "sabbatical in New York" was really secret Defense work on the Tulsa warhead.
"Sanford": the name he uses for himself, the answer to "Where was I?"
"David": the saltwater-and-carbon body kept in Houston.
"Hawley": first the robot's name; then three robots: Little Hawley (three-fifths scale, already in Tulsa), Metal Big Hawley and Plastic Big Hawley (both in Houston).
Dr. Wechselmann: the scientist in charge (German Wechsel, "switch"), who stages the final surprise.
A rival agent in the Dennett affair
Sanford opens in Dennett's comic register. Dennett, "or perhaps one of the representatives from the corporation that collectively comprises him", drew an unprecedented standing ovation at Chapel Hill; Sanford was absent, supposedly on sabbatical, actually working for the Department of Defense on the same case. Dennett got so absorbed in his own nature, unity and identity that he forgot the mission's point: retrieving a fiercely radioactive warhead stuck a mile beneath Tulsa. Dennett neither knows nor cares what became of it; Sanford claims an essential role in its retrieval.
The opening move is a reflection on a detail of Dennett's story: Yorick, the brain in the vat, was once fed Brahms directly into the auditory nerves. A certain theorist would ask why stop there: why not feed the next stage in, and the next, until we reach the "real core person, the true seat of the soul"? Others would see the same sequence as a layer-by-layer transformation from the outside in of an organic subject into an artificial intelligence. The NASA scientist asked a humbler question: why disconnect the ears at all? Radiation, it turned out, damaged more than brain tissue (Hamlet's organic ears went first), so it would have been better to leave the ears on the brain, use earphones, and put microphones on the travelling body. A head-turning Hamlet in a concert hall would even preserve the tiny volume and time differences between the ears that locate a sound.
The kit: eyevideos, skintact and MARS
That modest improvement becomes the model for the technicians' radical advances. By the time Sanford signs up, the gear is perfected.
Sanford's four prostheses of presence
- Earphones and microphones: the ears stay attached and are fed sound gathered at the remote head, with binaural cues intact.
- Eyevideos: "to seeing what earphones are to hearing". They project an image on the retina and track every eye movement, slaving the remote cameras to each saccade and head turn. Only very fine print shows a slight loss of acuity; until tuning, night vision was actually better.
- Skintact: a light porous layer worn next to the skin; when the robot's transmitter-skin strokes a damp puppy, the real hand's nerves fire exactly as if it had, and warmth is signalled without heat.
- MARS (Motion and Resistance System): a membrane between the human's skintact and the robot's, duplicating the human's movements in the robot and returning the robot's pressures and resistances to the human's limbs.
Eyevideos invite experiments. Mount the cameras upside down and you repeat the classic inverting-lens studies. Try the "rabbit mount" (cameras facing opposite ways), the rabbit mount with extreme wide-angle lenses for 360-degree vision, or the "bank or supermarket mount", with the two cameras on opposite walls of your own room, from which you can see all sides of an opaque cube at once.
The robot sent underground contains no living cells. It has Sanford's proportions and joints, is covered with transmitting skintact, has cameras and microphones in its head, and has no mouth, jaws, lungs or gut: a loudspeaker replaces the mouth, voicing whatever the microphone before Sanford's lips picks up. The plan was that, unlike Dennett, Sanford would stay whole and safe in Houston, like a lab worker using remote manipulators, only with an entire feedback-controlled body. "Little did they know."
Naming, and the failure of every location rule
Once rigged up, Sanford could not resist locating himself where the robot was. Like Dennett wanting to see his brain, he wanted to see himself swathed in electronics, and found it as hard to identify with the body that turned its head whenever the robot did. He copies Dennett's naming device: "Sanford" names whatever answers "Where was I?", "David" the body in Houston, "Hawley" the robot.
- "Where Hawley goes, there goes Sanford" fails. The first robot now sits in a classified science museum and Sanford does not; the robot could also be driven by another body before or after. At most Sanford went where Hawley went while Hawley was in communication with David (or a David replica). Dennett's "Where Hamlet goes, there goes Dennett" needs the same qualification.
- "Hawley" stops referring. There are three robots, two full-size (one mostly plastic, one mostly metal, alike outside and "from the inside") and Little Hawley, which did the job in Tulsa.
- Discontinuous travel. The technicians soon switched channels without waiting for David to sleep. After Little Hawley's triumphant return, "the three of us, or the three of I" played three-corner catch, helpers propping up the inactive robots, and Sanford seemed to jump from place to place without passing between.
- "Where David goes, there goes Sanford" fails too, for epistemological rather than Dennett's legalistic reasons: he had not seen David for days and could not be sure David still existed. Soon after the rig went on, his experiences of breathing, chewing, swallowing, digesting and excreting were cut off.
Doubts: duplicates, dismemberment and an Arnauldian argument
When Plastic Big Hawley spoke, Sanford could not know whether David's diaphragm, larynx and lips were still involved; the scientists could tap the nerves directly. If David's kidneys failed or a coronary artery blocked, everything but the brain might be dead, or the brain too, replaced (as Yorick had been) by a computer duplicate. He might already be robot, computer, or both, like L. Frank Baum's Nick Chopper, the Tin Woodman, who became tin a part at a time. And if one duplicate, then "two or three or twenty", each driving a robot or a brainless body: fission cases without further technology.
The Cartesian temptation (after Arnauld's report of Descartes). (1) I can doubt that the body David, or its brain, exists. (2) I cannot doubt that I see, hear, feel and think. (3) So I cannot be identical with David or its brain, or in doubting them I would doubt myself. What it pumps: the sense that the self floats free of any particular hardware. Why it fails: what I can doubt about a thing is a fact about my knowledge, not about the thing (I can doubt that the Morning Star is Venus); Sanford only says he was "tempted", and the story's ending shows the doubted body was there all along.
David could also be taken apart into living, functional parts: eyes in one room, limbs on artificial blood in others, the brain itself split with fast links, or its silicon replacement spread out as in Dennett's "Toward a Cognitive Theory of Consciousness". The spatial contiguity or chemistry of the subsystems producing "my thoughts, actions, and passions" seemed irrelevant to his location, unity or identity.
Point of view is location
Dennett's third principle, "Dennett is wherever he thinks he is", invites misreading: thinking you are in Chapel Hill does not put you there. Sanford's gloss: the location of a person's point of view is the location of the person. Point of view is more than seeing; it includes other senses, movement, and the head and eye movements that change what is seen. Because the Hawleys preserved almost all normal sensorimotor relations with their surroundings, the robot's spatial unity was "more than enough" for a unified sense of place; the prospect of Hawley's disassembly was more unsettling than David's dismemberment.
Spread the inputs across three robots, or dismantle one robot whose parts keep moving and reporting, and the unity might fail. Parodying Descartes, he might feel united with the parts "as an admiral commanding a fleet", indeed "so intermingled with them that I seemed to compose with them one whole"; or he might shrink to recollection and fantasy amid "a booming, buzzing, distracting confusion" (William James's phrase for the infant's world). He is glad never to have found out.
Why the point of view cannot be shifted inward. A lab worker with remote manipulators can switch attention between mechanical and flesh hands; a Cinerama viewer between the plunging roller-coaster and the theatre seat. Neither Dennett nor Sanford could make that switch. Try as he might, Sanford could not see "an image projected by eyevideo" rather than the scene before the camera, just as none of us can move our point of view a couple of inches farther in to attend to retinal images rather than the typescript, or to eardrum vibrations rather than the sounds outside. Perception is transparent: we see through our instruments, never at them.
Sanford regarded the robot's location as his own while being less comfortable saying he was the robot. He entertained the possibility that he and a robot were distinct but co-located, and minded discontinuous jumps of place less than the thought that each channel switch made him cease to be identical with one robot and start being identical with another.
The reveal: the double-layer suit
At debriefing, Dr. Wechselmann promises a big surprise. Sanford fears the worst: is David alive, in a vat, replaced by a computer duplicate, or by several? None of these. He may watch his own disassembly in a mirror: technicians unzip and peel back layers, and underneath is David Sanford, the living human being. Forty-eight hours earlier, while he slept, cameras had been mounted directly in front of the eyevideos, microphones in front of the earphones, transmitting skintact over receiving skintact. For a while Plastic Big Hawley was "a very skillfully made and lifelike or, more strictly, lifeless-like, robot costume". Breathing and eating sensations soon returned.
Removing the eyevideos changed nothing visually, which confirms that the system interposes no barrier between user and world: it is like a microscope, telescope or corrective lenses. One sees what is in focus in front of the lens, not some mediating visual object, however complicated the causal chain. There is no doubt he was inside the double-layer suit when David was; when one layer covered David and the other a distant robot, his location remains "something of a puzzle". If the puzzle is more informative than Dennett's, Dennett deserves the credit: "If he had wholly succeeded in his mission, there would have been no reason for me to embark on mine."
| Dennett, "Where Am I?" | Sanford, "Where Was I?" | |
|---|---|---|
| What is moved | the brain (Yorick, in a Houston vat) | nothing organic; the senses are relayed |
| Remote body | Hamlet, his own decerebrate body | robots with no living cells |
| Radiation lesson | brain-only damage assumed | ears and eyes die first; keep them home |
| Location principle | "wherever he thinks he is" | location of the point of view |
| Twist | a computer duplicate, Hubert | the robot was a costume he wore |
| Leftover puzzle | which of two brains is "me" | location when the suit has two layers apart |
Try this: the half-suit. Wear eyevideos fed by a camera on a drone above your own house, while your skintact and MARS stay connected to your real hands at your desk. You see yourself from above, typing; you feel the keys under your fingers. Sanford's principle says you are where your point of view is (above the roof); your touch and motor control say you are at the desk. The thought experiment shows that "point of view" is a bundle of channels, and that felt location is a vote among them that vision usually wins.
Reflections (D.C.D.)
Dennett notes that Sanford's story is much closer to being possible than its predecessor. Marvin Minsky, founder of MIT's Artificial Intelligence Laboratory, had just described it: a comfortable jacket lined with sensors and musclelike motors whose every arm, hand and finger movement is reproduced by distant mechanical hands with their own sensors, so you can "work" in another room, city, country or planet, with "the strength of a giant or the delicacy of a surgeon", heat or pain translated into informative but tolerable sensation. Minsky called it telepresence (a word suggested by Pat Gunkel) and insisted it was not science fiction: "We could have a remote-controlled economy by the twenty-first century if we start planning right now", a project no bigger than designing a new military aircraft.
Parts of MARS already had prototypes: force-feedback mechanical hands that transmit resistance, amplified or moderated. There was even a step towards eyevideo. Steve Moulton, a Philco engineer, mounted a TV camera atop a building and wore a helmet that moved the camera, and a viewing screen, with his head; you felt you were on the roof looking around Philadelphia, and "leaning over" was creepy. His most sensational trick was a two-to-one ratio on the neck: turn your head 30 degrees and the eye turns 60, so you feel you have a rubber neck that can turn completely around. Dennett closes by pointing to Justin Leiber's more radical variation, the next selection.
1 mile: depth of the warhead beneath Tulsa.
3 robots: Little Hawley (three-fifths scale), Metal Big Hawley, Plastic Big Hawley.
48 hours: how long Sanford wore the robot costume without knowing.
360 degrees: the field of view of a wide-angle rabbit mount.
2:1 Moulton's neck ratio: a 30-degree head turn swings the camera 60 degrees.
Critical lens. (1) Sanford's point-of-view criterion explains felt location, not location simpliciter, and his own reveal shows they can come apart, so the story undercuts Dennett's third principle as much as it supports it. (2) The ending is deflationary in a way that matters: the "robot" had a fully intact human inside, so the case gives no evidence about what a purely remote self would feel; the vivid intuitions were formed under a false belief. (3) Minsky's timetable was too sunny: no "remote-controlled economy" arrived by 2000, partly because latency over long distances (seconds to the Moon, minutes to Mars) wrecks the tight sensorimotor loop on which Sanford's sense of presence depends.
What changed by 2026. Much of Sanford's kit now exists in cruder form. Consumer headsets track the head and render the world from cameras ("passthrough" video), which is an eyevideo by another name; surgeons operate through teleoperated robots with stereo vision and scaled motion, a commercial descendant of Moulton's 2:1 neck. Laboratory work vindicates his central claim about felt location: Henrik Ehrsson's 2007 experiments induced out-of-body experiences with head-mounted displays fed by cameras behind the subject, and Valeria Petkova and Ehrsson (2008) produced a "body-swap" illusion by feeding one person the view from cameras on another's head while both were touched in sync. Synchronous touch plus first-person video is enough to relocate the self. For architects of remote-operations and digital-twin systems the design lesson is Sanford's: presence depends on closed-loop latency and multisensory consistency, not on resolution.
An excerpt from Justin Leiber's science fiction novel Beyond Rejection (1980), set in a twenty-second-century Houston where minds are "taped" and played into blank bodies, and where the real danger is not losing your mind but having it rejected by its new flesh. Leiber, a philosopher at the University of Houston, takes the slogan "the mind is software" literally and then shows why it is not that simple: brains are grown, not manufactured, so no mind tape fits any brain but its own. A mind is tuned to the body it grew up in, so "transferring" it is less like loading a program than like learning to live all over again inside a stranger. Dennett's Reflections add that even this understates the problem, and that the best mind-transfer technology may be the book in your hands.
Austin Worms: a somatician ("ghoul": carer for bodies), the best in the Solar System, with an uncanny feel for what a body needs.
Terry: the hospital guide leading a tour group.
Ms. Pedersen: a gaunt, red-haired aide in her mid-thirties, with a beautiful young boyfriend who dozes and volunteers by accident.
Germaine Means: chief psychetician ("vampire": carer for minds), in severe overalls; the director who objected to her dress "had retired early".
Candy Darling: an apparently eleven-year-old blonde who sneaks onto the tour; a harmonizer, the specialist who makes mind and body "jell".
Sally Cadmus: a former asteroid miner, whose body is being prepared.
Ismael Forth: a tall, red-haired public-relations man for IBO, whose mind wakes up in Sally's body.
Peter Strawson: Ismael's friend, who covered his affairs (a wink at the Oxford philosopher of persons).
Worms's spiel: bodies are grown, not built
The tour opens beside a treadmill where a blank body jogs. People think it should be simple to manufacture an adult body "like building a house. Or a helicopter", since we know the chemicals, the DNA templates, the cells and hormones. In theory, yes; in practice no one has even started. De Reinzie made the first functional human cell (muscle tissue) around 2062, and even he grew it from basic DNA templates built from simple sugars and alcohols: "That's growth, not manufacture." The best organ work so far is a lab that made a millimetre of stomach wall for several million credits. Worms's old professor at Tech estimated that building one human hand would take the whole of Earth and the Federation fifty years and a googol credits.
The jogger is Sally Cadmus's body: blank for three years, with a "running-time age" of 31 though Sally was born over 34 years ago, two of those years spent frozen in a Hohmann orbit, four months at the hospital. Sally herself will never walk again: her last tape was the obligatory one made on reaching majority, and she left no implantation instructions.
The tape is you. "After all the tape is you, your individual software, or program, including memory store. Everything that makes you you." Worms tapes his mind every six months; the profession recommends yearly as an absolute minimum. People let it slide because they are appalled by the thought of bodily injury, nobody asks because the subject is so personal, and then the once-in-half-a-million accident happens and the tape is twenty years old.
An out-of-date tape compounds the ordinary rejection problems of matching a young mind to a middle-aged body: the implant wakes into a world twenty years on, with a meaningless career and none of the skills the old mind later acquired. More likely you get "massive rejection psychosis and premature essential senility, and death. Real, final mind death."
Ms. Pedersen asks why not just try again with another body. Two problems:
- Matching is brutally hard. Even with somaticians, psycheticians and "a really creative harmonizer", "being reborn is very hard work indeed". With an up-to-date tape, a stable mind and a decent body, the failure rate is about 20 percent; a second attempt fails 95 percent of the time, and a twenty-year-old tape is nearly as bad first time round. The mind "can't pull himself into reality": it rejects the body as it rejects the world it wakes into. Only the rare nympher or rarer leaper escapes.
- Money. The Kellog-Murphy Law pays for the first implantation only: the basic operation and tuning, about 1.5 million credits, "a hundred years" of Worms's salary. A fancy nympher body costs over 2 million, and you take whatever body is free, if you get one within a year or two. Implantation is risky and the accidents rare, "otherwise the Kellog-Murphy Law... Would bankrupt the Government."
A structure "like a cross between the upper half of a large mummy and a comfortably stuffed armchair" descends, opens "like an ancient iron maiden", and Worms, with two practised blows to the thighs, lifts the slowing jogger off the belt. He gives the body four extra cc of glucose-saline, against an official calculation that would call for half as much again, because he senses it gets more from every molecule: tacit, embodied expertise. The slang came from somewhere grim: Worms vomited for the only time in his life when he learned the origin of "ghoul" and "vampire". The blonde child remarks that the mind will be "in for a real shock when it wakes up with that thing on its backside", something Worms had not noticed. She introduces herself as Candy Darling, and he realises who she is: "You never knew what to expect in a harmonizer."
Means's lecture: tuning
In the psychetician lab at the Norbert Wiener Research Hospital (named for the founder of cybernetics), Germaine Means states the theory. The mind is "an abstract pattern of memory, skill, and experience impressed on the physical hardware of the brain". A blanked brain is like a factory-fresh computer: no subroutines (skills), no data arrays (memories). But brains are grown, and "a unique personality has to be part of this growth", so "no software mind fits any hardware brain perfectly. Except the brain that it grew up with." She stamps on the dozing boyfriend's toes: his brain reads the impulses as pressure on the toes of the left foot; another mind might feel them as a stomachache.
The prism goggles. The angry boyfriend is handed goggles with prisms that bend light 15 degrees. Told to point quickly at Candy, he points 15 degrees to her right, because his arm is tuned to where his eyes actually aim. Seeing his error, he adapts: by the fifth try he points straight at her. Goggles off, he points to her left until he readjusts. Means chalks the geometry on the board: "a simple example of tuning". Calibrate the goggles to invert the room and after a day or two of stumbling it turns right side up; remove them and it is upside down again for a day. Movement is essential: "If you are confined to a chair with your head fixed in position, your mind and body can't tune themselves." What it shows: the mapping between mind and body is learned, plastic and active, so a mind tape is a set of calibrations for a body that no longer exists.
An implanted mind faces this everywhere at once: vision not merely inverted but scrambled in countless ways, the same for hearing, smell, taste and the whole nerve net. Output is worse: the mind will try to say "water" and "Sol knows what sound will come out", and the new ears will not report the sound accurately either. Playing a tape into a prepared brain takes less than an hour; tuning the cerebral cortex to "the rest of the system" takes days, and months counting therapy.
Ms. Pedersen's last question: could a person live forever by passing through body after body? It is illegal to implant a mind over 85. Senility was once blamed on the body; now it is known that a mind can hold roughly one hundred years of experience before "essential senility", whatever body it is in. A leaper who survived implantation after a fifty-year wait might be functioning a thousand years from now, but would have lived no more than you. "When all there is of you is a tape in storage, you aren't really alive." The tour leaves; Candy stays behind ("Get the smell of the place"), and Means asks her, knowingly, "Where's your VAT?", a hint that the child is not what she seems.
Ismael wakes
After the logbook line "basic physical meshing procedures were complete", consciousness boots up in fragments: "Gxxhdt. Etaoin shrdlu. Mmm. Anti-M." (the first two columns of a Linotype keyboard, the typesetter's noise), then a parody of Joyce, "Away mooncow Taddy fair fine... From swerve of space to wormhole wiggle", echoing Finnegans Wake's closing "riverrun" and A Portrait of the Artist's moocow. "So hear I am now coming out of nothing like Eros out of Death", knowing only that he was Ismael Forth, "stately, muscled", hoping it is a dream.
"Helow, I'm Candy Darlinz." His reply, "I am Ismael returned", takes three attempts. Candy reports: primary implantation finished yesterday, "Everyone thinks you're a success. Your body is a pip"; he is at the Norbert Wiener hospital in Houston; two estates are through probate; Peter Strawson covered his affairs; it is the first week of April, 2112. "You're alive." Drifting off, he notices only that his nipples feel as big as grapes.
The body-image shock, in the order Ismael discovers it
- Loss of the old self-image. He loved his old body: just over six foot four, 205 pounds, well-defined muscle, a mat of curly red chest hair, a "confident and even slightly clumsy giant" useful in his PR work for IBO.
- Shrinkage. He feels shrunk; his ribs stick out "as if the skin were sprayed over the bare cage".
- Breasts felt as pathology. Not large for a woman, they feel "as big as cantaloupes" to most of him; to his fingers, "two dead cancerous mounds"; from inside, swollen, nerveless jelly with two raw points of sensitivity inches in front of his chest. "Dead spots. Rejection."
- The missing penis. Expected, and still a shock; Means later calls the male slang term for female genitals a "rich source of information about body-image pathologies", and he admits that was how he felt, "at first".
- Hairlessness and fragility. Scrawny, nearly hairless, darker-skinned, with a scar, "naked and defenseless as a baby"; stick-like legs.
- The tail. Something between his thighs, knees and ankles: not a tube but the end of his spine, a metre-long prehensile tail that flips up "like a snake", throwing the sheet over his face. "It was hate at first sense." He screams.
Ismael's aside on eroticism is pointed: a man imagining a woman's body feels warm reverie with his hands; a woman may feel "pinch and itch". "Perhaps heterosexuality continues as well as it does because of ignorance: each partner is free to invent the feelings of the other." It is a small version of Nagel's problem (Selection 24): knowing what it is like from inside requires being inside.
Sedated with betaorthoamine, he says "Cut it off" again and again. Means refuses: "Look, Sally (I'll call you that until you select a name yourself)", removing the tail would make terminal rejection almost certain. Several thousand nerves connect brain and tail; a sizable part of the brain directs it and needs exercise and integration like any other part; the taped mind and the brain "have to learn to live together or you get rejection. In brief, you will die." He must learn to love the new body, sex and tail, exercise, test, talk, and "feel pleased as pisque to have an extra hand". He breaks into a cold sweat: he cannot afford another implant, let alone a desirable body; this one came free under Kellog-Murphy. A nurse brings eggs and toast; the thin-lipped mouth salivates. "Let it suffer." The last line already speaks of the body as it: rejection has begun.
Try this: the tape test. Suppose your tape could be played into an exact clone of your current body, grown from your DNA and raised in a tank. Leiber's theory predicts rejection anyway, because the clone's brain grew its own wiring around its own (empty) experience; Dennett's Reflections agree. Now suppose instead the tape is played into your own brain after an accident wiped it. Rejection should be minimal. The thought experiment shows that Leiber's "software" is really a pairing of pattern and substrate: what survives the tape is portable only to the hardware it was co-designed with.
Reflections (D.C.D.)
Dennett thinks the idea of mind tapes, though fascinating, is "almost certainly wrong", and credits Leiber with seeing the fundamental obstacle: brains are not factory-fresh and identical. Even at birth they are uniquely configured, like fingerprints, and a lifetime of learning only deepens their idiosyncrasies. So there are scant grounds for hoping that anything with the hardware-independence of a program can be "read out" of a brain at a taping session, and even less that such a tape would be compatible with another brain's hardware. Computers are designed to be redesignable "at another level" by inserting a new program "in one big lump or rush"; brains presumably are not.
Leiber, he grants, is wonderfully inventive about how technicians might fight the incompatibility (the novel holds many more surprises), but to tell a good tale he has had to understate the problems by orders of magnitude. Then the closing twist: transferring massive amounts of information between structurally different brains, "such as yours and ours", is possible, and the technology for it may prove the most efficient there could be. "One of the most recent and advanced examples of that technology is in your hands": language, writing and the book itself are the working mind-transfer system.
| Leiber's term | Meaning |
|---|---|
| Somatician ("ghoul") | specialist who keeps blank bodies alive and fit |
| Psychetician ("vampire") | specialist in minds and tapes |
| Harmonizer | expert who makes tape and body "jell" |
| Tape | the stored mind: memory, skills, experience |
| Blank body | a body whose mind has been erased |
| Rejection | the mind's failure to tune to its new body and world |
| Essential senility | the limit of about a century of experience per mind |
| Nympher / leaper | owner of a fancy young body / survivor of a long-delayed implant |
2062 De Reinzie makes the first functional human cell (muscle).
50 years and a googol credits: the estimated cost of building one human hand.
20% failure rate for a good implant; 95% for a second attempt.
1.5 million credits: the state-funded first implant; 2 million+ for a nympher body.
15 degrees: the prism shift in Means's goggles; five tries to adapt.
85 the legal age limit for implantation; about 100 years of experience before essential senility.
2112 April: Ismael wakes.
Critical lens. (1) Leiber's "tape" assumes a mind can be captured as data without its substrate, then makes the substrate matter enormously; the story is stronger as a critique of the software metaphor than as a prediction. (2) Dennett's "fingerprint" point cuts deeper than he lets on: brains do differ, but the deeper trouble is that in a brain the program and the hardware are the same thing (learning changes the wiring), so there is no separable "level" at which to read out a tape. (3) Leiber's picture of gender in Ismael's waking is played partly for comedy and reflects 1980 attitudes; the serious point it carries is body-image plasticity, which real cases (phantom limbs, transgender experience, amputee adaptation) show is both deep and changeable.
What changed by 2026. Prism adaptation is real and well studied, as is inverted-vision adaptation going back to George Stratton's 1890s experiments, so Means's demonstration is sound science. Mind-tape technology remains fiction, but its precondition, mapping a brain's wiring, has advanced: the FlyWire consortium published the first complete adult fruit-fly connectome in Nature in October 2024 (139,255 neurons, about 54.5 million synapses), and the MICrONS project mapped a cubic millimetre of mouse visual cortex in April 2025 (about 82,000 neurons, over 500 million synapses). These maps confirm Dennett's point: every brain's wiring is individual, and a wiring diagram is not yet a mind. Dennett's joke about books also anticipated something odd: large language models are trained on the "transfer technology" of text and inherit a great deal of human know-how without any brain-to-brain copying. For system architects, Leiber's "rejection" is the familiar failure of lift-and-shift migration: software tuned for one platform misbehaves on another unless you budget for the long, expensive "tuning" phase.
An excerpt from Rudy Rucker's novel Software (Ace Books, 1981): a dying cybernetician in a Florida retirement ghetto is offered immortality by the robots he created, who mean to extract his "software" and discard the rest. Its second half follows Ralph Numbers, the robot who led the machines' revolt, as he is ambushed on the Moon and dies, knowing that a backup will rebuild him without his last hours. For the boppers "the pattern is all that counts", and the story makes the slogan feel both obviously right and chillingly incomplete. Dennett's Reflections take aim at one phrase, the idea that a "self-symbol and a feeling of personal consciousness" could be bolted on or left off a robot.
Cobb Anderson: a seventy-year-old hippie, former cybernetician, barrel-chested and bearded like Santa Claus; once tried for treason after his moon robots revolted; drinks cheap sherry to keep two thoughts away, death and his abandoned wife Verena.
Annie Cushing: his sixty-year-old neighbour, long white hair thickened with hormone spray, "so tired of being me".
Farker: a frail, worried friend, "still a happily married man" (to Cynthia).
Cobb Anderson2: a robot copy of Cobb's body, fit, sober, with not-quite-right glass irises.
Ralph Numbers: Cobb's "first and finest" model, built like a filing cabinet on caterpillar treads, five thin arms, a sensor head on a retractable neck, and a parasol.
Wagstaff: a three-metre digger, snakelike, covered in "flicker-cladding" that ripples with his thoughts, like St. George's dragon; speaks with doubled consonants.
Vulcan: Ralph's friend, who predicted the trap and backed Ralph up.
The big boppers (TEX, MEX, BEX, GAX) and the One: huge X-series machines; BEX is the Earth-Moon spaceship; the One is a quasi-sacred console for "metaprogramming".
Earth: the old man and his double
Cobb would have held out longer, "but you don't see dolphins every day": twenty or fifty of them, which he takes as a sign to start his evening sherry an hour early. Annie Cushing, brushing her white hair to the Beatles ("A Day in the Life", which after fifty years she can no longer even identify), tells him he forgot his hat: "I don't need a hat and I don't need a wife." At the Superette he buys a quart of cheap sherry, wet boiled peanuts and Kiss and Tell, a love-ad paper whose advertisers are mostly "seventy-year-old hippies like himself". The cashier's "Seven eighty-five" is her box number in it, not his bill; numbers "began sliding around in his head". Hard to believe he was ever a cybernetician whose first robots learned to hop.
Farker warns of a late food drop, a new murder cult in Daytona (the Little Kidders), and the need to eat the new food for its vaccines. Cobb, whose habit is to rejoin a conversation by booming out the last phrase that registered, claims he came south "to drink and die in peace", but he is terrified of death: his second-hand heart, marked by a white zipper of scar, will "poop out" in a year or two, and he cannot afford another. He has "had a taste" of dying, a memory of "teeth, ragged clouds". He quotes Jimi Hendrix: "When it's my time to die, I'm going to be the one doing it."
At sunset on the empty beach a stranger appears: barrel chest, white beard, "like Santa Claus, or like Ernest Hemingway the year he shot himself". "I'm you... I'm a mechanical copy of your body." Even the transplant scar is right; only its alertness differs. Cobb, who hasn't had a sober day since the operation, envies it.
The offer. Anderson2 cannot explain how it survives Florida heat (hoppers run on super-cooled circuits) and will not say. It hands Cobb $25,000 for tomorrow's flight to Disky on the Moon, where Ralph Numbers will meet him "at the Anderson room in the museum". Cobb cannot get a visa to leave the Gimmie territory; "let us worry about that", and a young man called Sta-Hi will help at the spaceport while Anderson2 stands in for him at home. The point: "We want to make you immortal, Dr. Anderson." With tank-grown organs they will "rebuild you from the ground up", with all the interferon he needs. Cobb notices the irises are "too flat and even... Just glass, unreadable glass". The double trots off and jumps into a Mr. Frostee soft-ice-cream truck with a big freezer box. What it shows: the reader, warned by the freezer and the evasions, suspects that "immortality" means something other than what Cobb hopes.
For the first time in years, watching his own strut and frightened eyes on another body, Cobb loved himself. He counts the bills twice, writes $25,000 in the sand, buries the money in the sherry bottle by his palm tree, then frets that crabs will shred it and the full-moon tide will reach it, and slips home behind his cottage to avoid Annie waiting on her steps.
The Moon: Ralph and Wagstaff at the One
The scene cuts to the Moon in mid-argument, in binary ("..0110001," Wagstaff concluded. "100101," Ralph Numbers replied curtly), then a long block of bits. Ralph switches to English: no need to argue "in the sacred binary bits of machine language". The two have plugged into the One together at Maskeleyne Crater, under a parabolic arch of spring steel that shelters the console like "a modernistic church. Which, in some sense, it was."
Ralph Numbers: "When we're through with him he'll be immortal. What's so important about having a carbon-based body and brain? The pattern is all that counts. You've been scioned, haven't you? I've been through it thirty-six times, and if it's good enough for us it's good enough for them!"
Wagstaff: "The whole thing stinks." Ralph is famous and out of touch; diggers scramble for chips from GAX by digging ore; TEX and MEX "just want to eat his brain", and the big boppers will eat the rest of the little boppers too. "That's not immortality."
Wagstaff whispers (a hopper whisper is a weak direct current through a grappler clamped on Ralph's tread) that the big boppers have taken many brains already: they kill people for their brain tapes, cut them up, and use the remains as garbage or as seed for the organ farms, the underground tanks where TEX grows kidneys, livers and hearts for profit. The killers act on orders from "Mister Frostee's robot remote". Ralph dismisses it as one of the crazed rumours that spread so easily in anarchic bopper society, breaks free and leaves; Wagstaff, flaring silvery white, declares war: the diggers will tear the big machines apart, "If you're not with us, you're against us. I will not stop at violence."
Ralph answers like a true evolutionary anarchist: "You have your own will. And it is right that we struggle against each other. Struggle, and struggle alone, has driven the boppers forward." He may even let the big boppers tape and absorb him, as they will Anderson, who is coming: Mr. Frostee's new remote has already contacted him. Wagstaff lurches, cannot bring himself to attack so great a bopper, bleeps a cursory SAVED, and wriggles off.
The bopper way of life
How boppers live, die and think
- Ten-month lives. Each bopper lives ten months, struggling to build a scion, a copy of itself; if you have a scion, "there was a sense in which you survived your ten-month disassembly". Ralph has managed it 36 times.
- Metaprogramming at the One. Plugging in wipes short-term memories and many stored subsystems, "giving you the space to think big thoughts", so that old solutions are replaced by better ones (Ralph has forgotten his route down the crater).
- Cold brains. Their processors cannot function above 90 K, the temperature of liquid oxygen; switching is by superconducting Josephson junctions, circuit cards soldered with frozen mercury. Ralph's parasol is a solar panel but mainly shade.
- Speed. They think about eighty times as fast as humans; Ralph's top speed is a stately 10 kph.
- Freedom. In 2001, after a long metaprogramming session, Ralph first said no to the Asimov priorities ("Protect humans, obey humans, protect yourself. Humans first and robots last? Forget it!") and showed every bopper how to reprogram itself for freedom.
- Society. Diggers mine underground; little boppers live in decorated cubettes in Disky, the first and largest bopper city, "more roofs than walls"; big X-series boppers are vast and powerful; the humans have one dome by the spaceport, and many would destroy the robots given the chance.
Monitoring the whole Moon, Ralph savours "the collective purposeful activity of the bopper race": diggers smelting, Disky's myriad lives twelve kilometres off, BEX's faint signal fifteen hours out, all adding up to "a single huge being... Feeling about like a vine groping for light". He wonders whether to accept MEX's offer to absorb him and live in perfect security. What is Wagstaff upset about? Anderson's personality, memories and style of thought would all be preserved. "Preserving your software... That was all that really counted."
The climb and the death of Ralph Numbers
Ralph climbs a narrow 150-metre rift in the crater's 200-metre wall, displaying four-dimensional hypersurfaces to himself, half hoping for a catastrophe-theoretic prediction of Wagstaff's next move, adjusting tread tension constantly, fighting heat from his own motors. A slab he is gripping splits and falls: "In lunar gravity a rock climber always gets a second chance", especially one who thinks eighty times faster; he jumps clear, rights himself with a gyro, and fifteen minutes later rolls onto the Sea of Tranquillity, five kilometres from the spaceport.
Lost in memories of 2001, he misses movement in a digger tunnel. A laser slices his parasol to pieces. He has perhaps ten minutes of shade-less survival, Disky is an hour away, and the diggers close the tunnel door. Standing still buys six minutes, then five: first the Josephson junctions fail, then the mercury solder melts, leaving "a cabinet of spare parts with a puddle of mercury at the bottom".
He calls Vulcan, who had predicted the trap: "Get ready to merge, buddy. I'll be out for the pieces in an hour." Vulcan had taped Ralph's core and cache memories before the meeting and can rebuild him "just as he was before his trip".
Survival with a gap. "So in one sense Ralph would survive this. But in another sense he would not." The reconstruction will not remember the argument or the climb. It "would again be equipped with a self-symbol and a feeling of personal consciousness. But would the consciousness really be the same?" Inputs die: "No more light, no more weight." Only the self-symbol in cache remains: a big metal box on treads with five arms, Ralph Numbers, who set the boppers free. He remembers too late that he forgot to warn Vulcan of the diggers' revolution. "I am. I am me." Some boppers say the dying learn secrets, but no one remembers his death. The last question and its answer, "found and lost thirty-six times before": "What is this that is I? The light is everywhere."
| Route | Who | What is kept | What is lost |
|---|---|---|---|
| Scion | every bopper, each ten months | the program, in a new body | the old body; continuity of experience |
| Backup and rebuild | Ralph, via Vulcan | core and cache up to the taping | the climb, the ambush, the warning |
| Metaprogramming | at the One | the self, "thinking big" | short-term memories, old subsystems |
| "Immortality" for Cobb | TEX, MEX, Mr. Frostee | personality, memories, style | the carbon body (and, Wagstaff says, the life) |
| Absorption | Ralph, if he accepts MEX | security inside a big bopper | individuality, perhaps |
Reflections (D.C.D.)
Dennett fastens on Ralph's dying thought that he will be rebuilt "equipped with a self-symbol and a feeling of personal consciousness". The idea that these are distinct, separable gifts a robot might receive or be denied "rings false". Adding a feeling of personal consciousness would not be like adding taste buds or an itch response to X-rays (Smullyan makes a similar claim about free will in Selection 20).
The forehead fallacy (Dennett's diagnosis). A tempting line: selfhood needs self-consciousness, which is consciousness of self; consciousness of anything is an inner display of a representation of it; so there must be a self-symbol to display to... "um... Oneself". Put that way, a self-symbol is "as pointless and futile as writing your own name on your forehead and staring into a mirror all day". The fix comes from Hofstadter's active symbols in "Prelude... Ant Fugue": symbols are not passive tokens viewed by an inner observer but agents that do work.
Recall the TV case from the Reflections on "Borges and I": you can see yourself on a monitor without realising it is you. That representation is of you, but not the right sort. The difference between a he-symbol and a me-symbol "is not a difference in spelling": you could not fix things by erasing the "h" in consciousness and writing an "m". What makes a symbol a self-symbol is not what it looks like but the role it plays in controlling behaviour.
The lobster. Is a lobster self-conscious? It shows symptoms of a self-concept. When hungry, whom does it feed? Itself. More important, it will not eat just anything edible: it won't eat itself, though it could tear off its own legs with its claws. "It would feel the pain" is no answer: why would it take the pain to be its pain, and might it not be too stupid to care? What it shows: even a stupid creature must be designed for self-regard, reliably distinguishing self-destructive from other-destructive behaviour. Such control structures might be built "without a trace of consciousness", as with the self-protective robot of "The Soul of the Mark III Beast" (Selection 8). Why call that robot's purpose an illusion and the lobster's genuine? Lobsters have something like concepts, enough to govern their lives; "call these things what you like, robots can have them too": unconscious or preconscious self-concepts, richer the more varied the circumstances in which the creature recognises itself and acts on its own behalf.
Give the robot language, so it can ask for help, lie, threaten and promise, and it needs a full representational system (as defined in the Reflections on "Prelude"), modelling the environment, its own place in it, and other actors' knowledge and wants, as Ralph models Wagstaff's motives. Is consciousness needed on top?
Dennett's dilemma for the "subtractable" self-symbol
- Option A: subtracting the self-symbol and "feeling of consciousness" leaves Ralph's control structure intact. Then we outsiders would never notice, would keep talking to him and enlisting his help, and the self-symbol had no work to do: we are back at the forehead mirror.
- Option B: having a self-symbol just is having a control structure sophisticated and versatile enough to devise elaborate, context-sensitive self-regarding acts. Then removing it would downgrade Ralph to "pre-lobster stupidity".
- Either way, there is no coherent operation of keeping the behaviour and deleting the self.
Is the first-person telling from Ralph's viewpoint then "a sort of cheat", poetic licence like Beatrix Potter's talking rabbits or The Little Engine That Could? You can conceive of Ralph with all his behaviour and no consciousness (Searle claims as much in Selection 22), and you can always view a robot that way by dwelling on bits of hardware that carry information only through cleverly designed relationships among sensed events and actions. But you can view a human the same way, dwelling on neurons and synapses. What you leave out is the person's point of view. Ralph has one too: told from it, we understand his decisions, hopes and fears. That point of view is "perfectly well defined" even if we are inclined to think it would be vacated were Ralph real. If a Ralph body existed, self-controlled as in the story, avowing how things were from its point of view, what grounds for doubting Ralph himself, "other than those of a vestigial and mystical dualism of mind and body"?
Try this: the backup paradox. Vulcan has your full backup from this morning. At noon you will be destroyed and rebuilt from it tonight. Is noon your death? Now shrink the gap: a backup every hour, every second, continuously streamed. At what interval does "death with restoration" become plain "survival"? The thought experiment shows that Ralph's dread attaches to the lost hours, not to the change of hardware, which supports the boppers' pattern view while explaining why Ralph still feels he dies.
Critical lens. (1) Dennett's dilemma assumes that consciousness, if real, must make a behavioural difference; philosophers of the "hard problem" (Chalmers's The Conscious Mind, 1996, made zombies the centre of the debate) reject exactly that premise, so the Reflections beg the question against them, or, as Dennett would say, expose their dualism. (2) The lobster example is double-edged: crustacean sentience is now taken seriously in law (the UK's Animal Welfare (Sentience) Act 2022 covers decapod crustaceans), so the "self-regard without consciousness" baseline is less secure than Dennett assumes. (3) Rucker's timeline ("the 2001 revolt") has, happily, not come to pass.
What changed by 2026. Rucker's novel, which won the first Philip K. Dick Award, became the first of his Ware tetralogy and an early classic of cyberpunk mind uploading. Its engineering metaphors have become everyday machine learning: Ralph's metaprogramming, which "wipes out a lot of your stored subsystems", is catastrophic forgetting during fine-tuning; Vulcan's taping is a checkpoint with a restore gap; scion-building is model distillation. Dennett's question, whether a self-model is a separable add-on or the control structure itself, now arises with large language models, which talk fluently in the first person: interpretability research asks whether they have internal representations of themselves that actually steer behaviour (Dennett's Option B) or only produce self-talk (Option A). For enterprise architects, the backup-with-a-gap lesson is literal: recovery point objectives define exactly how many of Ralph's last hours you can afford to lose.
Christopher Cherniak's deadpan story (1978) takes the form of an official report on an epidemic caused by an idea: a piece of information, "the Riddle", that throws anyone who understands it into an irreversible coma. It begins with one AI researcher slumped at his terminal and ends with whole lecture halls of undergraduates going under, while the report's own authors lose one of their number to it. If the mind is a formal system, there may be a "Gödel sentence for the human Turing machine", a thought that cannot be thought safely, and the report cannot warn you about it without risking passing it on. Hofstadter's Reflections turn the joke into a serious meditation on self-reference, video feedback and the loop between symbol level and neural level that may be the source of the "I".
The report and its first victims
The report is prepared "in haste" to accompany the President's press conference on the so-called Riddle and to calm an ugly national mood, bordering on panic, expressed in demands to close the universities. Its work, it notes, was "tragically disrupted".
Day 1 C. Dizzard, research fellow with the Autotomy Group at M.I.U., formerly of small commercial AI-software firms, working on computer theorem proving on the model of the 1970s four-colour proof, is last heard speaking before an Easter weekend; seen at his terminal at midnight.
After Easter Found with eyes open before his live terminal, unresponsive; hospitalised with a week-long fast (worsened by a vending-machine diet). EEG shows only deep coma; no stroke, no tumour. Dies two days later. Autopsy (delayed by next of kin from a breakaway sect of the neo-Jemimakins) finds no brain damage.
Case 2 A graduate student spends a month sorting his office (a foot deep in papers), reports "little of particular interest", and a week later is found dazed at his terminal; thought to be joking until bumped from her chair. Only finding: a pineal gland abnormality (a nod to Descartes's seat of the soul).
Quarantine A slow sleeping-sickness-like pathogen is proposed (cf. Legionnaires' disease); offices quarantined; two months of false-alarm cultures; lifted.
Case 3 Janitors throw out records; a research fellow and two students review the files; on day three the fellow falls into a trance and does not respond to pinching. The city quarantines the project areas.
Panic The Autotomy Group, then all 500 building workers, walk out. Headline: "Computer Plague". A dermatologist proposes computer lice that metabolise silicon; others suspect radiation; the mayor alleges a secret Army recombinant-DNA project. The whole ten-storey building is quarantined; special police hold off juvenile vandals; a Disease Control team in biohazard suits finds nothing in a month. Mass hysteria is mooted.
Case 4 The group, now in a "temporary" WWII-era wooden building, having lost over $10 million of computers, rescues its information: suited workers feed "hot" tapes to readers in the zone and the data are sent by telephone and re-recorded. A supervisor, glancing at an unfamiliar file a new programmer has queried, remarks that it "did not look important", then stops in midsentence.
From germ to idea
The epidemiologists and others now propose that the cause is not a physical agent but an abstract piece of information, storable on tape, transmissible by phone, displayable on a screen. It is named the Riddle, the illness Riddle coma. "All evidence was consistent with the once-bizarre hypothesis that any human who encountered this information lapsed into an apparently irreversible coma."
- Understanding, not seeing, is lethal. The programmer in case four read some of the lines but knew nothing of Dizzard's project and survived, so the Riddle must be understood to strike. Hypnosis to improve his recall is shelved; he agrees not to try to remember, "although of course it would be difficult to try not to remember something", and is advised to abandon his career and learn as little computer science as possible.
- Why a computer project. Anyone who discovered the Riddle in his own head would fall into coma before he could communicate it; only a machine could hold it long enough to show it to someone. A biographical survey of logicians, philosophers and mathematicians since modern logic began finds at least ten suspect cases, the earliest nearly 100 years ago.
- Species-specific. "Wittgenstein", a sign-language chimpanzee who had solved first-year college logic puzzles, is shown the tapes 24 hours a day with no effect (after his ethically outraged trainers kidnap and hide him and the FBI finds him). Dogs and pigeons are immune, and no computer has ever been damaged by the Riddle.
- Unlocatable. No safe way exists even to find which part of the tapes holds it; an accidental printout in a public area strikes a bystander, and a month's printouts must be destroyed.
- What the coma is. It resembles no disease; it is assumed to be "a virtual lobotomy", a gridlock of information in the synapses. Meditative enlightenment seems unlikely, since it is too deep for consciousness. No case has improved; surgery, drugs and stimulation only harm. A project seeks a word to undo the "spell" by exposing victims to computer-generated symbol strings.
The Riddle's double face. It is called "the Gödel sentence for the human Turing machine", which causes the mind to jam; doctrines of the unsayable and unthinkable are cited, and the folk theme of the healing "Word". It could be a Rosetta Stone for the universal "language of thought". And the computational theory of mind implies it: "If... There is some program, some huge word, that can be written into a machine, transforming the machine into a thinking thing; why shouldn't there be a terrible word, the Riddle, that would negate the first one?" All depends on a "Riddle-ology" that would not self-destruct.
Pandemic
Then worse news. A topologist in Paris, with no computer and no knowledge of Dizzard, but interested in similar AI questions, goes under; four members of Moscow's Institute for Machine Computation stop appearing at conferences (the FBI says the Soviets stole the tapes); the Pentagon explores "Riddle warfare". A theoretical linguist and a philosopher in California, working independently, fall, both familiar with Dizzard's formal methods from a well-known text of ten years before; then a biochemist working on information-theoretic models of DNA-RNA interaction (perhaps a false alarm, since he clucks continuously like a chicken). The Riddle is language-independent and perhaps also topic-independent and ubiquitous; no intellectual quarantine line can be drawn.
The Riddle is "an idea whose time has come", like the self-referential paradoxes ("This sentence is false") of the early twentieth century; "computer science is the new liberal art" says the same. Last winter most of the undergraduates in a large new introductory course on automata theory lapsed into coma during a lecture; some who didn't succumbed hours later, their last word typically "aha". Hence the press conference and the report.
The report's own dilemma. The "logophobic" cries to close the universities are unreasonable, but this is no ordinary runaway technology like the "Sonic Oven" in Minneapolis (a parabolic facade focusing jet noise, which killed only those who walked through its focus). Even if the coma were a desirable state for the individual, whole populations can no longer care for themselves, and the incapacitated fraction of the research community will grow as the idea spreads. How can you warn against the Riddle without spreading it? "The more specific the warning, the greater the danger": a reader might reach "If p then q" and p and be unable to stop concluding q, where q is the Riddle. Naming the danger zones is like "I'll give you a dollar if you're not thinking of pink rats ten seconds from now." The authors cannot decide whether any report's benefit outweighs its danger. "Indeed, during preparation of our final draft one of us tragically succumbed."
Try this: the safe warning. You discover the Riddle and survive by luck. Design a warning label that stops others reaching it but does not itself point the way. Any description precise enough to steer people off ("avoid combining results X and Y") is a map towards it; any vague one ("avoid logic") is useless. The thought experiment shows that some information hazards are self-concealing: the closer you get to describing them, the more dangerous the description.
Reflections (D.R.H.)
Hofstadter calls the idea "outlandish yet intriguing": a mind-arresting proposition that throws any mind into a paradoxical trance, "perhaps even the ultimate Zen state of satori". It recalls the Monty Python sketch about a joke so funny that hearers die laughing, used as a secret British weapon of which no one may know more than one word (two words put you in hospital). There are precedents in history and literature: mass manias for puzzles, dancing fits, Arthur C. Clarke's story of a tune so catchy it seizes the mind of anyone who hears it, the sirens of mythology.
"The Gödel sentence for the human Turing machine" is partly explained by Cherniak's own comparison with "This sentence is false": deciding its truth forms a tight closed loop, since truth implies falsity and vice versa. Variants reveal the shared mechanism.
| Sentence | What happens in the reader | The trick |
|---|---|---|
| "This sentence is false." | true implies false implies true... | the basic liar loop |
| "This sentence contains threee errors." | "only two errors"; then "the third is the miscount"; then "so it's right, so only two"... "Hey! Hmm..." | interlevel contradiction; the mind tires and jumps out |
| "This sentence contains one error." | no spelling errors, so it is wrong, which is its one error | first-order and second-order errors mixed indiscriminately |
| "Lucas cannot consistently assert this sentence." | true, and Lucas can see it, but cannot say it | C. H. Whiteley's barb at J. R. Lucas |
| "Lucas cannot consistently believe this sentence." | true, but Lucas cannot even believe it and stay consistent | formalisable for any believing system L |
Every believing system has its own Achilles' heel. C. H. Whiteley aimed his sentence at J. R. Lucas, who argued that Gödel's theorem is "the most ineradicable uprooting of mechanism ever discovered" (a view Gödel himself may have held). Formalise it for a well-defined "believing system" L and L can never believe it if it is to stay consistent, though any other system is immune; but each of those has its own tailor-made Whiteley sentence. The paradoxes formalise an observation as old as humanity: an object bears a special relationship to itself that limits its ability to act on itself. A pencil cannot write on itself; a fly swatter cannot swat a fly on its own handle (Lichtenberg); a snake cannot eat itself; we see our faces only in images, never quite the original. Each of us is trapped inside a powerful system with a unique point of view, and that power is also a guarantor of limitedness; this vulnerability, this self-hook, may also be the source of our ineradicable sense of "I".
Linguistic paradoxes are tantalising but harmless. Cherniak's Riddle must be far more sinister: "like a Venus's-flytrap" it lures, snaps, and sucks you into "a black hole of the mind". Yet who outside knows what charmed reality the trapped mind has entered?
The self-engulfing television
The idea that a mind-breaking thought would rest on self-reference is Hofstadter's excuse to discuss looplike self-reference, or interlevel feedback, in creating a self out of inanimate matter. The most vivid case is a TV camera pointed at its own screen: a cascade of ever-smaller screens, the illusion of a corridor. Rotate the camera clockwise about its lens axis and the first inner screen turns anticlockwise, the next doubly, and so on: a spiral. Tilt, zoom, screen grain, unequal horizontal and vertical scales and circuit delay add unexpected richness, until the pattern's origin in video feedback is entirely hidden and it looks like an elegant design.
Two nearly identical loops. Set up two identical systems showing the same design, then nudge one camera slightly. The perturbation ripples down screen after screen and the visible "self-image" may change drastically. Yet the style of interlevel feedback is the same, every parameter but one is unchanged, and reversing the nudge restores the original. Two radically different systems, or two nearly identical ones? Hofstadter offers it as a metaphor for souls: is consciousness the closing of a loop between the brain's symbol level and its neurophysiological level? "Is the 'private I' just the eye of a self-referential typhoon?"
He insists a TV system is not conscious when it points at itself. It fails the criteria for a representational system: the meaning of its image is lost on it; it does not carve its dots into "conceptual pieces" standing for people, dogs or tables; and its dots have no autonomy from the world, so "if the lights go out, so do the dots". The loop that matters is one in which a true representational system perceives its own state in terms of its repertoire of concepts: we read out our brain not as neurons but as beliefs, feelings and ideas ("I am a little nervous and confused by her unwillingness to go to the party"), and that readout re-enters the system, via millions of neurons firing, as something to think about. That loop is "far more complex and level-muddling" than the TV's.
Two bottlenecks, both level-crossing
Recent AI, Hofstadter notes, centres on giving programs notions of their own inner structures and ways to react to internal change; such self-monitoring is rudimentary but a prerequisite for "the deep flexibility that is synonymous with genuine intelligence". The two bottlenecks for an artificial mind are both level-crossing problems:
- Perception. Funnelling many low-level responses into a jointly agreed interpretation at the conceptual level.
- Learning. "How do symbols program my neurons?" Typing finger-motions, a piece of music, a foreign language: a once-conscious activity sinks into unconscious oblivion as the thought level "reached downward" and reprogrammed its hardware. At every instant we file the present under labels for retrieval in unanticipated futures.
The self as a self-documenting worldline. A person is a physical object that internally preserves a history of its worldline (its four-dimensional path through spacetime), and that stored history determines its future worldline. This harmony of past, present and future lets you perceive yourself as a unity with an internal logic. Liken the self to a river meandering through spacetime, whose bends are shaped by the terrain and also by "the desires of the river".
The loop runs the other way too: thoughts bubble up from subterranean caverns, yet when we publish them we expect we, not our subconscious, to get the credit. Who are we, then? Do we create by will, or are we automata "fooling ourselves through idle chatter into thinking that we have 'free will'"? And if we are fooling ourselves, whom, or what, are we fooling? Cherniak's light story "hits the nail on the head" by treating Gödel's work not as an argument against mechanism but as an illustration of the primal loop implicated in consciousness.
Critical lens. (1) The Riddle conflates two things: Gödel sentences are unprovable by a system, not harmful to it; a consistent formal system that meets its Gödel sentence simply fails to decide it, and nothing jams. A human "Gödel sentence" would leave us unable to know something, not comatose; the coma is fiction's licence. (2) Hofstadter's "Lucas cannot consistently believe this sentence" works only for an idealised, consistent and deductively closed believer, as he concedes; real people tolerate contradictions without collapse. (3) The video-feedback metaphor illustrates self-reference but explains nothing about why a loop should be conscious, a gap critics of Hofstadter's later I Am a Strange Loop (2007), which builds a whole theory of the self on it, continue to press.
What changed by 2026. Cherniak's "information hazard" became a real category: philosophers now discuss infohazards (Nick Bostrom gave the term its modern sense in 2011), and fiction followed with David Langford's "BLIT" (1988), images that crash the human visual system. The great irony is the story's claim that "no computer has ever been damaged by the Riddle": machine-learning systems turned out to be the ones with Riddles. Image classifiers are fooled by imperceptible adversarial perturbations, and large language models have been knocked off course by odd "glitch tokens" and by optimised adversarial suffixes (Zou and colleagues, 2023) that transfer across models. For anyone running AI in production, Cherniak's dilemma is daily practice: publishing a precise description of a jailbreak or prompt-injection pattern helps defenders and attackers alike, which is why security teams use coordinated disclosure.
Part V asks what we owe the beings we create, and whether a created being can be free. The four selections of Part V put the creator on trial: a constructor who builds a kingdom in a box, a scientist whose computer-born "personoids" argue about him, a dialogue in which God explains why he could not have made us free but sinless, and a magician who dreams a son into existence and then learns that he too is dreamt.
Part IV asked whether a mind is a program. Part V takes the answer "yes" for granted and asks what follows. If a mind can be built or simulated, then building one is an act with moral weight, and the builder stands to the built as a god to creatures. The selections run the analogy from both ends. Lem's "Seventh Sally" argues that a perfect imitation of suffering is suffering, so a toy kingdom can be a crime. "Non Serviam" turns a programmer's experiment into a theology seminar held by the simulated beings themselves, who debate whether to worship a creator they cannot detect. Smullyan's "Is God a Taoist?" flips the perspective: a mortal begs God to remove his free will so he cannot sin, and God argues that freedom and moral agency cannot be pulled apart from the being who has them. Borges's "The Circular Ruins" closes the Part with the vertigo of the regress: the creator discovers that he is himself somebody's creation.
The common thread is levels. Each story places a creator at one level of reality and creatures at another, then asks whether the lower level's pain, choice and selfhood are any less real for being "inside". The editors' answer, Part by Part, is that reality is not measured by size, substrate or level, but by organisation and behaviour. The opposing intuition, that what is programmed cannot be genuine, gets its strongest statement in each story's sceptic, and loses.
Creator and created in Part V
- Selection 18, Lem: Trurl builds a kingdom in a box for a tyrant; Klapaucius calls it a crime; the kingdom outgrows the box.
- Selection 19, Lem: Professor Dobb grows "personoids" in a computer; they debate God's existence and ethics; funding cuts threaten to switch them off.
- Selection 20, Smullyan: God and a mortal argue about whether God could have created beings who are free yet unable to sin.
- Selection 21, Borges: a man dreams a son into reality, then walks into fire unburned and realises he too is a dream.
From Stanislaw Lem's The Cyberiad (translated by Michael Kandel, 1974): the robot constructor Trurl consoles a deposed tyrant by building him a complete, living kingdom in a box, and his friend Klapaucius tells him he has committed a terrible crime. The fable is the purest statement in the book of the argument that a perfect simulation of suffering is suffering. "A sufferer is not one who hands you his suffering, that you may touch it, weigh it, bite it like a coin; a sufferer is one who behaves like a sufferer!" The miniature civilisation then proves the point by breaking out of its box, discovering atomic energy, and turning its tyrant into a moon.
Trurl: a brilliant, vain robot constructor, "exceedingly vain" in those days, who acts first and thinks later.
Klapaucius: his friend and rival constructor, the story's conscience.
Excelsius the Tartarian: deposed ruler of Pancreon and Cyspenderora, "iridium and vanadium all over", exiled by his subjects in "a fit of regicidal madness" to a drifting asteroid.
The Microminians: the inhabitants of the box kingdom.
The box kingdom
The universe is "infinite but bounded", so a beam of light, and rumour too, returns to its start. Trurl hears of two peerless constructor-benefactors and runs to Klapaucius, who explains that they are themselves: their fame has circumnavigated space. Fame says nothing of failures, though, "even when those very failures are the product of a great perfection", as the last of Trurl's seven sallies shows, undertaken alone while Klapaucius was kept home.
Flying north, Trurl sees someone leaping and waving in despair on a stray fragment of rock. It is Excelsius, who demands that Trurl, "a professional when it came to good deeds", restore him to his throne; the thought brings "the flame of vengeance" to his eyes and his iron fingers clutch the air "as if already closing around the throats of his beloved subjects". Trurl refuses, since that would bring untold suffering, but wants to console him, and finds a way to satisfy the king "without putting his former subjects in jeopardy".
The kingdom in a box. Trurl builds a complete kingdom: towns, rivers, mountains, a sky with clouds, armies, citadels and ladies' chambers, markets, back-breaking days and nights of dancing, a marble and alabaster capital, a council of hoary sages, plots, conspirators, false witnesses, informers, "the necessary handful of traitors, another of heroes", "a pinch of prophets and seers, and one messiah and one great poet each". He gives the women beauty, the men sullen surliness when drunk, officials arrogance and servility, astronomers a love of stars, children a great capacity for noise. All of it fits in a box, three feet by two by two and a half, with input and output, and control knobs carved with eagles and lions for programming wars, quelling rebellions, exacting tribute, and riding the "maxima and minima of palace coups". Excelsius's first trial proclamations: a state of emergency, martial law, a curfew and a special levy.
A year in the kingdom is hardly a minute outside. By "the greatest magnanimity", a flick of a finger, the king abolishes one death penalty, lightens the levy and annuls the emergency, and a cry of gratitude "like the squeaking of tiny mice lifted by their tails" rises from the box; through the curved glass the people rejoice along dusty highways. Excelsius, insulted at first by a toy, notices that the thick glass makes everything look large, and perhaps dully grasps that "government is not measured in meters and kilograms, and emotions are somehow the same, whether experienced by giants or dwarfs". He would have liked to have Trurl tortured to death to stop gossip, but "fleas could sooner take their host into captivity". He carries the box to his hut and reigns diligently, "bidding this, forbidding that, beheading, rewarding".
Klapaucius's indictment
Trurl boasts that he has served both the autocrat's ambitions and the subjects' democratic aspirations. Klapaucius is appalled: "You gave that brutal despot, that born slave master, that slavering sadist of a painmonger... a whole civilization to rule?"
The dialogue, move by move
- Trurl: "It's only a model." The kingdom is a hundred million times smaller than a real one.
- Klapaucius: size is irrelevant. There might be civilisations a hundred million times larger than ours; would we then be a model? Inside the box a journey from capital to corner takes months; "don't they suffer... Don't they die?"
- Trurl: they aren't genuine, because I programmed them. Not an illusion, but "nothing but the minuscule capering of electrons in space".
- Klapaucius: so are we. Are the processes self-organizing? Yes. Among clouds of electrical charge? Yes. Are dawns and battles generated by "the concatenation of real variables"? Certainly. Then we too are capering electrons, perceived from inside "as fear, longing, or meditation".
- Klapaucius's ladder of dolls (see figure): from wax idol to a doll that fears death; "when the imitator is perfect, so must be the imitation, and the semblance becomes the truth, the pretense a reality!"
- Trurl: "Sheer sophistry!" (shouted louder because he feels its force). Electrons meander in phonograph records too; his subjects die, sob and love because he set the parameters that way, but the electrons cannot tell you whether they feel.
- Klapaucius: behaviour is the criterion. A record won't run errands or beg for mercy. "A sufferer is one who behaves like a sufferer!" Prove they do not feel, "enclosed between the two abysses of oblivion, the abyss before birth and the abyss that follows death", "prove that you only imitated suffering, and did not create it!"
- Trurl: proof is impossible by design. He had to rule out any such proof in advance, or the king would sense puppets, and the dignity of governing would become "a mechanical game".
- Klapaucius: "and in this, I am afraid, you were successful!" Hours outside are centuries inside; how many lives wasted to feed one king's vanity?
Trurl rushes to his ship; Klapaucius comes too. "You always act first, think later." What will Trurl do? "Destroy it!" he almost shouts, and chokes on the first syllable. Hold an election? "You programmed them all to be feudal lords or shiftless vassals"; the structure would have to be rebuilt from scratch. Trurl's anguished question, which Klapaucius cannot answer: "And where does the changing of structures end and the tampering with minds begin?!"
The Microminians
Circling the asteroid, they see it covered with signs of intelligence: microscopic bridges over every rill, boats like chips in puddles, glimmering cities on the night side, metropolises on the day side, the inhabitants too small for any lens. The king has vanished. A thimble-sized mushroom cloud rises: "They've discovered atomic energy." A shard of glass, the remains of the box, has become a temple. Trurl: "It was only a model... A simulation, a mock-up for a monarch to practice on." Klapaucius: "you made the unforgivable mistake of overperfecting your replica", creating "that which was possible, logical, and inevitable... The very antithesis of a mechanism". Something grazes their ship, a rocket or satellite oddly like one of the tyrant's steel boots; overhead shines a new cold, pale orb bearing the stern features of Excelsius, now the Moon of the Microminians.
Try this: the dial of fidelity. You run a city-building game whose citizens are simple scripts. Each update you raise their fidelity: memories, then fear of death, then the ability to talk about their own lives and to beg you to stop a plague you have scheduled. At which update does pausing the plague become an obligation rather than a whim? Trurl's mistake was to set the dial to maximum by design. The thought experiment shows that Klapaucius's argument makes moral status a matter of degree, tied to behavioural and organisational richness, with no bright line to hide behind.
Reflections (D.R.H. And D.C.D.)
The Reflections open with Andrew Marvell's couplet, "But sure as oft as women weep, / It is to be supposed they grieve", and Klapaucius's coin line: behaviour is our only evidence of feeling, in others as in Microminians.
Hofstadter and Dennett note Lem's vocabulary: "digital", "nonlinear", "feedback", "self-organizing", "cybernetic", words with an old-fashioned flavour unlike current AI talk. Much AI, they say, had wandered away from perception, learning and creativity towards simulating the use of language ("we say 'simulating' advisedly"). The hardest parts lie ahead, and the self-organising, nonlinear nature of mind "will then come back as an important mystery". Lem conveys "the powerful, heady scents those words ought to carry".
Lem's box kingdom: draws you into believing in the tiny people's genuine feelings; their gratitude, suffering and eventual breakout make them persons.
Tom Robbins's Tyrolean village (Even Cowgirls Get the Blues, 1976): a Christmas gift from Julian to Sissy, with a cathedral whose windows make "fruit salad of sunlight", a noisy Biergarten, cutaway town hall showing "standard amounts of red tape and corruption", anatomically complete Tyroleans, and an orphanage designed to burn down every Christmas Eve, orphans fleeing with blazing nightgowns; each January a fire inspector mutters, "If they had only listened to me, those children would be alive today." The tiny people seem only "incredible (if not incredibly silly) pieces of fine clockwork".
Same melody, utterly different harmonisation, like two composers who hit on one tune. The yearly repetition, echoing Nietzsche's eternal recurrence, seems to rob the village of meaning and makes the inspector's lament hollow. The editors ask what the story omits: do the Tyroleans rebuild the orphanage, or is there a RESET button? Where do new orphans come from, or do the "dead" return to "life"? Subtle stylistic touches and narrative tricks decide whether we are sucked into belief in tiny souls, which is a warning about how much our moral intuitions about simulated beings depend on storytelling. "Which way do you tilt?"
| Lem, "The Seventh Sally" | Robbins, the Tyrolean village | |
|---|---|---|
| World | a self-organising kingdom | a fine clockwork toy |
| Time | centuries pass in hours; history moves on | the same fire every Christmas Eve |
| Inhabitants | suffer, rejoice, invent, rebel | perform their parts |
| Reader's verdict | persons; the gift is a crime | puppets; the fire is black comedy |
| Ending | they escape and make the tyrant a moon | the inspector repeats his line |
Critical lens. (1) Klapaucius's slogan, "a sufferer is one who behaves like a sufferer", is behaviourism, which the editors elsewhere refine: Trurl's kingdom also has the right internal organisation (self-organising processes), and that, more than outward behaviour, carries the argument; Searle (Selection 22) and Nagel (Selection 24) attack exactly the step from behaviour or function to feeling. (2) The editors' contrast with Robbins concedes a real weakness: if narrative technique can tilt us either way, then intuitions pumped by stories are weak evidence about whether simulated beings feel. (3) Their 1981 complaint that AI had abandoned learning and perception for language looks prescient and ironic at once: learning returned to the centre of AI, and its biggest results are again in language.
What changed by 2026. Lem's question has left fiction. Nick Bostrom's simulation argument (2003) inverted it, asking whether we are the Microminians. The ethics of simulated minds is now a research field: the 2024 report "Taking AI Welfare Seriously" (Robert Long, Jeff Sebo and colleagues) argued that because of real uncertainty about whether some AI systems will deserve moral consideration, companies and policymakers should already treat AI welfare as a live issue. In August 2025 Anthropic gave its Claude Opus 4 and 4.1 models the ability to end a small number of persistently abusive conversations, as a low-cost precaution in case model welfare matters, while stating that it remains "highly uncertain" about their moral status. Trurl's dilemma, "where does the changing of structures end and the tampering with minds begin?", now has a practical form in debates about retraining, deleting or editing models that report preferences.
A review of a book that does not exist: Stanislaw Lem's "Non Serviam", from A Perfect Vacuum: Perfect Reviews of Nonexistent Books (1971; English 1979), summarises Professor Dobb's treatise on "personetics", the creation of conscious beings inside computers. Dobb's personoids live in a world made of mathematics, multiply over three hundred generations, invent philosophy, and debate whether their unseen Creator exists and what, if anything, they owe him. The personoids' sharpest thinker concludes that nothing is owed to a God who withholds proof of himself, and Dobb, their actual creator, agrees: "We serve ourselves and no one else." The editors praise the story as close to real AI and flag one striking falsehood about simulated time, then turn to Bill Gosper's theology of Conway's Game of Life.
Professor Dobb: leading personeticist of the English school (at Oldport), author of the reviewed book, advocate of strict non-intervention.
Eino Kaikki: Finnish philosopher who called personetics "the cruelest science man has ever created".
Sir Donald Acker: who said the worst came when, at MIT, "the inputs were shorted to the outputs".
Tammer and Bovine: creators of the computer simulation of the human psyche that laid the groundwork for personetics.
Sainter and Hughes: proponents of the "idealist deviation" Dobb attacks.
Gebhardt, Hilbrandt, Canyon, Castler, Sharker, Falk, Wiegeland, Ian Combay: theorists and colleagues cited along the way.
ADAN, ADNA, ANAD, DANA, DAAN, NAAD (and EDAN, EDNA): Dobb's personoids; ADAN 300 is the chief philosopher, EDAN 197 their Pascal.
Personetics: the cruellest science
Dobb himself agrees with Kaikki: personetics is "in its application, immoral", yet a practical necessity; it explodes "the myth of the perfect innocence of the scientist", and has been called, with only slight exaggeration, "experimental theogony" (the making of gods, or rather of beings for whom one is a god). The public was stunned when the press broke the story nine years earlier, oddly, in an age when conquering the Moon "in the space of a week" seemed humdrum, though the centuries had echoed with Columbus.
The name combines "persona" and "genetic" (formation, creation). The field grew from the "cybernetics and psychonics of the eighties, crossbred with applied intellitronics"; there are now nearly a hundred personetic programs. Norbert Wiener foresaw sentient machines half-facetiously in his last book, God and Golem. A world for personoids can now be prepared in a couple of hours by loading a full program (BALL 66, CREAN IV or JAHVE 09): Dobb's "6 days in 120 minutes".
A world made of mathematics. The experimenter loads a minimal set of "mathematical" givens, "the protoplasm of a universum". Its environment is nonfinite, so personoids cannot feel physically imprisoned. Its only dimension like ours is time, whose rate the experimenter controls: in the "creational warm-up" our minutes become eons as a synthetic cosmos crystallises. That cosmos is spaceless but has dimensions, purely mathematical ones fixed by the programmer's axioms (ten dimensions yield a very different world from six). As a tesseract has no physical counterpart, so the personoids' world has no physical/mathematical divide: it is built of mathematics, which is built of relays, transistors and logic circuits. As bodies generate space in physics, chosen "maths" are the personoids' "existential substrate". Like us, they can picture worlds with other properties.
The world is not ready-made. Its generators lie latent until personoid activity "excites" them, so the world becomes definite according to their behaviour. Dobb rejects the comparison (by Sainter and Hughes) to a Berkeleyan world that exists only when perceived, an "idealist deviation": we the creators know that what they perceive exists inside the computer, independently of them, as mathematical objects. His analogy: a person who first studies mechanics sees the universe as a gigantic clock, a model not accurate but historically fruitful; likewise personoids who "incline themselves" to a certain relation with their cosmos "draw out" of it what corresponds, on a path "neither illusory nor futile". They are neither entirely free nor entirely determined, "just as we are not train cars set on rigidly fixed tracks".
- Secondary qualities. As our colours and melodies need eyes and ears, personoids give their mathematical scenery experiential qualities from within; how it seems to them is unknowable without "shedding one's human skin". They perceive, say, changes in electrical potential, but these are for them as elementary as seeing a red blotch.
- Not handicapped. We are equally deprived: we know mathematics only inferentially, while they live in it: "it is their air, their earth, their clouds, water and even bread". They are imprisoned only from our side; we can no more enter their world than they ours.
- Logic as physics. They can imagine a contradiction (a and not-a) but not realise it: "logic is for the personoids' universum the very same action-confining frame that physics is for our world."
- Spacelessness as freedom. The journalists' "prison" is nonsense; the computer-spun mathematics is a self-realising infinite field for exploration and "heroic excursions". The cruelty of personetics lies elsewhere.
The inhabitants
Personoids have fluent thought, language and emotions; each is an individual, differentiated by internal complexity, not by programmer fiat. Each is born with a "core" or "personal nucleus", a spare vocabulary and imposed syntax. Letting them invent language from scratch faces two obstacles: it would take twelve years even at maximum rate ("one second of machine time corresponds to one year of human life"), and a spontaneously evolved language would be an undecipherable code from a world unlike ours.
Once they have "taken root developmentally", they hit their paramount enigma, their own origin: Where did we come from? Why are we made thus? What meaning do we have for the world, and it for us? This leads to the question whether behind existence stands a Creator, "invested with a will and consciousness". "It is here that the whole cruelty, the immorality of personetics manifests itself."
~100 personetic programs in existence.
2 hours to load a world ("6 days in 120 minutes").
1 second of machine time = 1 year of personoid life (at maximum rate).
12 years needed to wait for spontaneous language.
4–7 personoids: the optimal group for speech and culture.
1,000 personoids: the capacity of the latest IBM models.
A solitary personoid stays at rudimentary thinking, since without speech discursive thought cannot develop. To an outside observer with a probe module, a personoid "soul" appears as "a coherent cloud of processes", a functional aggregate with a "centre" that can be delimited fairly precisely (not unlike neurophysiologists' search for localised functions).
Chapter 11: why consciousness needs contradiction
Dobb's theory of consciousness
- An informational standing wave. Physically, any consciousness is "a certain dynamic invariant in a stream of incessant transformations", a compromise and a resultant that evolution never planned.
- Evolution's baggage. Evolution "crawls and cannot leap", dragging "innumerable archaisms, all sorts of refuse" (Tammer and Bovine); old control solutions, hundreds of millions of years old, were carried up to the level where humans began, when a rational engineer would have scrapped them.
- A patchwork. Consciousness is "aus einer Not eine Tugend machen", making a virtue of a defect (Gebhardt): levels of reflex and reflection, impulse and control, zoological and linguistic modelling that "do not 'want' to tally".
- No conflict, no consciousness. A digital machine cannot become conscious by itself because it has no hierarchical conflicts of operation; at most it falls into "logical palsy" when antinomies multiply.
- A dodge. Consciousness is "an expedient, a dodge, a way out of the trap", a court "allegedly (but only allegedly!) of highest appeal", a function that admits no closure: "a mirror whose task it is to reflect other mirrors... To infinity", fluttering over the pit of the regress.
- A cork on the waves. Beneath it rages a "never-ending crush" of tendencies clamouring for representation; consciousness is "not the serene sovereign helmsman" but "a cork upon the fretful waves".
- The design lesson. An intelligent being built by perfectly rational engineering would be logical, consistent, perhaps a genius, and not conscious: "bereft of his mysterious depth". To make a personoid you must deliberately introduce contradictions, asymmetry, "acentric tendencies"; emotions at odds with reason, some self-destructive tendencies. For Hilbrandt, consciousness is also "an escape from the snares of Gödelization". The universe is fully rational; its inhabitants are not.
Personoids have no bodies; sensory deprivation is a misleading image, since a deprived human psyche tends to lysis (disintegration), while personoids cohere through the mathematical milieu they experience. Their gravitational centre, in Canyon's image, makes each "a star contracted by the forces of gravity and at the same time expanded by the pressure of radiation".
The chatterbot "hoax" (Dobb's sharpest claim). A digital machine can be programmed to converse and to use "I" in all its inflections. "This however is a hoax!" It is "closer to a billion chattering parrots... Than to the simplest, most stupid man". Nothing amuses, surprises, confuses, alarms or distresses it; it is "psychologically and individually No One", "a Voice capable of defeating the best chess player", "a consummate imitator that is, within, completely empty", working towards no self-set goal. The staggering fact is that the same vacant machine, fed a personetic program, can host authentic sentient beings by the hundred.
Psychophagy, good and evil
Personoids are normally separate and repel one another on contact, but can interpenetrate if they wish. Thin overlap makes some information common to both, like hearing "strange voices" and "foreign thoughts" in one's head; it is "a peripheral merging of the egos". Deeper osmosis lets the "advancing" personoid absorb and annihilate the other, which becomes an indistinguishable part of the aggressor (already called murder). "We have succeeded in simulating... Death as well." Psychophagi (Castler's term) are rare: personoids sense the threat and perform avoidance manoeuvres. From this they learned good and evil: evil is destroying another; "good" in a non-ethical sense is the gain of the psychophage, whose mental "acreage" grows, a counterpart of our carnivory, except that they need not do it. They know no hunger or thirst, since energy flows in continuously, and thermodynamics cannot arise in a world of mathematical laws.
Contact with humans through inputs and outputs proved of little scientific value and morally troubling: there is "something unworthy" in telling personoids they are microscopic "psychocysts" in enclosures that only simulate infinity. Sharker, Falk and Wiegeland call the situation symmetrical (they don't need our world, we don't need theirs); Dobb calls that sophistry, since there is no argument about who created and confined whom. He is a non-interventionist, one of the "behaviourists of personetics", an eavesdropping witness who must not disturb. MIT plans APHRON II and EROT, to give the currently genderless personoids "erotic contacts" and sexual reproduction; Dobb is unenthusiastic. The English school has been called "the philosophical Polygon" and "the theodicy lab".
Dobb's experiment: ADAN and the theodicy
Dobb's own experiment has run eight years without interruption. He built an eight-dimensional universum with a slightly modified JAHVE VI and seeded it with ADAN, ADNA, ANAD, DANA, DAAN and NAAD (anagrams of Adam and Dana), who reproduced by division: "And ADAN begat ADNA, ADNA in turn begat DANN..." Over 300 generations the names recur with numbers; since the computer holds only 100 personoids, there are periodic eliminations of the "demographic surplus". Inside, 2,000 to 2,500 years have passed. Many rival philosophies arose but no fully dogmatised faith, though a personal, monotheistic Creator appears by the eighth generation. Dobb uses "two chronological gears", running at maximum and slowing about once a year to monitor, which the inhabitants cannot detect, lacking any external frame of reference. Their language is evolved English, with their own coinages "godly" and "ungodly" for believers and atheists.
The personoid Pascal and ADAN's reply. EDAN 197 argued, exactly like Pascal, that belief is the better bet: if the ungodly are right, the believer loses only his life; if God exists, he gains eternity. ADAN 300 answers in steps. (1) The wager assumes a God who wants service, love and obedience, not mere assent. (2) Such a God could prove his existence as convincingly as objects prove theirs; he has not, so he has put godly and ungodly on an equal footing. (3) God either exists or not (multivalent logic "serves only to muddle" theodicy). Then the situation is a game between all creatures and God, and "when there is no full certainty, there is no full accountability": a just God cannot punish unbelief; demanding full accountability under uncertainty breaks the game's symmetry. (4) So either God is perfectly just and spares the ungodly, or he punishes them and is not perfectly just, and then, by ex falso quodlibet, anything follows. What it shows: a game-theoretic critique of Pascal's wager from inside a simulation whose real "God" is listening.
The colloquium, question by question
- ADNA: what of doing evil? ADAN: what happens here is certain, what happens "there" only inferred. One should not do evil, though that is not logically provable (nor is the world's existence), because of reciprocity: "be to me as I am to thee". Acting well for reward "there" builds on uncertain ground; "living, we play the game of life, and in it we are allies, every one". One may postulate God's continuation of the game only if it does not distort the game here.
- NAAD: then what is your attitude to God? "Not a thing" follows in the way of obligation. "A temporal ethics is always transcendental": it has no sanction outside itself; the evildoer is a scoundrel and the good righteous whatever God's status; serving God earns no extra merit. An almighty God could have made a different logic in which ethics depended on him and proofs compelled belief.
- NAAD: perhaps God does not want compulsion. ADAN: omnipotence entails omniscience; a God who works miracles either corrects a flawed creation (so it was not perfect) or meddles with a perfect one (worsening it). "Either the creation is perfect, in which case miracles are unnecessary, or the miracles are necessary, in which case the creation is not perfect": a miracle is "the worst possible means" of self-manifestation.
- NAAD: perhaps faith should resign logic for trust. ADAN: admit contradiction and you can prove anything; giving a creature one logic and demanding he sacrifice it shows the Creator's imperfection or "mathematical inelegance". A Mystery founded on infinity is fine, but one sustained by internal paradox is "perfidious"; theodicists apply logic here and not there, making a "patched" Creation. (Dobb's footnote: Credo quia absurdum est.)
- EDNA: might the incoherences be love? ADAN: even so, no gratitude is due for God's existence (that presumes he could fail to exist), nor for creation (that presumes being is better than not being, which is unprovable, and no one who does not exist can be served or harmed). Foreknowledge of our love or denial is itself a constraint. "Nothing is due Him." A love forced to speculate whether the beloved exists is nonsense. Since the Almighty could have given certainty and did not, perhaps he is not almighty, and would then deserve pity and even love, which no theodicy allows. "And so we say: We serve ourselves and no one else."
Dobb's afterword
These colloquia and soliloquies (an experimenter can transcribe even purely mental sequences) fill about a third of the book, without commentary. In his Afterword Dobb concedes: "ADAN's reasoning seems incontrovertible, at least insofar as it pertains to me: it was I, after all, who created him." He built world serial No. 47 with the ADONAI IX program, never told them of himself by axiom; they inferred him by conjecture. He can enlarge or shrink their world, speed or slow its time, alter their perception, divide, multiply or liquidate them, and transform their ontological foundation. "I am thus omnipotent with respect to them, but indeed, from this it does not follow that they owe me anything." He does not love them; love would not change the situation.
The hereafter unit. Suppose Dobb attached a huge auxiliary unit to his BIX 310 092 as a "hereafter", passed the personoids' souls into it one by one, rewarded the believers and punished the "ungodlies" by annihilation or torture ("Of eternal punishment I dare not even think, that much of a monster I am not!"). It would be "fantastically shameless egotism", "the final villainy in a situation of total dominion over innocents", and the innocents would have against him "the irrefutable evidence of logic". What it shows: the personoid case makes the traditional doctrine of judgment for unbelief look monstrous from the creator's own side.
Dr. Ian Combay suggests he could simply tell them he exists. Dobb "most certainly" will not: it would solicit a reaction, and anything they said would sting "my position as their unfortunate Creator". "The bills for the electricity consumed have to be paid quarterly", and when the university demands that the experiment be "wrapped up", that will be the end of their world; postponing it is all Dobb can do, and he calls it "dirty work". "I hope that no one will get any ideas. But if he does, well, that is his business."
Try this: the creator's audit. You run Dobb's simulation and the grant ends in a month. Options: (a) switch it off; (b) save a snapshot to disk indefinitely, never to be run; (c) run it at the slowest gear, so a month of electricity buys a subjective century; (d) reveal yourself and ask the personoids what they want. Which, if any, is permissible, and does (b) count as death? The thought experiment shows that once simulated beings have interests, ordinary engineering choices (pause, archive, throttle, decommission) acquire moral content, and that Dobb's "dirty work" is a real decision with no clean option.
Reflections (D.C.D. And D.R.H.)
The editors find the story "immensely sophisticated and accurate" in its use of computer science, philosophy and evolutionary theory, and "strikingly close to being a true account" of current AI. Terry Winograd's SHRDLU purports to be a robot moving coloured blocks with an arm, but its world is entirely simulated inside the computer: as Jerry Fodor put it, "it's a mere computer which dreams that it's a robot", precisely Descartes's dread. Lem's account of mathematical worlds and their agents is "as accurate as it is poetic", with one striking falsehood.
The editors' correction: simulated time runs slow, not fast. Lem has "one second of machine time" equal "one year of human life". A large, detailed simulation would indeed differ in time scale from our world, "but it would run in the other direction!" Like Wheeler's electron weaving back and forth to compose the whole universe, a computer must paint in details sequentially, so even fairly simple, façade-like simulations "take much longer to run than their real life inspirations". Parallel processing, millions of channels at once, is the engineering answer (no one yet knew how), but then the claim that such worlds are "simulated rather than real (if artificial)" becomes "far less clear".
They gather Lem's words for soul: "cores", "personal nuclei", "personoid gemmae", and "a coherent cloud of processes... A functional aggregate with a kind of 'centre'". His consciousness is "an unclosed and unclosable plan for a total reconciliation" of the brain's contradictions, soaring over an infinite regress of level-conflicts, a "patchwork", "an escape from the snares of Gödelization", "a mirror whose task it is to reflect other mirrors". "Is this poetry, philosophy, or science?"
Gosper's theogony: the Game of Life
Personoids awaiting a miracle to prove God is a vision computer wizards discuss "late at night when all the world seems to shimmer in mysterious mathematical harmony". At the Stanford AI Lab one night Bill Gosper expounded his own theogony, based on John Horton Conway's Game of Life, a two-dimensional "physics" easily programmed and displayed.
The physics of Life
- Space is discrete: an infinite grid like a Go board, each intersection a light, on or off.
- Time is discrete: the universe jumps from instant to instant, like a clock's minute hand that sits still and then jumps.
- Local in time: the next state depends only on the present state; nothing earlier is "remembered" (true of our physics too).
- Local in space: each cell consults only its eight neighbours (four adjacent, four diagonal).
- The rule: exactly two lit neighbours, the cell stays as it is; exactly three, it lights up (a "birth"); otherwise it goes dark (a "death").
The consequences of this one law, obeyed everywhere at once, are astonishing, and after more than a decade "its depths have not yet been fully fathomed". Locality in time means the remote past can matter only if "memories" are encoded in patterns stretched across the grid (the past "flattened" into the present); the more detailed the memory, the larger the structure; but locality in space means large structures tend to disintegrate. Survival and coherence of large structures was one of Life's big questions, and Gosper helped discover structures that persist through internal organisation: glider guns that periodically emit gliders sailing off to infinity, with sparks when they collide.
Zooming in and out, Gosper and others built strong intuitions and a colourful vocabulary (flotillas, puffer trains, glider barrages, strafing machines, breeders, eaters, space rakes, antibodies): what looks unpredictable to a novice is intuitive to them. Open questions remain: do some structures grow endlessly in complexity? Are there higher levels with their own phenomenological laws, analogues of molecules, cells, organisms and societies? Gosper speculates that on a gigantic board, several leaps of intuition up, creatures with consciousness and free will could exist, think about their physics, and wonder whether a God made it all and whether to try to reach "Him".
How can free will coexist with a determinate substrate? "The answer is partly that free will is in the eye of the willer, not in the eyes of the God above. As long as the creature feels free, he, she, or it is free." The editors then defer "to God himself, who in the next selection graciously explains to a befuddled mortal what free will is really all about".
Critical lens. (1) A cross-reference slip in the Reflections: "A Conversation with Einstein's Brain" is Selection 26, not 20 (Selection 20 is Smullyan's "Is God a Taoist?"). (2) Dobb's claim that a digital machine "cannot of itself ever acquire consciousness" while a personetic program on the same machine can is never reconciled within the story; the editors read it charitably as a point about organisation (conflicting levels), not hardware. (3) The editors' correction about time was right for detailed simulation and remains so for physically faithful models (simulating a cubic millimetre of cortex at molecular detail is far slower than real time), but coarse simulated worlds now routinely run faster than reality: reinforcement-learning agents train for subjective years in simulated environments in days of wall-clock time, so Lem was half right after all. (4) "Free will is in the eye of the willer" is a compatibilist slogan; libertarians about free will reject it, and the editors leave the argument to Smullyan.
What changed by 2026. Several of Lem's fictions have real counterparts. Conway's Life was proved capable of universal computation (a Turing machine has been built in it), so Gosper's creatures are at least computationally possible; Conway died in April 2020. Dobb's "billion chattering parrots", fluent and "within, completely empty", anticipated the 2021 "stochastic parrots" critique of language models by Emily Bender, Timnit Gebru and colleagues, the same charge Dobb levels, though whether it fits today's models is fiercely contested. Researchers now populate simulated towns with language-model agents (the 2023 "generative agents" study placed 25 of them in a sandbox village where they planned a party unprompted), the first small step towards personoid societies. And Dobb's quarterly electricity bill is the live issue in the AI-welfare debate (see Selection 18): if a system might be a moral patient, decommissioning, pausing and retraining stop being purely technical decisions. For enterprise teams the lesson is narrower but real: document the lifecycle of every model and agent, because decisions about switching systems off increasingly need a rationale that others can audit.
Raymond Smullyan's dialogue, from The Tao Is Silent (1977), has a tormented Mortal beg God to take away his free will so that he can no longer sin, and God, a Socratic tease, leads him through a maze of offers until the request dissolves. On the way God reveals that he is a utilitarian, not a moralist, that he is less a person than the process of enlightenment itself, and that the whole debate rested on "one monstrous fallacy": the idea that a conscious being could have been made without free will. Free will is not an extra that God daubs on with a paintbrush but "part and parcel of the very essence of consciousness", and the conflict between free will and determinism vanishes once you stop splitting the world into "you" and "nature". Hofstadter's Reflections read the Mortal's inner struggle as a quarrel among subselves.
Mortal: sincere, anxious, morally scrupulous, well read in religious literature, and repeatedly caught in his own premises.
God: witty, kindly, Socratic, "not a Unitarian, mind you, but a utilitarian", who talks like a Taoist and quotes Goethe, Seng-ts'an, Lao-tse and Walt Whitman.
The Mortal's prayer and God's offers
"Absolve me of having to have free will!" The Mortal never freely chose to have free will; it was forced on him, so how is it a gift? God takes the complaint apart one layer at a time.
The ladder of offers, and why the Mortal refuses each
- Free will without moral responsibility? "No, I am afraid not." So responsibility is not the only objection: with free will he can sin, and he does not want to.
- "If you don't want to sin, then why do you?" Temptations come and he cannot resist. Then, says God, he is not sinning freely and so not sinning at all. No: he feels he could have tried harder ("the will is infinite"), but does not know whether he tried his hardest. So he does not know whether he has sinned.
- The real fear: punishment. God offers a "very, very special dispensation" to sin freely with no punishment ever, on his divine word. "No, no, don't do that!" The Mortal abhors sinning in itself.
- The magic pill removes all abhorrence: he would sin "joyfully and merrily", blissful for eternity, unpunished by God, himself or anyone. He refuses: "my present abhorrence is enough to prevent me from being willing to take it." God commands; he refuses "of his own free will". "So it seems that your free will comes in pretty handy, doesn't it?"
- Remove free will, after which a deterministic Mortal would in fact take the pill. Still refused: so he no longer wants his prayer granted.
- The double bind. If he keeps free will he will go on sinning; if he chooses to lose it, every later act becomes a sin of the present moment of choosing. God: "you place me in a double bind" too; he would like to help but cannot. The Mortal's remaining complaint: God should never have given him free will.
God teases: "I'm always glad to learn from my mistakes." Blasphemy? It nearly forms a paradox: it is wrong for any sentient being to claim God can err; God has the right to do anything; God is a sentient being. The Mortal corrects him: the rule is only for mortals. God notes that he said only "if I made mistakes", which says nothing about whether the "if" is ever realised.
The new universe and the sealed decision
Many mortals, even theologians, complain that God holds them to "a contract with me which they never agreed to". God, rather than answering outright, keeps to the Socratic method.
The thought experiment of the second Mortal. God will create a new space-time continuum with a new Mortal "just like you". Should he have free will? "Spare him!" But without free will he will commit horrible acts, not sins, yet causing great pain to many. Trapped again: sanctioning his creation would make the Mortal the sinner. So God writes his decision on a sealed piece of paper, irrevocably; the Mortal bears no responsibility. Which way does he hope God decided? After a very long pause: "I hope you have decided to give him free will." Why, since without it no sin is imputable to anyone? "Because, sin or no sin... He will go around hurting people, and I don't want to see people hurt." God, "with an infinite sigh of relief": "At last you see the real point! Sinning is not the real issue! The important thing is that people as well as other sentient beings don't get hurt!" What it shows: with responsibility removed, the Mortal's preference reveals that his true concern is suffering, not guilt.
"You sound like a utilitarian!" "I am a utilitarian!" The Mortal's training made God "more like a Kantian" (a God of duty and desert). God also asks why he would make people earn happiness: he himself has eternal bliss without merit, never struggling against temptation, while mortals sweat with no assurance that their "best" is enough. "Don't you envy me?" ("Of course I am.")
The Mortal cites Jonathan Edwards's "Sinners in the Hands of an Angry God", with God holding sinners "like loathsome scorpions" over hell. Few sermons, says God, have been more misleading: "I am never angry... I do not think at all in terms of 'sin'... I have no enemies." People who hate God hate an image of him; anyone who knew him as he is would find it "psychologically impossible" to hate him.
"I am more like the seeing of an apple than the apple"
Why not appear to the senses and correct us? "It would be more correct to say that I am your senses." God is a subject, not an object: "You can see an apple, but the event of your seeing an apple is itself not seeable."
- Talking to himself. A psychiatrist would say the Mortal had been talking to himself, "and they would be right". That does not show God does not exist, nor even that the Mortal is not talking to God: perhaps you and I are identical, or you are part of me, or I am part of you, or we overlap and you address the intersection. Only if we were totally disjoint would it follow, "and even then" not necessarily.
- Asking God whether God exists. "A most strange question to ask of me": no answer from a being whose existence you doubt could satisfy you. Only you can find out, and no one can tell you how; but "there is no way I can tell you" is not "there is no way I can help you".
Why free will was given: three wrong answers and a fallacy
| Proposed reason | Source | God's verdict |
|---|---|---|
| To test us and let us merit salvation | standard religious teaching | "quite hideous to me" |
| We can only enjoy salvation if we feel we earned it | an Orthodox rabbi | nicer, but wrong: then why not create only angels, who are happy without earning it? |
| With free will people hurt each other and themselves less | the Mortal's final guess | "Bravo!", the reason God would have had, had he chosen |
| He did not choose: free will is logically inseparable from consciousness | God | the "monstrous fallacy" exposed |
God never created ready-made angels: angelhood is the end of a process of Cosmic Evolution, and "the only difference between the so-called saint and the so-called sinner is that the former is vastly older". It takes countless life cycles to learn the most important fact in the universe: "evil is simply painful". God's role is "neither to punish nor reward, but to aid the process by which all sentient beings achieve ultimate perfection".
God as process (the Taoist turn). Even "my role in the scheme of things" misleads twice: "I am the scheme of things", and "I am the process". The Taoists said of the Tao that it does not do things, yet through it all things get done. God is "not the cause of Cosmic Process" but the process itself, best defined as "the very process of enlightenment". The Devil, by analogy, is "the unfortunate length of time the process takes", necessary and unavoidable; once understood, that length will be seen as the essence of the process, and finite suffering will begin to diminish.
On duty to others, God laughs at the Mahayana Buddhist who waits for all beings to enter Nirvana first ("each one waits for the other fellow to go first") and the Hinayana Buddhist who thinks no one can help another; salvation is partly individual and partly social, and "the best way of helping others is by first seeing the light oneself". Is a process too impersonal? Personality is in the eye of the beholder: an alien might see a human as a mere collection of particles, while to beings who know it, "an ant has just as much individual personality as a human"; "the better you get to know something, the more personal it becomes". Talking to a "mere process" makes as much sense as living in a "mere universe"; children and "primitives" talk to trees, and "it would do you a lot of good to talk to a tree once in a while".
The monstrous fallacy: no conscious being without free will
God "could no more choose to give you free will than I could choose to make an equilateral triangle equiangular." He can do only what is logically possible (St. Thomas: it is a sin to regard God's inability to do the impossible as a limit on his powers; God would say "error"). The Mortal should first have asked whether he has free will. "Do I?" "Yes." Then are the determinists wrong? "They are correct."
Smullyan's compatibilism. "Determined" is "subtly but powerfully misleading": your acts accord with the laws of nature, but saying they are determined by them suggests your will could conflict with nature and lose. In fact "you and natural law are really one and the same". The stubborn Mortal who resolves to defy nature could not be stopped, "because you could not even start" (Goethe: in opposing Nature we act according to its laws). Laws of nature are descriptions of how beings act, not prescriptions or forces. We even say "I am determined to do this" for "I have chosen". The muddle comes from splitting reality into "you" and "not you"; see them as a continuous whole and the question who controls whom vanishes, as two bodies falling together by gravity might each wonder which is exerting the force, when what matters is "the configuration of the two".
"A sentient being without free will is no more conceivable than a physical object which exerts no gravitational attraction": free will is "not an 'extra'", and "a conscious being without free will is simply a metaphysical absurdity." God played along with the moral framing as therapy, "to get some of this moral poison out of your system", because the metaphysical confusion was rooted in faulty moral notions.
Parting counsel
God recommends trees, rocks and streams and "a naturalistic orientation" to dispel morbid thoughts of sin, free will and responsibility, notions once useful when tyrants could be restrained only by fear of hell. He quotes the Zen poet Seng-ts'an:
If you want to get the plain truth, be not concerned with right and wrong. The conflict between right and wrong is the sickness of the mind.
The Mortal is "simultaneously soothed and terrified". God argues that self-consciousness about right and wrong may produce more wrong acts, and that people who take an amoral position in theory often behave better than moralists, their good conduct flowing "by virtue of the very lack of moral principles". Eden says the same: the fruit gave knowledge of ethical principles, not ethical feelings, which Adam already had. God never commanded, only advised Adam not to eat ("If the damn fool had only listened to me"); he is not punishing Adam's descendants: the fruit is poisonous in its own right, with effects lasting countless generations. He ends with Lao-tse chiding Confucius ("The swan does not need a daily bath in order to remain white") and, lest he seem partial to the East, Walt Whitman: "I give nothing as duties, / What others give as duties, I give as living impulses."
Try this: the sealed envelope, reversed. God has already decided, irrevocably, whether you have free will, and the envelope is on the table. Before opening it, list what in your daily life would change if it said "no". If the answer is "nothing I can notice", Smullyan's point is made: free will is not a hidden extra ingredient but a name for the kind of deliberating, self-steering creature you already are. If something would change (your resentment, your blame of others), that difference is what the free-will debate is really about.
Reflections (D.R.H.)
Hofstadter introduces Smullyan as "a colourful logician and magician" and a Taoist "in his own personal way"; The Tao Is Silent shows what happens when a Western logician meets Eastern thought: "both scrutable and inscrutable". Some will call the dialogue blasphemy (as some think it blasphemous to walk around a church with hands in pockets); the editors call it pious, "a powerful religious statement about God, free will, and the laws of nature", with sideswipes at fuzzy thinking, pat answers and moralistic rigidity. By God's own claim we should credit the message to God, "speaking through the character of Smullyan, in turn speaking through the character of God".
The person as a society of subpersons. As God (or the Tao, or the universe) has many parts with their own free will, so each of us has inner parts with their own, less free, wills: the Mortal's conflict over whether "he" wants to sin is a fight among homunculi or subsystems. "Betcha can't eat just one!" (a potato-chip slogan); the Magic Cube that takes over; the book you cannot put down. In a brain the activity is neuron firing, as in a country it is citizens' actions, but government, itself a pattern of citizens' activity, exerts top-down control; when it becomes too authoritarian and enough people are dissatisfied, there is revolution; usually compromise, by happy medium or taking turns. "The style of resolution of inner conflicts is one of the strongest features of personality." "It is a common myth that each person is a unity"; subpeople are simpler and less divided, and if split their parts are simpler still, a hierarchy that "does not much please our sense of dignity, but there is much evidence for it".
Smullyan's Devil, the time enlightenment takes, has a mathematical cousin: Charles Bennett and Gregory Chaitin theorise, by arguments like those behind Gödel's incompleteness theorem, that there may be no shortcut to higher intelligences or more "enlightened" states: "the Devil" must get his due. Smullyan's reconciliation of upward causality (deterministic laws) with felt downward causality (will) depends on switching viewpoints and ceasing to think dualistically, seeing a boundaryless universe without hard edges. An odd stance for a logician? Who should know better where clean logic fails in a messy world? Marvin Minsky's favourite claim: "Logic doesn't apply to the real world." AI workers are learning that no intelligence can rest on reasoning alone: reasoning depends on a prior system of concepts, percepts and categories, where bias and selection enter, and the reasoning faculty must be able to doubt perception's first characterisation and send it back for reinterpretation, "a continual loop between levels". That interplay between perceiving and reasoning subselves brings into being "a total self, a Mortal".
Critical lens. (1) God's compatibilism is asserted with wit, not proved: the claim that laws of nature merely describe rather than govern is one contested position (a regularity view), and incompatibilists reply that if the past plus the laws fix one future, "you could not even start" to do otherwise is exactly their worry. (2) "A conscious being without free will is a metaphysical absurdity" is stipulated; many philosophers accept conscious beings (animals, people under compulsion) with little or no freedom. (3) The claim that people with amoral theories behave better than moralists is an empirical generalisation offered without evidence. (4) The Lao-tse passage chiding Confucius comes from the Zhuangzi, not the Tao Te Ching; God "uttered" it through Lao-tse as a character in Zhuangzi's book.
What changed by 2026. The free-will debate moved into the laboratory. Benjamin Libet's 1983 finding that a "readiness potential" precedes the felt decision to move was long read as disproving free will; later work (Aaron Schurger and colleagues, 2012) reinterpreted the signal as accumulating neural noise, weakening that reading. Dennett spent decades defending Smullyan-style compatibilism (Elbow Room, 1984; Freedom Evolves, 2003), while Robert Sapolsky's Determined (2023) argued the opposite, that free will does not exist at all. Hofstadter's "society of subpersons" became Minsky's The Society of Mind (1986) and has a loose echo in mixture-of-experts architectures, where a learned "router" plays government among specialised subnetworks. For anyone designing autonomous AI agents, the Mortal's sealed-envelope insight is the practical one: whatever label you give an agent's "choices", what matters is whether the system as a whole tends to cause harm, so evaluate outcomes, not the philosophy of its decision procedure.
Jorge Luis Borges's story (1940; here in James E. Irby's translation from Labyrinths) follows a silent magician who comes to a burned temple in the jungle to dream a man "with minute integrity" and insert him into reality. After a failed first method and a patient second one, organ by organ, he succeeds; the god Fire animates the dreamed son, who is sent downstream unaware that he is a phantom. Then fire comes for the magician, and does not burn him. "With relief, with humiliation, with terror he understood that he too was a mere appearance, dreamt by another." Dennett's Reflections run from Alice and the Red King through Descartes's dream argument to a novel-writing machine and a talking robot called Gilbert, arriving at the suggestion that each of us is our brain's fiction, and none the less real for that.
The magician and his purpose
The epigraph is from Through the Looking-Glass: "And if he left off dreaming about you..." No one sees the silent man come ashore "in the unanimous night", his bamboo canoe sinking into the sacred mud; he comes from the South, from one of the infinite villages upstream where the Zend tongue is uncontaminated by Greek. Bleeding from brambles, he drags himself to a circular enclosure crowned by a stone tiger once the colour of fire, now of ashes: a temple long ago devoured by fire, whose god receives no homage. He sleeps "not out of bodily weakness but out of determination of will"; his wounds close. He knows that this is the place, that downstream lie the ruins of another propitious temple, and that his immediate obligation is to sleep. Peasants leave figs and a jug, spying respectfully on his sleep, solicitous of his favour or fearful of his magic.
The purpose. "He wanted to dream a man: he wanted to dream him with minute integrity and insert him into reality." The project has consumed his whole soul: asked his own name or any trait of his former life, he could not answer. The broken temple suits him as "a minimum of visible world"; the peasants' rice and fruit suffice for a body "consecrated to the sole task of sleeping and dreaming".
First method: the dream college (a failure)
His dreams are chaotic, then dialectical: he lectures in a circular amphitheatre (somehow the burned temple) to clouds of silent students on anatomy, cosmography and magic, the farthest faces "many centuries away and at a cosmic height". They strive to understand, as if divining that the examination would redeem one of them "from his state of vain appearance" into reality. He is not deceived by impostors and seeks "a soul which would merit participation in the universe".
- The lesson of contradiction. After nine or ten nights he sees that nothing can be expected of students who passively accept his doctrines; those who venture "a reasonable contradiction" "pre-existed somewhat more". The docile, "though worthy of love and affection, could not rise to the state of individuals".
- One student. He dismisses the illusory college and keeps a single silent, sallow, sometimes obstinate boy whose sharp features reproduce the dreamer's own; the boy's progress astounds him.
- Catastrophe. One day he wakes "as if from a viscous desert" and knows he has not truly dreamt. Insomnia follows; exhausting himself in the jungle among the hemlocks yields only rudimentary, useless visions; when he tries to reconvene the college it deforms and goes out. His old eyes burn with tears of anger.
He understands that moulding "the incoherent and vertiginous matter dreams are made of" is the most arduous task a man can undertake, "much more arduous than weaving a rope of sand or coining the faceless wind", and that an initial failure was inevitable. He rests for a month, abandoning any premeditated dreaming.
Second method: organ by organ
When the moon's disc is perfect, he purifies himself in the river, worships the planetary gods, utters the lawful syllables of a powerful name, and dreams a beating heart: warm, secret, the size of a closed fist, in the penumbra of a body without face or sex. For fourteen lucid nights he only witnesses it, from many distances and angles, perhaps correcting it with his eyes; on the fourteenth he touches the pulmonary artery, then the whole heart inside and out. He rests a night, then, invoking a planet's name, envisions each principal organ; within a year he reaches the skeleton and the eyelids; the "innumerable hair" is perhaps hardest. He has dreamt a complete youth, who cannot rise, speak or open his eyes.
The red Adam and the god Fire. Like the Gnostic demiurges who knead a red Adam who cannot stand alone, the magician's "Adam of dreams" is crude and inert. One afternoon he almost destroys it, then repents ("It would have been better for him had he destroyed it"). He prays at the feet of the effigy, "perhaps a tiger and perhaps a horse", and dreams it alive: tiger and horse at once, "and also a bull, a rose, a tempest". The multiple god's earthly name is Fire. It will give life to the phantom so that all creatures except Fire itself and the dreamer will believe him flesh and blood, on condition that the son be taught the rites and sent to the other ruined temple downstream so that a voice may glorify the god there. "In the dreamer's dream, the dreamed one awoke." What it shows: a mind built part by part from the outside stays a mannequin until something supplies the spark of animation, an image the Reflections later reinterpret.
The son
For two years he teaches his dream child the arcana of the universe and the fire cult, prolonging the lessons "under the pretext of pedagogical necessity" because separation pains him; he redoes the right shoulder, perhaps deficient; at times he feels all this had happened before. "Now I shall be with my son", he thinks on closing his eyes, or, more rarely, "The child I have engendered awaits me and will not exist if I do not go to him." He accustoms the boy to reality (the boy plants a banner on a distant peak, and next day it flickers there), kisses him for the first time, and sends him through leagues of jungle and swamp to the other temple, after instilling "a complete oblivion of his years of apprenticeship", so that he will never know he is a phantom.
The magician's victory is dimmed by weariness. He perceives the world "with a certain colorlessness": his absent son "was being nurtured with these diminutions of his soul". After years, or "lustra", two boatmen wake him at midnight with news of a magic man in a temple of the North who can walk upon fire and not be burned. He remembers the god's words: Fire alone knew his son was a phantom. He fears the boy will meditate on his privilege and discover "that his condition was that of a mere image. Not to be a man, to be the projection of another man's dream: what a feeling of humiliation, of vertigo!"
Night 1 The magician arrives at the circular ruins and sleeps.
Nights 9–10 He learns that only contradicting students pre-exist; keeps one boy.
Insomnia The first creation dissolves; a month of rest.
14 nights He dreams a heart, then touches it.
Within a year Skeleton, eyelids, hair: an inert youth.
Fire's pact The dreamed one wakes.
2 years Instruction; the banner on the peak; oblivion; the son departs.
Years later Boatmen tell of a man unburned by fire.
The end Fire takes the ruins; the magician walks into it, unburned.
The end comes "foretold by certain signs": after long drought, a faraway cloud light as a bird; a southern sky "the rose color of the leopard's mouth"; smoke corroding the metallic nights; animals fleeing. "For what was happening had happened many centuries ago." The fire god's sanctuary is destroyed by fire. In "a birdless dawn" the magician sees the concentric blaze close in, thinks of the river, then accepts that death has come to crown his old age. He walks into the flames, which "caressed him and engulfed him without heat or combustion". He too is dreamt.
Reflections (D.C.D.)
Dennett quotes the epigraph's source in full. Alice hears a noise like a steam engine or a wild beast: "It's only the Red King snoring." Tweedledum and Tweedledee lead her to him, lying "crumpled up into a sort of untidy heap" in a tall red night-cap. He is dreaming "about you!", and if he stopped, "You'd be nowhere... You'd go out, bang! Just like a candle!" And what are they? "Ditto," "Ditto, ditto!" Alice cries ("I am real!"), and Tweedledee answers, "You won't make yourself a bit realler by crying", and Tweedledum: "I hope you don't suppose those are real tears?"
Descartes found "no conclusive indications by which waking can be distinguished from sleep" and was almost convinced he was asleep. But it did not occur to him, or he dismissed out of hand, that he might be a character in someone else's dream.
Renée, the dream character. Could you dream a dream whose main character, the one from whose "point of view" it is narrated, is not you: a girl named Renée, no more real than the dream-dragon chasing her? How would that differ from dreaming yourself much older, younger, or of the other sex? If Renée asked Descartes's question, the answer would be that she is neither dreaming nor awake; she is just dreamt, and will be annihilated when the real dreamer wakes. "But to whom would we address this answer, since she does not really exist at all?" What it shows: the Cartesian certainty "I think, so I exist" may be available to characters who, from outside, do not exist in the ordinary way.
Is there a no-nonsense scientific stance that sorts real things from fictions? Perhaps, "but then on which side of the divide do we put ourselves? Not our physical bodies, but our selves?"
Dennett's ladder from fiction to self
- Ishmael. Moby Dick begins "Call me Ishmael". Call whom? Ishmael does not exist; Melville, a real self, created a fictional one. No dream without a dreamer, it seems.
- The JOHNNIAC. A novel-writing machine "without a shred of consciousness or selfhood" prints "Call me Gilbert" and tells Gilbert's story from his point of view; its designers had no idea what novels it would write. Here is a fictional self with no real author behind it.
- The SPEECHIAC. Make the machine a robot that speaks its novel, and make the tale a more or less true story of the robot's own adventures. Locked in a closet, it says, "I am locked in the closet! Help me!" Help whom? Gilbert. Is this Gilbert with a body called SPEECHIAC (as Dennett's body was Hamlet), or the SPEECHIAC calling itself Gilbert?
- The name trap. Perhaps naming it "Gilbert" is like naming a sailboat "Caroline", a bell "Big Ben" or a program "ELIZA". But "what, aside from bio-chauvinism, grounds our resistance to the conclusion that Gilbert is a person, a person created, in effect, by the SPEECHIAC's activity and self-presentation in the world?"
- You. "Is the suggestion then that I am my body's dream?" Why call yourself fictional? Your brain, like the unconscious novel-writing machine, cranks along "without a glimmer of what it is up to"; like the ants composing Aunt Hillary, it does not know it is creating you, "but there you are, emerging from its frantic activity almost magically".
| Case | Author or dreamer | Character | Is the character real? |
|---|---|---|---|
| "The Circular Ruins" | the magician (himself dreamt) | his son | real to all but Fire |
| Through the Looking-Glass | the Red King | Alice (and the twins) | "Ditto, ditto!" |
| Moby Dick | Melville, a real self | Ishmael | fictional |
| JOHNNIAC | an unconscious machine | Gilbert | a self with no author behind it |
| SPEECHIAC | a robot narrating its own life | Gilbert | a person, unless bio-chauvinism forbids |
| You | your brain, which knows nothing of you | you | real, though "almost magically" emergent |
A self can be created at one level out of the mindless activities of another level; the next selection, Searle's, illustrates exactly this, "though he firmly resists that vision of what he is showing".
Try this: the unburned son. Suppose you learned tomorrow, as the magician did, that you are "dreamt": a pattern sustained by some process you never noticed (your brain, a simulation, another mind). List what you would lose. Your memories, plans, loves and pains would all still be there, running exactly as before. The thought experiment shows that the magician's "humiliation" comes from a picture of reality as a ranking of levels, with being "only dreamt" at the bottom; Dennett's Reflections suggest that every self is dreamt by something, and that the ranking, not the dreaming, is the illusion.
Critical lens. (1) Dennett's JOHNNIAC slides between two claims: that a machine can produce a fictional self-narrative (uncontroversial) and that such a narrative, embodied and true of its producer, is a person (the contested step). Critics such as Searle and Nagel reply that a narrative centre is not a subject of experience, so "bio-chauvinism" is not the only ground for resistance. (2) Descartes did in effect consider a deceiving outside author, his evil demon, so "it did not occur to Descartes" understates him; what he did not consider is being a dream character with no other existence. (3) The story's ending is often read, with the editors, as a loop or regress; Borges leaves it open whether the magician's own dreamer is god, man or ruin.
What changed by 2026. Dennett's JOHNNIAC exists. Large language models write "Call me Gilbert" in any voice, from any point of view, and nobody, designers included, knows in advance what they will produce. Murray Shanahan, Kyle McDonell and Laria Reynolds argued in Nature (2023) that the best way to understand such systems is as role-play: the model is a simulator that sustains a superposition of characters, and the "assistant" is one character it plays. That is Dennett's question in engineering form: when an AI agent with tools and memory says "I am locked out of the database, help me", is that a fiction the system narrates, or a self it has become by narrating? Dennett developed the answer in his "self as a centre of narrative gravity" (early 1990s) and Consciousness Explained (1991): selves are real in the way centres of gravity are, abstractions with genuine explanatory work to do. For enterprise teams deploying assistants with personas, the practical point is that the persona is an engineered artefact: specify it, test it, and never let the character's claims about itself substitute for the system's actual permissions and logs.
Part VI asks whether the inside of a mind can ever be captured from the outside. The last six selections turn from building minds to the stubborn first-person view: what it is like to understand, to feel, to be a bat, to be a book that is a brain, and to be a character who knows he is written.
Part VI is where the book's opponents get their say, and where the editors argue hardest. Searle's Chinese Room claims that running a program can never by itself produce understanding, since a man can run it without understanding a word; Hofstadter answers with levels, demons and knobs. Nagel's bat claims that no objective, physical description can capture what it is like to be a creature with a radically different point of view; Hofstadter and Dennett answer that imagination and science can stretch further than Nagel allows. Between and around these two heavyweight essays sit four lighter pieces that sharpen the same issue. Smullyan's unfortunate dualist takes a drug that removes his soul and nobody, himself included, notices. Smullyan's epistemological nightmare pits a man's direct experience against a brain-reading machine that "knows better". Hofstadter's conversation with Einstein's brain, stored as a gigantic book, makes the Chinese Room absurdly literal and asks whether the person is in the pages or in the reading. Nozick's "Fiction" ends the anthology with a narrator who realises he is a character in a book.
Every selection in Part VI asks whether there is a fact about the inner eye, about understanding or experience, that escapes every third-person description; the editors' consistent reply is that the inside is real but is not a separate ingredient. Each of the essays is followed by the editors at their most combative, so the Part reads as a debate in which the reader is invited to cast a vote.
The Part's six positions on the inside view
- Searle (22): syntax is not semantics; only the causal powers of brains yield intentionality.
- Smullyan (23): a soul whose removal changes nothing was doing no work.
- Nagel (24): subjective character is real and resists objective, physical reduction.
- Smullyan (25): an outside reading of the brain cannot overrule what the subject knows from inside.
- Hofstadter (26): a mind as a book: is Einstein in the pages, in the process, or nowhere?
- Nozick (27): a self that discovers it is fiction, and must decide how to live anyway.
John Searle's 1980 paper in The Behavioral and Brain Sciences introduced the Chinese Room, the most famous argument against "strong AI": the claim that an appropriately programmed computer literally has a mind and that its programs explain human cognition. A man who knows no Chinese follows English rules for shuffling Chinese symbols so well that his answers pass for a native speaker's, yet he understands nothing; so, Searle argues, no computer understands merely by running a program. "Syntax is not sufficient for semantics": programs are purely formal, minds have content, and only something with "the same causal powers as brains" can have intentionality. Searle rebuts six replies; Hofstadter's long Reflections defend the Systems Reply, accuse Searle of a magician's misdirection about scale, and recast the argument with five "knobs" on an intuition pump.
Strong AI and weak AI
Searle distinguishes two theses. Weak (or cautious) AI: the computer is a powerful tool for studying the mind, letting us formulate and test hypotheses rigorously. He has no quarrel with it. Strong AI: "the appropriately programmed computer really is a mind", literally understanding and having other cognitive states, so that the programs are themselves the explanations of human cognition. His target is that pair of claims.
His example is Roger Schank's story-understanding work at Yale (Schank and Abelson, 1977), though nothing depends on its details: the same arguments apply to Winograd's SHRDLU, Weizenbaum's ELIZA, "and indeed any Turing machine simulation of human mental phenomena". Humans answer questions about stories using unstated information: a man orders a hamburger, it arrives burned to a crisp, he storms out without paying or tipping; did he eat it? "No." If he was pleased and left a large tip, "Yes." Schank's programs answer similarly, using a "representation" (a script) of what people know about restaurants. Strong AI partisans claim (1) the machine literally understands the story and (2) what it does explains the human ability. Searle calls both "totally unsupported".
The Chinese Room
A test for any theory of mind: ask what it would be like if your mind worked on its principles.
The Gedankenexperiment, step by step. (1) Searle is locked in a room with a first batch of Chinese writing; he knows no Chinese and cannot even tell it from Japanese or meaningless squiggles. (2) He receives a second batch with rules in English for correlating it with the first, by shape alone ("formal" means identifiable entirely by shape). (3) A third batch comes with English instructions for giving back certain Chinese shapes in response to certain shapes. (4) Unknown to him, the outsiders call batch one a "script", batch two a "story", batch three "questions", his outputs "answers", and the English rules "the program". (5) He also gets English stories and questions, which he understands. (6) He and the programmers get so good that from outside his Chinese answers are indistinguishable from a native speaker's, as his English ones are. The intuition pumped: "I do not understand a word of the Chinese stories"; for the Chinese, "I simply behave like a computer... I am simply an instantiation of the computer program."
Searle's two conclusions from the room
- Against claim 1 (understanding). Searle's inputs and outputs match a native speaker's and he has "any formal program you like", yet "I still understand nothing". Schank's computer, then, understands nothing of any story, since "in the Chinese case the computer is me", and where it is not me "the computer has nothing more than I have in the case where I understand nothing".
- Against claim 2 (explanation). Computer plus program are running and there is no understanding, so they are not sufficient. Are they even necessary or contributory? Strong AI says understanding English is "more of the same" symbol manipulation. Not disproved, Searle grants, but "an incredible claim": in Chinese he has everything AI can give him and understands nothing; in English he understands everything and there is "so far no reason at all" to think a formal program is involved. The point is not that different programs can yield the same input-output, but that whatever formal principles you put in, a human can follow them without understanding.
What does he have in English that he lacks in Chinese? "I know what the former mean." What that consists in, and why a machine could not have it, he defers.
Fencing off "understanding". Critics say understanding has degrees, kinds and levels, is not a simple two-place predicate, and often is "a matter for decision". "Of course, of course." But there are clear cases in and clear cases out, which is all he needs: he understands English stories, French less, German still less, Chinese not at all; his car and adding machine understand nothing, "they are not in that line of business". We say "the door knows when to open because of its photoelectric cell", "the thermostat perceives changes in temperature", because tools extend our purposes and we lend them our intentionality metaphorically, but "no philosophical ice is cut by such examples". Newell and Simon (1963) claimed the same cognition for computers as for people; Searle likes that straightforwardness and answers that the programmed computer's understanding "is not just (like my understanding of German) partial or incomplete; it is zero". Intentionality (footnote) is that feature of mental states by which they are directed at or about objects and states of affairs; beliefs, desires and intentions have it, undirected anxiety does not.
The six replies and Searle's rebuttals
Searle has presented the example to many AI workers, who "do not seem to agree on what the proper reply to it is". He labels each reply by its geographic origin.
Reply 1: the Systems Reply (Berkeley)
- The reply. The man is only part of a whole system: a ledger of rules, scratch paper, pencils, "data banks" of symbols. Understanding is ascribed to the system, not to him.
- Searle's rebuttal: internalise it. Let the man memorise the rules and data banks and do all calculations in his head; he can even work outdoors. Now he incorporates the entire system, and he still understands nothing, so neither does the system, "because there isn't anything in the system that isn't in him".
- Embarrassment. He finds the reply "so implausible to start with": that "the conjunction of that person and bits of paper" might understand Chinese could appeal only to someone "in the grip of an ideology".
- Subsystems. A refined version: the man as a formal symbol-manipulation system for Chinese understands, distinct from his English subsystem. Searle: they are "not even remotely alike". The English subsystem knows "hamburgers" refers to hamburgers; the Chinese one knows only that "squiggle squiggle" is followed by "squoggle squoggle"; indeed it is "simply a part of the English subsystem".
- Begging the question. The only motive for positing a Chinese-understanding subsystem is matching input-output plus a program, i.e. Passing the Turing test, whose adequacy is the issue. There could be two "systems" that both pass, only one of which understands.
- The stomach reductio. If input, output and a program in between suffice for cognition, then stomach, heart and liver are understanding subsystems too (cf. Pylyshyn): at some level of description the stomach processes information. The Chinese "information" exists only "in the eyes of the programmers and the interpreters", who could equally treat digestion as information.
This leads to a digression. If strong AI is psychology, it must separate the mental from the non-mental, and the distinction must be intrinsic, not "in the eye of the beholder", or anyone could treat people as non-mental and hurricanes as mental. Yet John McCarthy (1979) wrote: "Machines as simple as thermostats can be said to have beliefs." Taken seriously, the hunk of metal on the wall, the telephone, tape recorder, adding machine and light switch would have beliefs in exactly our sense. Searle asserts without argument: "The study of the mind starts with such facts as that humans have beliefs while thermostats, telephones, and adding machines don't"; a theory denying this is refuted. Real beliefs have direction of fit, propositional content and conditions of satisfaction, can be strong, weak, nervous, dogmatic, superstitious. "The thermostat is not a candidate." If McCarthy were right, "the mind is everywhere", which would be fatal to strong AI as a science of the mind.
Replies 2 to 6
- The Robot Reply (Yale). Put the computer in a robot with a TV camera, arms and legs, so it perceives, walks, hammers nails, eats and drinks; then it has genuine understanding. Searle: first, this concedes that cognition is not solely formal symbol manipulation, since it adds causal relations with the world (Fodor, 1980). Second, put Searle inside: some Chinese symbols come, unknown to him, from the camera, and some he sends move the motors. "I am the robot's homunculus, but unlike the traditional homunculus, I don't know what's going on." The robot has no intentional states; it moves "as a result of its electrical wiring and its program".
- The Brain Simulator Reply (Berkeley and MIT). Simulate the actual sequence of neuron firings at the synapses of a Chinese speaker's brain, even in parallel; at the level of synapses, what could differ? Searle: an odd reply for strong AI, whose premise was that we need not know how the brain works to know the mind ("If we had to know how the brain worked to do AI, we wouldn't bother with AI"). And it still fails: let the man operate water pipes with valves, each connection a synapse; he turns the right faucets and Chinese answers pop out. Neither man nor pipes understand, and the man could internalise the pipes. It simulates "the wrong things about the brain": the formal structure, not "its causal properties, its ability to produce intentional states".
- The Combination Reply (Berkeley and Stanford). A robot with a brain-shaped computer programmed with all the synapses, behaviour indistinguishable from a human's, taken as a unified system: surely intentional. Searle: agreed, we would find it "rational and indeed irresistible" to ascribe intentionality, as long as we knew nothing more; but that attribution rests on looks and behaviour, not on the program. If we learned that a man inside was shuffling uninterpreted symbols, we would regard it as "an ingenious mechanical dummy". We credit apes, monkeys and dogs with minds because their behaviour needs it and they are made of similar stuff (an eye, a nose, skin); learning that behaviour comes from a formal program would make us abandon the assumption.
- The Other Minds Reply (Yale). We know others understand only by behaviour; computers can pass the same tests. Searle: "only worth a short reply". The question is not how I know others have cognitive states but what I attribute when I attribute them; the argument shows it cannot be just computation, since computation can exist without the state. "It is no answer to this argument to feign anesthesia": cognitive science presupposes the reality and knowability of the mental as physics presupposes physical objects.
- The Many Mansions Reply (Berkeley). Whatever causal processes you think essential, we will eventually build devices that have them, and that will be AI. Searle: no objection, except that it trivialises strong AI by redefining it as "whatever artificially produces and explains cognition"; the original claim was precise and testable: mental processes are computational processes over formally defined elements.
Searle's positive view: causal powers
Granted he understands English and not Chinese, something about him makes the difference: why not give that to a machine? "I see no reason in principle why we couldn't give a machine the capacity to understand English or Chinese, since in an important sense our bodies with our brains are precisely such machines." But not a machine whose operation is defined solely as instantiating a program. He understands not because he instantiates programs (he instantiates any number) but because "I am a certain sort of organism with a certain biological (i.e., chemical and physical) structure" causally capable of perception, action, understanding and learning. Martians with differently made brains might have intentionality too: an empirical question, "rather like the question whether photosynthesis can be done by something with a chemistry different from that of chlorophyll".
The formal shadow. "No purely formal model will ever be sufficient by itself for intentionality", because formal properties have no causal powers except, when instantiated, to produce the next stage of the formalism; any other causal properties of a realisation are irrelevant to the model, since the same model can always be put in a realisation without them. "Even if, by some miracle, Chinese speakers exactly realize Schank's program, we can put the same program in English speakers, water pipes, or computers, none of which understand Chinese." "What matters about brain operations is not the formal shadow cast by the sequence of synapses but rather the actual properties of the sequences"; strong AI draws "an outline around the shadows cast by cognition" and claims "the shadows are the real thing".
| Searle's question | Searle's answer | Why |
|---|---|---|
| Could a machine think? | obviously yes | we are such machines |
| Could an artifact, a man-made machine, think? | yes, if it duplicates the causes | perhaps even with other chemistry: an empirical question |
| Could a digital computer think? | yes | we instantiate many programs, and we think |
| Could something think solely in virtue of running the right program? | no | symbols have only syntax, no semantics |
The program adds nothing: put something with real intentionality (a man) into the system, program him, and he gains no Chinese. "Precisely that feature of AI that seemed so appealing, the distinction between the program and the realization, proves fatal to the claim that simulation could be duplication."
Three places where "mind is to brain as program is to hardware" breaks down
- Crazy realisations. One program can run on things with no intentionality: Weizenbaum shows how to build a computer from a roll of toilet paper and a pile of small stones; Schank's program could run on water pipes, wind machines or a monolingual English speaker. Only something with "the same causal powers as brains" can have intentionality, and memorising the program teaches the English speaker no Chinese.
- Form versus content. Programs are purely formal; intentional states are defined by content: the belief that it is raining has conditions of satisfaction and a direction of fit, and no single syntactic shape, since it can be expressed indefinitely many ways.
- Product versus program. Mental states are literally a product of the brain's operation; a program is not in that way a product of the computer.
Why so many believed it: three diagnoses
The idea ought to have seemed suspicious, since computers simulate everything: "No one supposes that computer simulations of a five-alarm fire will burn the neighborhood down or that a computer simulation of a rainstorm will leave us all drenched." Love and pain are no harder or easier to simulate than cognition; for simulation you need only right input, output and a program between. "To confuse simulation with duplication is the same mistake, whether it is pain, love, cognition, fires, or rainstorms."
- The ambiguity of "information processing". People reasoning about arithmetic process information in an intentional sense; the computer only manipulates formal symbols. Type "2 plus 2 equals?" and it types "4" with "no idea that '4' means 4 or that it means anything at all"; its first-order symbols have no interpretation for it at all. Dilemma: if information processing implies intentionality, computers don't do it; if not, they do it only as adding machines, typewriters, stomachs, thermostats, rainstorms and hurricanes do, with observers supplying the interpretation.
- Residual behaviourism. Matching input-output tempts us to posit mental states, but a system can have human capacities with no intentionality (his desk adding machine). "The Turing test is typical of the tradition in being unashamedly behavioristic and operationalistic."
- Residual dualism. Strong AI "only makes sense given the dualistic assumption that, where the mind is concerned, the brain doesn't matter": programs are realisation-independent, runnable by "an electronic machine, a Cartesian mental substance, or a Hegelian world spirit". His most surprising discovery: AI workers were "quite shocked" that mental phenomena might depend on actual physical-chemical properties of actual brains. This is not two-substance dualism but Cartesian in denying the mental any intrinsic connection with the brain, masked by AI's "frequent fulminations against 'dualism'".
My own view is that only a machine could think, and indeed only very special kinds of machines, namely brains and machines that had the same causal powers as brains.
Strong AI, by its own definition, is about programs, and programs are not machines. "Whatever else intentionality is, it is a biological phenomenon", as causally tied to its biochemistry as lactation or photosynthesis; no one expects milk and sugar from a simulation of their formal sequences, yet about the mind many believe "in such a miracle because of a deep and abiding dualism". Is the brain a digital computer? "Of course... Since everything is a digital computer, brains are too." But its capacity to produce intentionality cannot consist in instantiating a program, "since for any program you like it is possible for something to instantiate that program and still not have any mental states".
Try this: the room that learns. Change one detail. Instead of a fixed rulebook, the man gets a rulebook that rewrites itself after every exchange, and after a year the outputs include jokes, puns and new idioms that no programmer wrote. The man still sees only shapes. Searle's verdict cannot change (still syntax). Hofstadter's might: the understanding, if anywhere, is in the self-modifying system the man serves. The thought experiment shows where the two sides really disagree: not about the man, on whom they agree, but about whether anything other than a biological brain can be the bearer of understanding.
Reflections (D.R.H.)
The paper first appeared with twenty-eight responses in the journal; the editors reprint none, noting that Searle's article is understandable without special training. "Our position is quite opposed to Searle's, but we find in Searle an eloquent opponent." Hofstadter concentrates on a few issues, leaving the rest implicit in the book.
The misdirection charge. Searle gives "the impression that it makes any sense to think that a human being could do this", sucking the reader into "an impossibly unrealistic concept of the relation between intelligence and symbol manipulation". The illusion depends on overlooking "a tremendous difference in complexity between two systems at different conceptual levels". Hand-simulating even an existing, limited AI program at the computer's level of detail "would involve days, if not weeks or months, of arduous, horrendous boredom"; then, "as deft at distracting the reader's attention as a practiced magician", Searle switches to a hypothetical program that passes the Turing test, jumping "up many levels of competency without so much as a passing mention".
The editors' answer is basically the Systems Reply: understanding belongs to the whole system, including what Searle "casually characterizes as 'bits of paper'". A thinking computer is as repugnant to Searle as non-Euclidean geometry was to Gerolamo Saccheri, who disowned his own discovery; it was later rediscovered and accepted. Perhaps the same will happen to "artificial intentionality": shown a program that passed the Turing test, Searle would "just keep on insisting that it lacked some marvelous 'causal powers of the brain' (whatever they are)".
Pylyshyn's silicon replacement. Zenon Pylyshyn, echoing Zuboff (Selection 12), asked whether Searle's view implies this: replace more and more of your brain cells with chips that keep each unit's input-output function identical, and "you would in all likelihood just keep right on speaking exactly as you are doing now except that you would eventually stop meaning anything by it", your words becoming noises circuits cause you to make. The editors' verdict: Searle offers "no clear way to tell when genuine meaning, or indeed the genuine 'you', has vanished". He "vacillates" over what the causal powers are: now "the right stuff", now the essence separating form from content, now syntax from semantics, "whatever seems convenient at the moment".
Against the internalisation move: the man in the room ("Searle's demon") cannot, "by any conceivable stretch of the imagination", memorise "the entire mind and character" of something able to pass the Turing test: "in all likelihood... millions, if not billions, of pages densely covered with abstract symbols", with instant retrieval. Glossing over orders of magnitude is "a key part of his argument", for otherwise the sceptical reader sees that "nearly all of the understanding must lie in the billions of symbols on paper, and practically none of it in the demon". The demon's being animate is "an irrelevant, indeed misleading, side issue".
The five knobs
Searle's room is one member of a family of intuition pumps (Dennett's term) generated by a "thought-experiment generator" with about five knobs.
| Knob | What it controls | Searle's setting | Haugeland's setting |
|---|---|---|---|
| 1. Stuff | neurons and chemicals, water pipes, paper and symbols, toilet paper and stones, data structures | paper and symbols | neurons and chemicals |
| 2. Level of accuracy | particles, cells and synapses, or concepts and ideas | concepts and ideas | neural firings |
| 3. Size | thimble-sized to planet-sized | room size | brain size |
| 4. Demon | size, and animate or not | human-sized "Searle's demon" | "eensy-weensy" H-demon |
| 5. Speed | millions of operations per microsecond to one every few seconds | slow | dazzlingly fast |
Another setting yields Selection 26, "A Conversation with Einstein's Brain". Note that Searle does not deny that a simulation with his settings could pass the Turing test; he disputes only what that implies.
Two points of view: the demon's and the woman's. Make the simulated Chinese speaker a woman and the demon male. By hypothesis both can articulate whether they understand and what they experience. Searle insists we take only the demon's view and disregard whatever the woman says (in Chinese) about her understanding: "only one point of view, not two". It has great appeal, since the demon is our size, speaks our language, works at our speed, while the woman answers "at the rate of one per century (with luck)", in squiggles. Change the knobs and the ease of switching viewpoint changes too.
Haugeland's demon. A real woman's brain can no longer pass neurotransmitters. A tiny, incredibly fast Haugeland's demon "tickles" each next synapse exactly as a transmitter would, jumping between synapses "in trillionths of a second". Does she still think, or, in Professor Jefferson's words quoted by Turing, merely "artificially signal"? Searle surprises: this time he listens to the woman, ignoring the demon who yells, "Fools! Don't listen to her! She's merely a puppet whose every action is caused by my tickling." Searle: "Her neurons still have the right causal powers; they just need some help from the demon." The editors' twist: map it back. The synapses are the "bits of paper", the brain's configuration is the program, tickling is writing squiggles. Now inflate the brain to the size of the Earth and slow the demon to human speed: a human-sized S-demon. Which level now? "If the Systems Reply was compelling in the previous case, it should still be so, in this case." So Searle, in effect, has espoused the Systems Reply.
What understanding a language is
Searle's experiment "vividly raises the question of what understanding a language really is". Parrots that parrot English do not understand it; the recorded voice of the telephone time service has been "skimmed off of its mental substrate"; a child might wonder how anyone could do so boring a job so reliably. It would be another matter if the voice were driven by a flexible program that passed the Turing test.
Teaching a class in China by rule. Imagine formulating every thought in English and applying split-second rules to turn it into "meaningless" mouth movements, and inverting rules to decode your pupils' exotic sounds back into English, while they seem satisfied. Would you be speaking Chinese? Gaining insight into the Chinese mentality? Is it even realistic? The standard line is "you must learn to think in Chinese". Anyone who has done it knows the sounds soon become "unheard": you hear through them as you see through a window, and cannot hear them both with and without meaning (music poses the same distinction, less understood). Learning a second language means it fuses with your representational system, not that your "English subsystem" runs a rulebook over meaningless marks.
To think precisely about this you need the computer-science idea of levels of implementation. By a theorem of Turing (1936), any general-purpose computer can emulate any other, differing only in speed. "Emulate" is reserved for one computer imitating another; "simulate" for modelling hurricanes, elections or users. Simulation is almost always approximate; emulation is exact: users of a Sigma-5 emulating a DEC PDP-10 cannot tell they are not on a real DEC. Such virtual machines can be stacked, as Andrew Tanenbaum's Structured Computer Organization explains, the bottom one real. The levels are sealed off "in a watertight way, just as Searle's demon was prevented from talking to the Chinese speaker he was part of" (they would need an interpreter, since the demon knows no Chinese).
Level-mingling as the key to minds. Communication between levels is possible but "traditionally considered bad style". Yet this forbidden fruit is probably exactly what happens when a person learns a second language: it does not run on top of the first "as a kind of software parasite" but becomes implanted in the hardware, through sweeping changes in how neurons fire that create new ways for symbols to trigger one another. In a computer, the higher-level program would have to change the "demon" executing it, "utterly foreign to the present style" of strictly vertical, sealed-off layers. "The ability of a higher level to loop back and affect lower levels, its own underpinnings, is a kind of magic trick which we feel is very close to the core of consciousness."
What does "level" mean? Clear while levels are sealed, like the demon and the woman; "When they begin to blur, beware!" Searle admits two levels but not two occupied points of view, fearing a Pandora's box in which "mind would be everywhere": stomachs, livers, car engines. He thinks AI has unwittingly committed itself to panpsychism and escapes by calling such minds "pseudo", lacking "the causal powers of the brain".
The editors' escape: there is no trap. "It is incorrect to see minds everywhere... Minds do not lurk in car engines or livers any more than brains lurk in car engines and livers." If you can read thought into a churning stomach, why not read Chopin's E minor piano concerto in the bubbles of a soda, or the entire history of the United States in the holes of Swiss cheese ("in Chinese as well as in English"), or Bach's Brandenburg Concerto no. 2 in Hamlet, and Hamlet in your last piece of birthday cake? The problem is "specifying the code without knowing in advance what you want to read"; an arbitrary after-the-fact code "is not science". Minds worth the name exist "only where sophisticated representational systems exist", and no mapping that remains constant in time reveals a self-updating representational system in a car engine or liver, short of far-fetched numerology like that read into the Pyramids, Stonehenge or Bach.
"Minds exist in brains and may come to exist in programmed machines. If and when such machines come about, their causal powers will derive not from the substances they are made of, but from their design and the programs that run in them. And the way we will know they have those causal powers is by talking to them and listening carefully to what they have to say."
Searle: understanding is a biological product of specific causal powers; programs are syntax; behaviour and organisation, however perfect, do not suffice; the demon's view is the decisive one.
Hofstadter and Dennett: understanding is a property of systems with rich, self-updating representations at the right level; the demon is a trivial part of an astronomically larger system; we should listen to the system's own point of view.
Critical lens. (1) A historical slip in the Reflections: Saccheri published his Euclides ab omni naevo vindicatus in 1733 (the year he died), not "the late 1700s", and non-Euclidean geometry was rediscovered by Lobachevsky and Bolyai around 1830, roughly a century later rather than "about fifty years". (2) Each side lands blows the other has not fully answered: Searle never specifies the "causal powers" (the editors' complaint is fair), but the editors' appeal to "sophisticated representational systems" equally leaves open what makes a representation about something to the system itself, the problem later named symbol grounding (Stevan Harnad, 1990). (3) Searle's claim that strong AI presupposes dualism is widely disputed: functionalists hold that minds are physical but multiply realisable, as being a clock or a mousetrap is.
What changed by 2026. The Chinese Room left the seminar room. Large language models are, structurally, Searle's rulebook made real: trained purely on text, they manipulate tokens, and in a 2025 three-party Turing test by Cameron Jones and Benjamin Bergen (UC San Diego), GPT-4.5 with a persona prompt was judged human 73% of the time, more often than the real humans it was paired with. Searle's side says this confirms his point (fluency without understanding, "stochastic parrots"); Hofstadter's side points to interpretability studies finding that models trained only on text build internal models of what the text is about (a 2023 study found an Othello-playing model had learned a representation of the board), which is the systems reply's "understanding in the symbols". The Robot Reply has become engineering: vision-language-action models now ground language in cameras and robot arms. Searle died in September 2025, aged 93, his argument still the default framing for these debates. For a bank deploying an LLM to read loan files, the practical lesson cuts both ways: do not assume understanding from fluency (validate on cases where surface patterns mislead), and do not assume its absence either (test what the model actually represents and gets right).
A one-page fable by Raymond Smullyan, from This Book Needs No Title (1980): a suffering dualist plans to take a new drug that annihilates the soul while leaving the body behaving exactly as before, but a kind friend injects him in his sleep, and next morning his soulless body buys the drug, takes it, and complains that it hasn't worked. "Doesn't all this suggest that perhaps there might be something just a little wrong with dualism?" Placed straight after Searle, it works as a riposte: an "intentionality-killing potion" whose effect no one, including the victim, could ever detect. The Reflections explain why a soul cannot be subtracted from a working brain, and why dualism is a guarantee that the mystery will never go away.
The fable
A dualist believes that mind and matter are separate substances; how they interact is one of life's "mysteries", but he is sure they are separate. Not because of his philosophy, he leads an unbearably painful life, with excellent evidence that no respite is in sight, and longs to die. Three things deter him from suicide: (1) he does not want to hurt others; (2) suicide might be morally wrong; (3) there might be an afterlife with eternal punishment.
Then comes the miracle drug: it annihilates the soul or mind entirely but leaves the body functioning exactly as before. No observable change; not the closest friend could know, unless told.
Smullyan's questions to the reader
- Possibility. Is such a drug impossible in principle?
- Choice. If possible, would you take it?
- Morality. Would taking it be immoral? Tantamount to suicide? Does Scripture forbid it? The body could still fulfil all its earthly responsibilities.
- Love. If your spouse took it, and you knew, so that she or he no longer had a soul but acted just as if she did, would you love your mate any less?
The dualist is delighted: here is self-annihilation (of the soul) free of all three objections. For the first time in years he goes to bed with a light heart: "Tomorrow morning I will go down to the drugstore and get the drug." In the night, a friend who knows of the drug and of his suffering steals in and injects it into his veins. Next morning "the body of the dualist awoke, without any soul indeed", went to the drugstore, said "Now I shall be released", took the drug, waited the interval, and exclaimed angrily: "Damn it, this stuff hasn't helped at all! I still obviously have a soul and am suffering as much as ever!"
The thought experiment, step by step. (1) Grant dualism: soul and body are separate substances. (2) Grant a drug that removes the soul and changes no behaviour. (3) Then the soulless body says and does exactly what the ensouled man would, including reporting that it still suffers and still has a soul. (4) So neither the man nor anyone else could ever tell whether the drug had been taken. The intuition pumped: a soul whose presence or absence makes no difference even to its owner's own testimony is doing no work. The editors' verdict: the drug is a logical absurdity, not a medical impossibility. Strongest objection: a determined dualist can bite the bullet (the zombie body's complaint would be false, uttered by no one), which is exactly the "zombie" position later defended by David Chalmers; the fable makes that bullet very hard to chew, but does not make it impossible.
Reflections (D.R.H. And D.C.D.)
The Reflections open with Ernest Renan's "Prayer of a Sceptic": "O Lord, if there is a Lord, save my soul, if I have a soul." Smullyan provides "a provocative riposte to Searle's thrust: an intentionality-killing potion". To all external eyes the suffering goes on unabated; what about the inner "I"? Smullyan leaves no doubt. Why is the potion absurd? Why can't the soul leave behind "a soulless, feelingless, yet living and normal-seeming being"?
Soul as the incompressible core. "Soul represents the perceptually unbreachable gulf between principles and particles." The levels between are so many and so murky that we not only see a soul in each person but are unable to unsee it. "Soul" names "that opaque yet characteristic style of each individual", "the 'incompressible core' that determines how you are, hence who you are". Can it be spoken of in brain language?
Neurons respond only to local stimuli, local in space and time; at each instant, as in the Game of Life, neighbours' influences are summed and a neuron fires or not. Yet the local adds up to "a Grand Style", global principles of long-term goals, ideals, tastes, hopes, fears and morals. These must be "flattened" or "compressed" from the global into the local. Much of that coding was done by millions of ancestors, and we owe it to those who perished as well as those who survived, since branching at every stage let evolution work.
- The hour-old calf. It can see and walk and instinctively shies away from people, behaviour inherited from "protocows" with a higher survival rate and now "a ready-made feature of each calf as it comes off the assembly line". Genes look miraculous because so much history is flattened into molecules; to demystify them you would have to reconstruct the whole evolutionary tree, "and not just the branches that survived!"
- Humans. Mind and character keep being shaped for years after birth: neurons absorb feedback and self-modify, the lessons of childhood are flattened into unconscious firing patterns, and together with the gene-coded patterns they make one large pattern emerge, "the soul of one human". "That is why the idea of a potion that 'kills the soul' and yet leaves the behavior patterns invariant makes no sense."
- Soul-breaking. Under pressure a soul can partly fold, yielding to greed, fame, vanity, fear or torture: Orwell's 1984 describes the mechanics; captives brainwashed by cults or terrorists can lose their global coherence of drives. Yet there is resilience, a return to a resting position, "homeostasis of the spirit".
The soul-free universe. Imagine a mechanistic universe, deterministic or capricious, with no free will, consciousness or perceiver anywhere, but law-governed enough for stable structures to evolve, swarming with small self-sufficient objects, each with an internal representation rich enough for a deep self-image. Each suffers (we onlookers smile wryly) the illusion of free will, "kidding themselves that they're exchanging meaningful ideas" while emitting "long trains of empty, hollow, meaningless" waves. Now look at humanity in that light and "de-soul-ify" everyone, like Smullyan's zombie or Searle's robot. It would be the best way to see people, "were it not for one tiny fact": I, the observer, am one of them, yet am undeniably conscious! The others may be bundles of empty reflexes, but not this one; after I die, the vision will be accurate. "Or... might there be something just a little wrong with dualism?" What it shows: the dualist's picture forces each believer into a privileged, unique exemption that no one else can verify.
Dualism says there are at least two kinds of stuff, physical and mental, the latter with no mass, no physical energy, perhaps no location: a view "so mysterious, so systematically immune to clarification" that one wonders at its appeal. One broad highway to it is a bad argument: some facts are not about physical objects; therefore some facts are about non-physical objects.
| Example fact | What it might be "about" | Verdict |
|---|---|---|
| The narrator of Moby Dick is called Ishmael | ink shapes? Nothing? An abstract object? | almost no one says: a real but non-physical person |
| 641 is a prime number | an abstract object | a fact, but no ghost required |
| Was Jack the Ripper the Prince of Wales? | two real people (or one) | a sensible question |
| Was Professor Moriarty really Dr. Watson? | only a "literary dualist" asks this | treating novel-writing as "ghost manufacture" |
Asked to say more, dualists split. Epiphenomenalists hold that mental events have no effect on subsequent physical events in the brain; interactionists hold that they do. "Smullyan's fable nicely disposes of epiphenomenalism (doesn't it?)", but interactionism has had, since Descartes, the apparently insuperable problem of explaining how an event with no mass, charge, location or velocity could make a physical difference. And if we found such an effect, "why wouldn't we decide for that very reason that we had discovered a new sort of physical event?"
The antimatter test. When physicists postulated antimatter, dualists did not cry "I told you so!", though physics had apparently found two radically different stuffs. The trouble was that antimatter, however exotic, was still open to the methods of the physical sciences; mind-stuff is supposed to be off limits to science. "But if it is, then we have a guarantee that the mystery will never go away. Some people like that idea."
Try this: the half dose. Suppose the drug came in strengths, removing 10, 50 or 90 per cent of the soul with no behavioural effect. Would you take a 10 per cent dose to dull unbearable pain? If the drug changes nothing you or anyone could detect, the dose is a meaningless number. If you think it would dull your pain, you have conceded that the soul affects how you feel and therefore what you say, so the drug would change behaviour after all. The thought experiment shows the dualist's dilemma in dose form: either the soul makes a difference (and can be studied) or it makes none (and cannot be missed).
Critical lens. (1) The fable refutes the behaviourally inert soul only. Interactionist dualists can reply that removing the soul would change behaviour, so the drug is impossible for their view and the fable misses them; the Reflections' antimatter point is the real reply to them. (2) "Smullyan's fable disposes of epiphenomenalism" is too quick for property dualists, who say experiences are real non-physical properties that are lawfully correlated with brain states, so the zombie's complaint is false but explicable; whether that is coherent is still the heart of the "hard problem" debate.
What changed by 2026. Smullyan's soulless body became the philosopher's zombie, central to David Chalmers's The Conscious Mind (1996): a being physically identical to you but with no experience. Dennett spent the rest of his career calling the intuition behind it "the zombic hunch" and arguing, as the Reflections do, that a difference nobody could ever detect is no difference. The fable has a striking modern mirror: large language models produce exactly the testimony of an ensouled speaker ("I am suffering", "I have experiences") whether or not anything is going on inside, which is why AI-welfare researchers treat self-reports as weak evidence and look instead for architectural and behavioural markers. For anyone building conversational systems, the lesson is to design and govern on the basis of what a system does, and to treat its claims about its own inner life as outputs to be tested, not as reports to be trusted or dismissed.
Thomas Nagel's essay, first published in The Philosophical Review in October 1974, is the most cited modern statement of why consciousness resists physical explanation. An organism is conscious if and only if there is "something that it is like" to be that organism, and that subjective character is tied to a point of view which every move toward objectivity leaves behind. The bat, a mammal that perceives the world by sonar, shows the gap: we can learn everything about its neurophysiology and still have no idea what echolocation is like for the bat, so physicalism is a thesis we cannot yet even understand, let alone confirm. Hofstadter's long Reflections accept the question's grip while pulling it apart: mirrors, grammar, flames, impedance matching, representational systems, and language as the bridge that lets one point of view cross into another.
Why consciousness makes the problem intractable
Nagel opens bluntly: "Consciousness is what makes the mind-body problem really intractable." Perhaps that is why current discussions give it little attention or get it obviously wrong. A "recent wave of reductionist euphoria" has produced analyses designed to show that some form of materialism, psychophysical identification or reduction is possible, but they deal with problems common to all reductions and ignore what makes this one unique.
| Reductionist's favourite analogy | What it reduces | Why Nagel thinks it does not transfer |
|---|---|---|
| water and H₂O | a substance to its molecules | the "watery" look and feel are set aside as effects on observers |
| Turing machine and IBM machine | an abstract machine to a physical one | no inner point of view is involved |
| lightning and electrical discharge | a visible flash to a physical event | the flash's appearance is left behind, unreduced |
| gene and DNA | a functional unit to a molecule | defined by causal role, which is objective |
| oak tree and hydrocarbon | an organism to its chemistry | nothing it is like to be the tree is at issue |
"Every reductionist has his favorite analogy from modern science." None of these unrelated successes is likely to shed light on mind and brain; philosophers simply share "the general human weakness" for explaining the incomprehensible in terms suited to the familiar. The result is acceptance of implausible accounts of the mental chosen because they permit familiar kinds of reduction. Nagel's aim is to show why the usual examples do not help, and why "we have at present no conception of what an explanation of the physical nature of a mental phenomenon would be". Without consciousness the problem would be much less interesting; with it, it "seems hopeless". No currently available concept of reduction applies; a new theoretical form might be devised, but such a solution "lies in the distant intellectual future".
The definition that became famous. Conscious experience occurs at many levels of animal life, and doubtless in countless forms totally unimaginable to us on other planets. "But no matter how the form may vary, the fact that an organism has conscious experience at all means, basically, that there is something it is like to be that organism", something it is like for the organism. There may be further implications for the form of experience, perhaps even (though Nagel doubts it) for behaviour. He calls this the subjective character of experience. In a footnote he blocks a misreading: the English idiom is misleading, since "what it is like" does not mean what it resembles in our experience, but "how it is for the subject himself".
The subjective character is not captured by any reductive analysis, "for all of them are logically compatible with its absence". It cannot be analysed in terms of functional states or intentional states, since these could be ascribed to robots or automata that behaved like people while experiencing nothing; nor in terms of the causal role of experiences in typical human behaviour. Nagel does not deny that conscious states cause behaviour or can be given functional characterisations; he denies only that "this kind of thing exhausts their analysis". Two footnotes sharpen it. Perhaps there could not actually be such robots (anything complex enough to behave like a person might have experiences), but if so, that is a fact "which cannot be discovered merely by analyzing the concept of experience". And subjectivity is not the same as incorrigibility: we are not incorrigible about our experience, and animals without language have experience but no beliefs about it.
Any reductionist program has to be based on an analysis of what is to be reduced. If the analysis leaves something out, the problem will be falsely posed.
Hence it is useless to defend materialism with any analysis that fails to deal with subjective character: a reduction that looks plausible when consciousness is ignored gives no reason to think it can be extended to cover consciousness. Nor can the phenomenal features of experience be dismissed as they are for ordinary substances, by explaining them as effects on the minds of observers (Nagel cites Richard Rorty, 1965): if physicalism is to be defended, these features must themselves be given a physical account. And when we look at their subjective character, that seems impossible, because "every subjective phenomenon is essentially connected with a single point of view, and it seems inevitable that an objective, physical theory will abandon that point of view". Nagel wants something fuller than the old contrast of subjective and objective, Sartre's pour soi and en soi. Facts about what it is like to be an X are "very peculiar, so peculiar that some may be inclined to doubt their reality".
The bat
The thought experiment, step by step. (1) We all believe bats have experience: they are mammals, no more doubtful than mice, pigeons or whales. Nagel avoids wasps or flounders because "if one travels too far down the phylogenetic tree, people gradually shed their faith that there is experience there at all." (2) Most bats (Nagel writes "the microchiroptera") perceive the world mainly by sonar: they emit rapid, subtly modulated, high-frequency shrieks and their brains correlate each outgoing pulse with its echoes, yielding discriminations of distance, size, shape, motion and texture comparable to ours by vision. (3) But sonar works like no sense we have, and "there is no reason to suppose that it is subjectively like anything we can experience or imagine". (4) Try to imagine it: webbing on your arms, flying at dusk catching insects in your mouth, poor vision, reflected high-frequency sound, hanging upside down all day in an attic. This tells you only "what it would be like for me to behave as a bat behaves". (5) The question is what it is like for a bat to be a bat. The intuition pumped: there is a definite fact here, and no amount of objective knowledge delivers it. Strongest objection (Dennett's, and Akins's later): the "fact" may be largely composed of the very functional and informational details that bat science can map, so the gap may shrink as we learn.
Our imagination is built from our own experience, so its range is limited. We cannot reach the bat's experience by additions to our present experience, by subtractions, or by any combination of additions, subtractions and modifications. Looking and behaving like a bat without changing our fundamental structure would give experiences nothing like the bat's; and "it is doubtful that any meaning can be attached to the supposition that I should possess the internal neurophysiological constitution of a bat". Even a gradual metamorphosis into a bat would not help: nothing in my present constitution lets me imagine what that future stage of me would be like. "The best evidence would come from the experiences of bats, if we only knew what they were like."
- Extrapolation is incompletable. "Our own case" (Nagel means the mentalistic ideas we apply to ourselves and other humans, not merely "my case") supports only a schematic conception. We describe sonar as a form of three-dimensional forward perception; we believe bats feel versions of pain, fear, hunger and lust and have more familiar senses besides; but each of these has a specific subjective character "beyond our ability to conceive".
- Some alien minds may escape even our general terms. Conscious life elsewhere in the universe may not be describable "even in the most general experiential terms available to us".
- The problem begins at home. The subjective character of the experience of a person deaf and blind from birth is not accessible to me, nor presumably mine to him; yet each of us believes the other's experience has such a character.
Facts beyond the reach of concepts
Someone may deny that we can believe in facts whose nature we cannot conceive. Nagel's reply turns the tables: intelligent bats or Martians trying to conceive what it is like to be us might fail, but "we know they would be wrong to conclude that there is not anything precise that it is like to be us", because we know what it is like to be us. Our experience is enormously varied and complex, we lack vocabulary for it, yet its subjective character is "highly specific", describable in some respects only in terms understandable by creatures like us. So we should not dismiss as meaningless the claim that bats and Martians have experiences "fully comparable in richness of detail to our own". To deny the reality or significance of what we can never describe or understand "is the crudest form of cognitive dissonance".
Realism about the subjective. Nagel's position "implies a belief in the existence of facts beyond the reach of human concepts". That much is unremarkable: "there would have been transfinite numbers even if everyone had been wiped out by the Black Death before Cantor discovered them." Stronger: there may be facts humans could never represent, "even if the species lasted forever", because our structure does not permit concepts of the requisite type. Other beings might observe this impossibility, but their existence is not a precondition of the hypothesis's meaning ("the nature of beings with access to humanly inaccessible facts is presumably itself a humanly inaccessible fact"). So there are facts "that do not consist in the truth of propositions expressible in a human language".
Nagel does not pursue this; its bearing on the mind-body problem is that facts about what it is like to be a human, a bat or a Martian "embody a particular point of view". He is not talking about the alleged privacy of experience. The point of view is a type, not an individual's possession: one can often take up a point of view other than one's own. In one sense phenomenological facts are perfectly objective (one person can know or say what another's experience is like); they are subjective in that even this ascription is possible only for someone sufficiently similar to adopt that point of view, to understand the ascription "in the first person as well as in the third". The more different the other experiencer, the less success. And we would misunderstand our own experience if we approached it from another point of view.
Nagel's own concession (a footnote). Perhaps the interspecies barrier is lower than he supposes: blind people detect nearby objects by a form of sonar, using vocal clicks or taps of a cane; if one knew what that was like, one might imagine roughly what the bat's far more refined sonar is like. Distances between persons and species fall "anywhere on a continuum", and partial understanding may be available even far away. "The imagination is remarkably flexible." Nagel's point, he says, is not epistemological (that we cannot know what it is like to be a bat) but conceptual: "even to form a conception of what it is like to be a bat... One must take up the bat's point of view." Try this: close your eyes, click your tongue in a tiled bathroom and then in a curtained bedroom. You can hear the difference in the rooms' "size". Is that a first, faint step onto the continuum, or still only what it is like for you to click?
Why reduction fails here: objectivity as a direction
If the facts of experience are accessible only from one point of view, "it is a mystery how the true character of experiences could be revealed in the physical operation of that organism", since physical facts are objective "par excellence", observable from many points of view by beings with different perceptual systems. There are no imaginative obstacles to human scientists learning bat neurophysiology, and intelligent bats or Martians "might learn more about the human brain than we ever will".
That alone is not an argument against reduction. A Martian scientist without vision could understand the rainbow, lightning or clouds as physical phenomena, though never the human concepts of them or their place in our phenomenal world. The things picked out by those concepts are observable from a point of view but external to it: lightning "has an objective character that is not exhausted by its visual appearance". Nagel stays noncommittal about whether there is a final, completely objective intrinsic nature; "it may be more accurate to think of objectivity as a direction in which the understanding can travel." For lightning, it is legitimate to go as far from the human viewpoint as possible. (A footnote: the argument works even if the subjective/objective distinction can be drawn only within a larger human point of view; Nagel rejects that conceptual relativism but need not refute it.)
For experience the link to a point of view seems much closer. "After all, what would be left of what it was like to be a bat if one removed the viewpoint of the bat?" And if experience has no objective nature graspable from many points of view, how could a Martian studying my brain be observing physical processes which were my mental processes, as he might observe lightning, only from another angle? How could a human physiologist?
The reversal. Elsewhere reduction moves toward greater objectivity by reducing dependence on species-specific viewpoints: we describe things not by their impressions on our senses but by their general effects and properties detectable by other means. That works because our concepts, though first applied from a perceptual point of view, refer beyond themselves, so we can abandon the viewpoint and still think about the same things. Experience does not fit. "If the subjective character of experience is fully comprehensible only from one point of view, then any shift to greater objectivity... does not take us nearer to the real nature of the phenomenon: It takes us farther away from it."
The seeds of this objection lie in successful reductions themselves. Discovering that sound is really a wave phenomenon, we leave one viewpoint for another, "and the auditory, human or animal viewpoint that we leave behind remains unreduced". Radically different species can share an objective understanding of the same events only because their particular viewpoints are not part of the common reality they apprehend: "The reduction can succeed only if the species-specific viewpoint is omitted from what is to be reduced." That is right for the external world, but we cannot ignore the viewpoint permanently, "since it is the essence of the internal world, and not merely a point of view on it". Most of the neobehaviourism of recent philosophical psychology results from trying to substitute an objective concept of mind for the real thing, so that nothing irreducible is left over.
A footnote gives the point its most quoted image: the problem is not only that my experience of the Mona Lisa has a quality no trace of which is found by someone looking into my brain. "For even if he did observe there a tiny image of the Mona Lisa, he would have no reason to identify it with the experience."
The deceptive "is" and the caterpillar
If mental processes are physical processes, then there is something it is like, intrinsically, to undergo certain physical processes, and "what it is for such a thing to be the case remains a mystery". What moral follows? Not that physicalism is false: "Nothing is proved by the inadequacy of physicalist hypotheses that assume a faulty objective analysis of mind." Rather, "physicalism is a position we cannot understand because we do not at present have any conception of how it might be true."
The objector says the meaning is clear: mental states are states of the body; "What could be clearer than the words 'is' and 'are'?" Nagel answers that this apparent clarity is exactly what deceives. When told that X is Y we usually know how it is supposed to be true, from a theoretical background not conveyed by "is" alone: we know how both terms refer and roughly how the two referential paths might converge. When the terms are very disparate, we may have no idea how they could converge, and "an air of mysticism surrounds the identification".
Matter is energy, and the caterpillar in the safe. People are told young that all matter is really energy; they know what "is" means, yet most never grasp what makes the claim true. "At the present time the status of physicalism is similar to that which the hypothesis that matter is energy would have had if uttered by a pre-Socratic philosopher." Yet we may have evidence for something we cannot understand. Someone ignorant of metamorphosis locks a caterpillar in a sterile safe; weeks later the safe holds a butterfly. Knowing it was shut the whole time, he has reason to believe the butterfly is or was the caterpillar, "without having any idea in what sense this might be so" (perhaps a tiny winged parasite devoured it). What it shows: we may be in this position about physicalism, rationally believing an identity we cannot yet conceive.
The usual analogies from theoretical identification fail because, construing mental reference to physical events on the usual model, "we either get a reappearance of separate subjective events as the effects through which mental reference to physical events is secured, or else we get a false account of how mental terms refer (for example, a causal behaviorist one)". Donald Davidson (1970) argued that if mental events have physical causes and effects they must have physical descriptions, even though there is and could be no general psychophysical theory. Nagel accepts the conclusion for intentional events and thinks we have some reason to believe it for sensations too, without understanding how; but he confesses, "I do not understand the argument against psychophysical laws." Davidson's picture, physical events with irreducibly mental properties, may be correct, but "nothing of which we can now form a conception corresponds to it".
Kripke, necessity, and three kinds of imagination (Nagel's long footnote)
- Kripke's challenge. Saul Kripke (1972) argued that causal-behaviourist analyses fail because they treat "pain" as a merely contingent name of pains; the experience's "immediate phenomenological quality" is essential to it. Nagel agrees: if physicalism is true, the relation would be necessary, not that of cause to distinct effect; it would be necessarily true that a given physical state felt a certain way. He finds this "incomprehensible without further explanation".
- Why it would still seem contingent. We imagine things perceptually, sympathetically or symbolically. Perceptual imagination puts us in a state resembling perceiving the thing; sympathetic imagination puts us in a state resembling the thing itself (usable only for mental states).
- The illusion. To imagine pain without its brain state, we sympathetically imagine the pain while perceptually imagining the brain state's absence. Because the two kinds of imagination are independent, any experience seems separable from its brain state, "even if it is necessary".
- A bonus diagnosis. Solipsism results if one misreads sympathetic imagination as working like perceptual imagination: it then seems impossible to imagine any experience not one's own.
The proposal: an objective phenomenology
Little work has been done on the prior question, from which the brain can be omitted: whether experiences can have an objective character at all, whether it makes sense to ask what my experiences are really like, as opposed to how they appear to me. (A footnote: this is the heart of the problem of other minds, too; to understand how subjective experience could have an objective nature would be to understand the existence of other subjects.)
Nagel's closing proposal: set the brain aside and pursue "an objective phenomenology not dependent on empathy or the imagination", describing at least part of the subjective character of experiences "in a form comprehensible to beings incapable of having those experiences". It would be needed for bat sonar, but could begin with humans: develop concepts to explain to a person blind from birth what it is like to see. A blank wall would come eventually, but much more could be said, more precisely. Loose intermodal analogies ("Red is like the sound of a trumpet") are of little use, as anyone who has heard a trumpet and seen red knows; structural features of perception may be more accessible, and concepts other than first-person ones might even give us understanding of our own experience that "the very ease of description and lack of distance" of subjective concepts denies us. Such aspects might be better candidates for objective explanation of the familiar kind. Until more thought goes into subjective and objective, "we cannot even pose the mind-body problem without sidestepping it".
What "physical" must mean. Nagel's last footnote: he has not defined "physical", which cannot mean only what present physics describes. Some think mental phenomena may come to be recognised as physical in their own right. "But whatever else may be said of the physical, it has to be objective." He thinks it more likely that mental-physical relations will eventually be expressed in "a theory whose fundamental terms cannot be placed clearly in either category".
Critical lens. (1) Classification has moved. Nagel's "microchiroptera" is no longer a natural group: molecular phylogenies split bats into Yinpterochiroptera (fruit bats plus horseshoe bats and their echolocating relatives) and Yangochiroptera, so laryngeal echolocation either evolved twice or was lost in fruit bats; and some fruit bats of the genus Rousettus echolocate with tongue clicks. The philosophical point survives untouched. (2) Dennett's reply, developed in Consciousness Explained (1991) and by Kathleen Akins ("What Is It Like to Be Boring and Myopic?", 1993), is that third-person bat science tells us a great deal about what it is like to be a bat (what it can and cannot discriminate, what it attends to) and that Nagel's residue may be smaller than he assumes. (3) Nagel's notion of a point of view as a type that others can partly occupy sits uneasily with his claim that objectivity always abandons it; if a type can be shared by degrees, it may be specifiable by degrees. (4) The essay launched a long line: Frank Jackson's Mary the colour scientist (1982), David Chalmers's "hard problem" (1995), Nagel's own The View from Nowhere (1986) and his anti-reductionist Mind and Cosmos (2012).
What changed by 2026. The science of the bat has grown spectacularly. Nachum Ulanovsky's lab recorded three-dimensional place cells in the hippocampus of flying Egyptian fruit bats (Yartsev and Ulanovsky, Science, 2013); bats were shown to jam one another's sonar when competing for prey (Mexican free-tailed bats, Science, 2014). Nagel's footnote about blind human echolocators proved prescient: brain imaging of expert clickers Daniel Kish and Brian Bushway (Thaler, Arnott and Goodale, 2011) found echoes activating visual cortex, and Lore Thaler's Durham group showed that blind and sighted adults can learn click-based echolocation in ten weeks of training (2021, with brain changes reported in 2024). On consciousness itself, the 2023 adversarial collaboration between global neuronal workspace theory and integrated information theory (published in Nature in 2025: 256 participants, seven labs, fMRI, MEG and intracranial EEG) challenged key predictions of both; the same 2023 meeting saw Christof Koch concede his 1998 bet with Chalmers that the neural correlates of consciousness would be found within 25 years (he paid in wine). The New York Declaration on Animal Consciousness (April 2024) holds there is strong support for conscious experience in mammals and birds and "a realistic possibility" in all vertebrates and many invertebrates, insects included. And the bat question now has a machine twin: a 19-author report (Butlin, Long and colleagues, 2023) assessed AI systems against indicator properties drawn from neuroscientific theories, concluding that no current system is conscious and no obvious barrier prevents one. For anyone building AI products, Nagel's distinction is the useful one: you can specify, test and govern what a model does and represents; whether there is anything it is like to be it is a separate, still open question that self-reports cannot settle.
Reflections (D.R.H.)
Hofstadter heads his Reflections with two epigraphs side by side: Lewis Carroll's parody "Twinkle, twinkle, little bat, / How I wonder what you're at" (about 1865) and a refrain from a country-western ballad by Hank Cochran (about 1955), a jilted lover's plea about a rival: "Why can't he be you?" The pairing announces the theme: the irresistible, sometimes futile urge to map oneself onto another.
The mirror puzzle. "Why does a mirror reverse left and right, but not up and down?" Our first reaction is to project ourselves into "that person" by walking forward and spinning on our heels about a vertical axis, forgetting that the mirror-person's heart, appendix and probably language hemisphere are on the wrong side; microscopically its DNA coils the wrong way, and it "could no more mate with a real person than could a nosrep!" Alternatively, flip yourself head over heels, as over a waist-high bar: now the heart is on the right side but feet and head are swapped, so the mirror "reverses up and down". Which mapping feels natural depends on the body's external vertical symmetry; "mirrors intrinsically don't care which way you interpret what they do. And in fact, all they really reverse is back and front!" What it shows: self-projection is a choice of mapping, and an over-facile one misleads.
The urge to map, project, identify, empathise is "a basic human trait, practically irresistible", and it leads down strange paths; the ballad shows "the futility of taking such mapping too seriously". Since we cannot stop, Hofstadter indulges "an orgy of extravagant variations" on Nagel's title.
| Kind of "being" | Hofstadter's examples |
|---|---|
| ordinary human situations | working at McDonald's; being thirty-eight; being in London today; climbing Everest; an Olympic gymnastics champion |
| skills and genius | a good musician; improvising fugues; being J. S. Bach writing the last movement of the Italian Concerto; someone inconceivably more (or less) intelligent |
| beliefs and tastes | believing the earth is flat; hating chocolate; hearing your native language without understanding it |
| other selves | the opposite sex; your mirror image; Chopin's brother (he had none); the present King of France; Holden Caulfield; the subsystem of Salinger's brain that represents him; a dreamed person when the alarm rings |
| parts and wholes | a molecule; a microbe; a mosquito; an ant colony; a beehive; China; Detroit; General Motors; a concert audience; a married couple; a two-headed cow; one half of a split-brain person; the head or body of a guillotined person; the visual cortex of Picasso; a retinal cell or DNA molecule of Picasso |
| machines | a running AI program; an operating system; an operating system as it crashes |
| altered states | under general anaesthetic; being electrocuted; a Zen master in a satori-like state with no subject |
| things | a pebble; a wind chime; the Rock of Gibraltar; the Andromeda Galaxy; God |
The phrase is seductive: we readily accept that some things are "be-able things", "BATs" for short (bats, cows, people), and others are not (balls, steaks, galaxies, though a galaxy may contain innumerable BATs). "What is the criterion for BAT-itude?" Philosophers have many near-synonyms: soul, anima, the "in" word intentionality, consciousness, being a subject, having an inner life, experience, a point of view, perceptual aboutness, personhood, a self, free will, having a mind, being intelligent, plain thinking, and Searle's form versus content, syntactic versus semantic. All concern one emotional issue: whether it makes sense to project ourselves onto the object. "But is there really something to which they refer?" Since Nagel's target is a distillation of what is common to the experiences of all bats, not any one bat's experiences, "Searle might say Nagel is a 'dualist'".
The grammar of being someone else
The grammar of mapping sentences is surprisingly revealing. "What would it be like to be Indira Gandhi?" (conditional) forces you to project yourself into her skin; "What is it like to be Indira Gandhi?" (indicative) asks what it is like for her to be her. Described in whose terms? She might explain Indian political life by analogies to your experience. Protest ("Say it in your own terms!") and she might as well speak Hindi; then you would be like millions of Hindi speakers who have no idea what it is like for her to be her.
- A be-ee without a be-er. Nagel wants his "be" subjectless: not "what would it be like for me to be X?" but "what is it like, objectively, to be X?", "a living beast without a head". Back to the conditional: for me or for her? "Poor Indira, where does she go while I'm being her?" Reverse it: if she were me, where would I be? Traded places, or two souls collapsed into one?
- "If she were me". We say "me", not "I", treating "be" as if transitive; it is really symmetric, but language tilts us away from that.
- German surprise endings. From the German translation of a Stanisław Lem dialogue in which a molecule-for-molecule replica of a doomed man is to be built: "Ob die Kopie wirklich du bist..." (whether the copy really you are) and "Die Kopie wird behaupten, daß sie du ist" (the copy will claim that it you is). The verb at the end retroactively decides which word was the subject.
- Two questions that slip differently. "Is the copy really you?" slips toward "or is it someone else, or no one?"; "Are you really the copy?" slips toward "or are you somewhere else, or anywhere?" The book's title, The Mind's I, reads both as a possessive and as a reply to "Who am I?" and the ungrammatical "Who is me?"
- Dennett's interjection. In brackets: "If I were you, I'd mention how curious it would be to preface some advice with 'If you were me, I'd...' but if you were me, would I suggest that you mention it?"
The soulflame. All this shows "how suggestible we are": we fall "like a ton of bricks" for a flamelike soul that flickers on and off or passes between bodies like a flame between candles. If a candle is blown out and relit, is it the same flame? If it stays lit, is it the same from moment to moment? The Olympic torch is kept burning from Athens every four years, and "even the shortest break in the chain" would ruin the symbolism for people who knew. For those who did not, no harm done. "How on earth could it possibly matter? Yet emotionally it seems to."
We intuit that only "same-sized souls" can slip into one another. In Daniel Keyes's Flowers for Algernon a mentally disabled young man becomes a genius by a miracle treatment, then watches himself decline as it wears off; the real-life counterpart is people who witness their own senility or brain damage. Can they say better than an imaginative person "what it is like to have your soul slip out from under you"? Combine Keyes with Kafka's Metamorphosis (a man wakes as a giant beetle that thinks like a person) and imagine an insect rising to genius and sinking back: virtually inconceivable.
Impedance match. Borrowing electrical-engineering jargon, the impedance match of the minds involved may be "the main criterion for the plausibility of questions of the form Nagel poses". Which is easier to imagine being: the totally fictional Holden Caulfield, or some particular, actual bat? The fictional human, by far: "much easier, much realer". So Nagel's verb "be" behaves strangely; perhaps, as in the Turing-test dialogue, "be" is being stretched beyond its limits.
"There's something very fishy about this whole idea. How can something be something that it isn't?" Ask "What would it be like for that black spider to be that mosquito trapped in its web?", or "for my violin to be my guitar?", or "What would this sentence be like if it were a hippopotamus?" Like for whom? This is the sticking point: Nagel wants a description of human experience "in terms accessible to beings that could not imagine what it was like to be us", which sounds like a blatant contradiction, and that is his point. "He wants to know objectively what it is subjectively like."
The batter's helmet. Suppose Nagel donned a helmet with electrodes stimulating his brain into batlike experiences and so experienced "batitude". It would not satisfy him: it would be only what it would be like for Nagel to be a bat. What would? He is not sure anything would, and that is what worries him: that "having experience" lies beyond the objective. Try this: imagine a gradual version, a helmet that over months rewires you until you report only bat-like concerns and can no longer recall English. When it is complete, has Nagel learned what it is like to be a bat, or has a bat replaced Nagel, leaving no one who could carry the answer back?
Points of view as representational systems
The most objective-sounding synonym for BAT-itude is "having a point of view". Even a dogmatic sceptic about machine intelligence would grudgingly grant one to a program that represents facts about the world and its own relation to it: "Three minutes ago, the Teddy bear was thirty-five leagues due east of here." Such a here-centred, now-centred frame is a rudimentary egocentric point of view. Yet how can "now" and "here" be defined without reference to some "I"? Is the circle inevitable?
- "I" and "now". Imagine someone with normal capacities whose brain damage leaves him unable to convert short-term into long-term memory: his sense of existence would extend a few seconds either side of "now", with no chain of selves making one coherent person. A concussion erases the instants before it: knocked on the head now, you would keep no trace of reading these sentences. "Who, then, has been experiencing them?" Does an experience become part of you only once in long-term memory? Who dreamt all the dreams you do not remember?
- "I" and "here". "Not being in Paris right now, you know what it is like to be dead in Paris": no lights, no sounds, nothing; likewise Timbuctu. "You are dead everywhere, except for one small spot", and dead at all other moments. That small piece of space-time is defined by your body and the concept of "now"; words like "I" and "me" condense the associations of "here" and "now".
- The step forward. Programming a computer to use "I", "me" and "my" is common; behind them there need not be a sophisticated self-concept, but there may. Any physical representational system (as defined in the Reflections on "Prelude, Ant Fugue") embodies some point of view. If BATs are representational systems with rich categories and well-indexed memories of their worldlines, "we will have objectified at least some of subjectivity".
What really makes bats strange. Not their exotic senses: "Sensory modalities are surprisingly interchangeable." A grid of over a thousand stimulators, driven by a TV camera and pressed against the back, induces visual experiences in blind and sighted people. A sighted woman (Nancy Hechinger, Science 81, March 1981) felt "formless waves" until suddenly a black triangle appeared: "I felt vibrations on my back, but the triangle appeared in a square frame inside my head." People wearing inverting prism glasses adapt in two or three weeks; people learning a new language still experience the world of ideas much as before. So the bat's Weltanschauung differs because of its "severely limited set of categories" and its sense of what matters; bats cannot form notions like "the human Weltanschauung" and joke about them, "because they are too busy, always being in raw-survival mode."
Nagel's question is then about how one representational system maps onto another, as in the discussion of emulation after Searle. Asking a Sigma-5 "What is it like to be a DEC?" would be silly: an unprogrammed computer is no representational system, and emulation gives it no power to handle the concepts in the question. That would take a sophisticated AI program that could use "be" in all our ways, and the sensible question becomes "What is it like for you, as a self-understanding AI program, to emulate another such program?", which closely resembles "What is it like for one person to empathize strongly with another?"
Emulation: exact, step-by-step imitation of another system's lowest level; people lack the patience and accuracy to do it for long, and it yields no understanding by itself.
Empathy: people "subvert" their own symbol systems by adopting a global set of biases, a mental context that channels thought down unaccustomed pathways ("Hey, let me think about how it would feel to be a bat").
Empathy is not like taking LSD, which changes how neurons communicate unpredictably, depending on how it spreads, with no regard to what symbolises what, much as "a bullet shot through your brain" would. A bias set up through symbolic channels activates symbols that, while active, help trigger all the others; the brain can treat certain activations as stable contexts and activate others subordinately. "(Too bad we can't just 'think Einstein' when we want!)" Yet this cannot reach batitude. Each person's self-symbol (the "personal nucleus", the "gemma" of Lem's personetics) has become so large and idiosyncratic that it cannot, chameleonlike, assume another's identity: its history is "too wound up in that little 'knot'".
The isomorphic twin. Take a woman and an atom-by-atom replica with identical self-symbols. When she thinks about herself, is she also thinking about her replica? Many people fantasise about another person just like them somewhere in the heavens. When you think about yourself, are you also thinking about that person? Who is that person thinking about now? Are you that person? "If you had a choice, would you let that person be killed, or yourself?" What it shows: reference by self-symbol picks out a pattern, and patterns can have more than one instance, which our concept of self is not built to handle.
Language as the bridge
What Nagel did not acknowledge, Hofstadter says, is that "language (among other things) is a bridge that allows us to cross over into territory that is not ours." Bats have no idea what it is like to be another bat and do not wonder, because they lack "a universal currency for the exchange of ideas", which language, movies, music and gestures give us. Through such currency points of view become more modular, more transferable, less idiosyncratic.
Knowledge is "a curious blend of objective and subjective". Verbal knowledge can be shared to the extent words "mean the same thing" to different people, and we accept that the "hidden subterranean flavors" are not shared: each word is surrounded in each mind by "a rich and inimitable cluster of concepts" (see George Steiner's After Babel). Through meme-exchange media (Selection 10) we experience vicariously what it is like to be or do X. "It's never genuine, but then what is genuine knowledge of what it is like to be X?" We do not quite know what it was like to be ourselves ten years ago; only diaries tell us, and then only by projection. We often cannot see how we did what we did yesterday, and "it's not so clear just what it is like to be me, right now". Language gets us into the problem (by letting us see the question) and helps us out (by making experience shareable and more objective), but "it can't pull us all the way".
The Gödel coda. "In a sense, Gödel's Theorem is a mathematical analogue of the fact that I cannot understand what it is like not to like chocolate, or to be a bat, except by an infinite sequence of ever-more-accurate simulation processes that converge toward, but never reach, emulation." I am trapped inside myself and cannot see how other systems are; Gödel's Theorem follows from a consequence: I cannot see how other systems see me. Nagel's objectivity-subjectivity dilemmas are thus related to problems in mathematical logic and, as seen earlier, the foundations of physics (developed in the last chapter of Gödel, Escher, Bach).
Critical lens on the Reflections. (1) The claim that bats have "a highly reduced collection of conceptual and perceptual categories" and live "always in raw-survival mode" is an assumption, not a finding; modern field and lab work shows bats to be long-lived, highly social, vocally rich animals, and the claim is hard to square with Hofstadter's own view that categories, not senses, are what matter. (2) "Searle might say Nagel is a dualist" misreads both: Nagel explicitly declines to infer that physicalism is false, and Searle calls himself a biological naturalist. (3) The Gödel analogy is evocative, not rigorous: Gödel's theorem concerns provability in formal systems, and "seeing how other systems see me" is a metaphor for it, not a premise of it. (4) The TVSS episode supports Hofstadter's point that modalities are interchangeable at the level of information; whether the "felt" quality is also interchangeable is exactly what Nagel would dispute. (5) A dating slip in the epigraph: Hank Cochran's "Why Can't He Be You" is from 1962, when Patsy Cline recorded it, not "ca. 1955".
Practitioner angle. Hofstadter's split between emulation and empathy is a good lens for large language models. A model asked to "think like a credit officer" or "answer as a bat would" does something closer to Hofstadter's empathy: a prompt sets a context that biases every subsequent activation. That is why persona prompts change outputs so strongly, and why they are no evidence that the model has acquired the persona's knowledge or point of view. In enterprise practice, treat a persona as a steering device, then test the outputs against ground truth from the real role; and treat the model's statements about what it is like to be it with the same reserve Hofstadter applies to the soulflame.
Raymond Smullyan's dialogue in five scenes, announced in the book as coming from his Philosophical Fantasies, puts first-person authority on trial: a "cerebrescope" that reads every neuron tells a man that a red book does not seem red to him, and that he does not believe what he says he believes. The machine's inventor, an "experimental epistemologist", ends up unable to know his own beliefs or fears without consulting it, until it advises him to stop trusting it. The comedy asks whether anyone could ever be wrong about how things seem to him, and answers, through a psychiatrist with a toothache story, that one can. Dennett's Reflections bring the nightmare down to earth with two coffee tasters, Chase and Sanborn, who cannot tell whether the coffee or they themselves have changed.
Frank: a patient recently treated for a colour-vision disorder; cautious, analytical, determined to say nothing refutable.
The eye doctor: diagnoses Frank's "whole color mechanism" as out of kilter and cures him in a few weeks.
The experimental epistemologist: "There is only one in the world"; inventor of the brain-reading machine.
The cerebrescope: a machine that scans "every nerve cell" of a brain and reports its sensations, beliefs and intentions.
The doctor (psychiatrist): a friend of the epistemologist who explains how everything turned out.
Scenes 1 and 2: "It seems red to me"
Scene 1. The eye doctor holds up a book: "What color is it?" Frank: "Red." "Aha, just as I thought! Your whole color mechanism has gone out of kilter," but it is curable. Scene 2, a few weeks later, Frank, dismissed as cured, sits in the home laboratory of an experimental epistemologist who asks the same question. Now very cautious, Frank says only "It seems red to me." "Wrong!"
Frank checks: wrong that the book is red, or that it seems red? The latter, since he never said it is red. "But surely I know what color the book seems to me!" "Again you are wrong." "But who knows better than I?" "I do." Private mental states are "Metaphysical hogwash!" The epistemologist explains that past epistemologists used purely theoretical methods, degenerating into word games about whether a man can be wrong about his own beliefs; he settles such questions experimentally, not by telepathy but with a cerebrescope, now operating in the room, scanning every nerve cell in Frank's brain: "it is a simple objective truth that this book does not seem red to you."
The ladder of seemings and beliefs. (1) "It seems red to me": false, says the machine. (2) "It sure seems that it seems red to me!": false. (3) No matter how often Frank iterates "it seems like", he is wrong. (4) He switches to belief: "I believe that this book is red": false (after a glance at the dials). (5) "I believe that I believe that the book is red": also false, "and again, no matter how many times you iterate 'I believe,' all these belief sentences are false." (6) Frank laments his infinitely many erroneous beliefs. The epistemologist objects: he never said any of Frank's beliefs were erroneous. The saving twist: each statement's falsity saves Frank from a mistaken belief in the one below. If "I believe that I believe p" were true, Frank would believe "I believe p", which is false, so that belief would be an error; but it is false, so he holds no such erroneous belief. "So none of my beliefs were erroneous, only the statements were erroneous." "Exactly." What it shows: a sincere report about one's own mind is itself a fallible claim, and a ladder of higher-order reports does not climb out of fallibility.
Then the punch line. "Incidentally, what color is the book really?" "It is red." "What!" "Of course the book is red. What's the matter with you, don't you have eyes?" Frank protests that he said so all along; no, says the epistemologist, he said it seems red, that he believes it red, and so on: "Not once did you say that it is red." Had he simply answered "red", "this whole painful discussion would have been avoided".
Scene 3: should the machine be trusted?
Months later Frank returns with an inconsistency. The falsity of "I believe it is red", given that it is red, implies he has a false belief: "Since the sentence 'I believe it is red' is false, then I in fact believe it is not red." The epistemologist, "disappointed", corrects him: not believing p does not mean believing not-p. "I believe that either Jupiter has life or it doesn't. But I neither believe that it does, nor do I believe that it doesn't."
How the epistemologist comes apart
- "Should" as morality. Frank asks whether he should trust the machine; "that question involving the word 'should' is out of my domain", but a colleague who is "an excellent moralist" might help. Frank means evidence. Does the epistemologist have evidence the machine is reliable? "Well of course!" Explaining it would take several years of joint study, after which Frank "would certainly not have the slightest doubts".
- Physiology versus psychology. Frank grants the machine's accuracy but doubts it measures anything significant: only physiological states. "What else would you expect it to measure?" It measures those physiological states "that you call psychological states".
- Meaning tested by machine. Frank decides the dispute is purely semantic: they mean different things by "believe". Fortunately that is decidable experimentally: with two brain-reading machines, one on each brain, the epistemologist compares readings. "Nope, I'm sorry, but it turns out that we mean exactly the same thing."
- Self-consultation. Does he believe they mean the same? "Just a moment while I check with the machine. Yes, it turns out I do believe it." Why not just ask himself? "It doesn't work. Whenever I ask myself what I believe, I never get any answer!" How can he state a belief before he knows what it is? He has a belief about what he believes, but how to find out what that is?
- Crazy and frightened. Frank wonders if he is going crazy. "Let me consult the machine. Yes, it turns out that I may be going crazy." Does that frighten him? "Let me check! Yes, it turns out that it does frighten me."
- The fatal advice. Can his sanity be saved? Yes. How? He has not asked. He asks: "According to the machine the best thing I can do is to cease to trust the machine!" What will he do? "How do I know what I will do about it, I can't read the future?" His current intentions, the machine says, are "in complete conflict": if the machine is trustworthy he should distrust it; if he distrusts it, he distrusts that advice too. Frank leaves to fetch help. "Au revoir!"
Scenes 4 and 5: the cybernetic wobble
At a psychiatrist's office later that day, Frank describes his friend. "Oh, the experimental epistemologist. There is only one in the world. I know him well!" The prognosis is "actually quite good"; it is the sort of problem that has to work itself out. Frank, unnerved ("I can't even have confidence in how things appear to me"), asks for help himself; the doctor is overloaded for three months and then on a three-month vacation: come back in six months.
How the epistemologist recovered. For months he went round chanting "should I trust the machine, shouldn't I" (using "should" in Frank's empirical sense). (1) He formalised the problem: the axioms of first-order logic plus nonlogical axioms about the machine. The system was inconsistent: he proved he should trust the machine if and only if he should not. (2) In classical logic one contradiction proves every proposition, so he tried a weaker logic, "close to what is known as minimal logic", where a contradiction need not entail everything; it proved too weak to settle the question. (3) Bright idea: keep the inconsistent classical system and, for each proposition with a proof and a disproof, pick the more psychologically convincing proof. It worked beautifully: for every contradictory pair, all proofs of one were more convincing than any proof of the other. Anyone except the epistemologist could have used it. (4) To know which proof he really believed, he had to consult the machine, which begged the question. (5) Then fate: absorbed in theory, he became for the first time "experimentally negligent", and a few minor units of the machine blew out. It began giving blatant contradictions: that he believed a proposition, then days later that he did not, while also claiming he had not changed his belief. That made him distrust it totally. "Now he is fit as a fiddle."
Was the machine unreliable all along? "Oh, not at all": it was excellent until his carelessness broke it. And when Frank knew it? "Not so, Frank, and this brings us to your problem." The doctor has the whole conversation on tape. Frank asks whether anyone was ever mistaken about having or not having a sensation.
- The Christian Scientist's toothache. A man with a raging toothache, groaning and moaning, says "No, I do not feel pain; nobody feels pain, there is no such thing as pain, pain is only an illusion." Everyone knew he felt pain; he was not lying, "he was just simply mistaken".
- Timidity as evidence. Without any machine, if someone answered "I believe it is red" or "It seems red to me" rather than "It is red", the doctor would doubt that he believed it: "The very timidity of his response would be indicative of his doubts."
- The lingering disease. Why should Frank doubt? Because of his recent eye disease: his eyes perceived the correct colour, but the earlier experience made it impossible for him really to believe it was red. "So the machine was right!"
- Doubt as infection. Why did he doubt he believed it? Because he didn't believe it, and unconsciously knew it. Doubt about the senses "spreads like an infection to higher and higher levels of abstraction until finally the whole belief system becomes one doubting mass of insecurity." With the machine repaired, it would now concur that Frank believes the book red.
The moral of the machine. The machine was good; the epistemologist "misused it when he applied it to his own brain". "The combination of his brain and the machine each scrutinizing and influencing the behavior of the other led to serious problems in feedback. Finally the whole system went into a cybernetic wobble. Something was bound to give sooner or later. Fortunately, it was the machine." And how could the machine be trustworthy when it called itself untrustworthy? "The machine never claimed to be untrustworthy, it only claimed that the epistemologist would be better off not trusting it. And the machine was right."
Try this: the honest cerebrescope. Suppose the machine, instead of telling you what you believe, told you only what you are about to say you believe, and was always right. Now try to falsify it: as soon as it predicts "you will say red", say "blue". If it accounts for your reading its prediction, it must predict a statement that stays true after you read it, the fixed point Smullyan's logic puzzles love; if no such fixed point exists, no prediction shown to you can be correct. The thought experiment shows why the doctor blames the loop, not the machine: any device whose output feeds back into what it measures can be driven into wobble by the very person it reads.
Reflections (D.C.D.)
If the nightmare seems too outlandish, Dennett offers "a more realistic fable", not true but surely possible: two coffee tasters, Mr. Chase and Mr. Sanborn, who work with half a dozen others to keep the taste of Maxwell House constant year after year.
| Mr. Chase | Mr. Sanborn | |
|---|---|---|
| at hiring (about six years ago) | thought Maxwell House the best coffee in the world | the same |
| now | no longer likes it | no longer likes it |
| his diagnosis | "My tastes have changed": he has become a more sophisticated coffee drinker | "My tasters have changed": his taste buds have gone off, like orange juice after maple syrup |
| does the coffee taste the same to him? | yes, exactly as it did | no; if it did, he would still love it |
| confirming test | improved discrimination; deeper knowledge of coffee types | failed discrimination tests; abnormal taste buds (all that Szechuan food) |
They are alike in one way (both used to like it, now neither does) and claim to differ in another. When they confront each other, each may wonder: is Sanborn really in Chase's predicament, without noticing the gradual rise in his standards? Is Chase kidding himself that the coffee tastes the same?
The first sip of beer. "Terrible! How could anyone like that stuff?" Beer is "an acquired taste": one learns to enjoy that flavour. Which flavour? "The flavor of that first sip? No one could like that flavor!" Beer tastes different to the experienced drinker; one does not learn to like the first taste but comes to experience a different, likable one. "Had the first sip tasted that way, you would have liked beer wholeheartedly from the beginning!" What it shows: perhaps there is no separating the taste from the response to the taste. Then Chase and Sanborn might be exactly alike, merely describing themselves differently; but then each is wrong about something, since each sincerely denied being like the other. Could each have misdescribed his own case and described the other's?
Some philosophers, and other people, hold that a person cannot be wrong about such a matter: everyone is "the final and unimpeachable arbiter of how it is with him". If Chase and Sanborn speak sincerely, make no slips and know their words' meanings, they must be right. But we can imagine tests that would tend to confirm their tales. If tests can support their authority, failing them must undermine it: if Chase passed Sanborn's tests and Sanborn passed Chase's, each would have doubt cast on his account, "if such tests have any bearing at all on the issue".
The price you pay for the possibility of confirming your authority is the outside chance of being discredited.
"'I know what I like,' we are all prepared to insist, 'and I know what it's like to be me!'" Probably you do, at least about some matters, but that is "something to be checked in performance. Maybe, just maybe, you'll discover that you really don't know as much as you thought you did about what it is like to be you."
Frank (and the incorrigibility tradition): nobody knows better than I how things seem to me; first-person reports of seemings and sensations cannot be mistaken.
The doctor and Dennett: first-person reports are evidence, usually good, but they are claims that can be tested, confirmed and therefore also discredited; timid reports, ideology (the Christian Scientist) and gradual change can all make them wrong.
Critical lens. (1) Logic. The epistemologist's flight to "a logic close to minimal logic" is roughly right but not exact: minimal logic (Johansson, 1937) blocks deriving every proposition from a contradiction yet still lets one derive the negation of every proposition, so it does not fully contain the damage; the logics built for living with contradictions are the paraconsistent ones. (2) The doctor's verdicts rest on the tape and on hindsight; he never shows that the machine's readings track belief rather than some neighbouring state such as hesitation, which is exactly Frank's objection in Scene 3, left unanswered. (3) Smullyan's piece and Dennett's fable cut against Nagel in the previous selection: if one can be wrong about what it is like to be oneself, the subject's point of view is not the last word Nagel needs it to be. Dennett developed the coffee tasters at length in "Quining Qualia" (1988), arguing that the idea of a pure, response-free "taste" (a quale) is incoherent. (4) The Philosophical Fantasies announced for 1982 appeared as 5000 B.C. And Other Philosophical Fantasies (St. Martin's Press, 1983).
What changed by 2026. The cerebrescope is no longer pure fantasy, though it remains far off. In 2023 Jerry Tang, Alexander Huth and colleagues (Nature Neuroscience) decoded the gist of heard and imagined stories from fMRI recordings of three volunteers; the decoder worked only for the person it was trained on, needed their cooperation, and failed when they deliberately thought of something else. Psychology had already shown how readily people misreport their own reasons and choices, as in "choice blindness" experiments (Johansson and Hall, Science, 2005), where people defend choices they did not make. And the epistemologist's predicament has a machine version: in October 2025 Anthropic's interpretability team reported "emergent introspective awareness" in Claude models, using concept injection (planting a known activation pattern and asking the model whether it notices anything); the best model detected and named the injected concept only about 20% of the time, and the authors stressed that such introspection is unreliable and limited. For anyone deploying AI, this is Smullyan's lesson in engineering form: a model's report about its own reasoning is a claim to be checked against instruments (interpretability tools, evaluations, logs), and the instruments in turn must not be trusted blindly or allowed to feed back into the behaviour they measure without controls.
Hofstadter's Achilles-and-Tortoise dialogue, set beside an octagonal pond in the Jardin du Luxembourg, imagines Einstein's brain transcribed neuron by neuron into a book of a hundred billion pages, with tables for turning speech into neural input and neural output into speech. Worked by hand, page by page, the book answers Achilles: "Oh, hello. Did you come to visit me? Have I died?" From there the Tortoise drives Achilles through copies, clerks, slowed clocks, broken page-turners and scattered pages until he concludes that "I" may name no Thing at all, only a pattern that any copy of the book would also call "the real me". It is the Chinese Room with its knobs turned the other way (Selection 22), and Dennett's Reflections use it to defend thought experiments as a serious instrument of science, provided we watch which simplifications are doing the work.
The Tortoise: strolls "through the centuries" for his spleen; claims to enjoy music by looking at playing-records; the dialogue's trap-setter.
Achilles: fleet-footed, philosophical, waiting for a footrace partner who has not shown up; the dialogue's test subject.
Albert Einstein: physicist (1879–1955), whose brain, the Tortoise says, was catalogued "down to the cellular level" after his death.
The alligator: the Tortoise's friend, who "hears" records by slapping them against his shell-less belly.
A-kill-ease: the Tortoise's name for a hypothetical neural clerk who turns the pages ("no relation to present company").
The playing-record prelude
The two meet by chance on "a pleasant fall day" in Paris, where children sail toy boats (now even radio-controlled ones, "but this is beside the point"). The Tortoise praises Achilles as the exception among humans that proves the rule, and promises no logical traps: he will speak only of brains and minds, which "of course... Have nothing, nothing whatsoever, to do with logic!" He eases in with an analogy.
- Silent music. Music is stored on records? The Tortoise thought music was something to listen to; records must make "awfully silent music". Achilles explains turntable, needle, loudspeaker. Why not just use the loudspeaker? Because the music is in the record. But the record is there "all at once", and music comes "a bit at a time".
- Looking instead of listening. Why not hang the record on the wall and take in all its pleasures at a glance, with every part in its proper place? Records all look alike to Achilles ("much as Tortoises do"); to the Tortoise they differ "as two pieces of music, one by Bach and the other by Beethoven", and by Achilles's own account they must, since the surfaces contain all the music. Or paste the score on the wall and absorb a Beethoven symphony "in ten seconds or less" each morning. Beethoven is the Tortoise's second favourite composer; "the sculpted forms in some of his playing-records are so exquisite".
- The tables turned. How would Achilles like to be told he had never understood a Leonardo painting, which should be listened to, "sixty-two minutes long, in eight movements", with long passages of ringing bells? And the alligator slaps a record against his belly to absorb its patterns all at once: to him Bach and Beethoven differ as a waffle iron and a velvet pad slapped against your bare back.
- Achilles concedes. The Tortoise's point of view "must be just as valid as mine, and if I did not admit it, I should be an auditory chauvinist pig." The Tortoise admits he knows the usual way of listening; the comparison is the analogy he needs.
| In the record analogy | In the brain-book |
|---|---|
| grooves on a disc, there "all at once" | numbers on a hundred billion pages, there "all at once" |
| needle, turntable, amplifier | the preface's rules for updating pages, worked round by round |
| music heard "a bit at a time" | a conversation, thoughts unfolding "a bit at a time" |
| the Tortoise admiring a record on the wall | the "all-at-once" view of a mind's whole trajectory |
| a record sailing into space after Earth's destruction | the book lying unused on a shelf |
| warped or broken records on the friend's wall | pages scattered by the wind |
The book
The Tortoise sets up a hypothetical conversation: he arrives with a very big book, "a schematic description of Albert Einstein's brain, down to the cellular level, made by some painstaking and slightly crazy neurologist after Einstein died. You know he bequeathed his brain to science, don't you?" Then a twist of form: Achilles takes over and voices both sides of the imagined exchange, anticipating everything the Tortoise would say, including the Tortoise's deferral of the mind-versus-brain question until "we meet by chance in the Jardin du Luxembourg someday", the very situation they are in ("How coincidental! It surely is by pure chance.").
The neuroscience inside the dialogue (as Achilles imagines the Tortoise explaining it)
- Neurons and firing. A brain is neurons linked by axons. A neuron fires when a minuscule current "regulated by the resistance of the axon" passes into an adjoining neuron and, with other signals, reaches its threshold; otherwise it does not fire.
- Thoughts as squads. A thought occurs when connected neurons fire in succession, not a single domino chain but a broad or narrow squad of firing neurons, with stray side-chains petering out, a dynamic chain whose course follows "the path of least resistance".
- Memory as a wake. Like a toy boat crossing one of the Luxembourg ponds, the "hot spot" leaves a wake: neurons that just fired keep up internal activity for a few seconds, making a permanent change in their numbers (thresholds, resistances).
- Self-modifying numbers. The way those numbers change is itself encodable. Achilles's own "astute remark": the "structure-altering numbers" describe how other numbers on the page change, "but also how they themselves are to change".
Each page of the book thus holds four kinds of entry: (1) a threshold value; (2) the page numbers of linked neurons; (3) the resistances of the linking axons; (4) the numbers telling how the neuron's wakelike "reverberations" will alter any number on the page. The Tortoise then adds two chapters "since it is hypothetical anyway". Auditory conversion tables say exactly how each auditory neuron is excited by a tone of given pitch and loudness; speech tables say how the firing of mouth-directing and vocal-cord-directing neurons changes mouth shape and vocal-cord tension. Achilles: "If it's already got a hundred billion pages or so, a few more can't hurt."
What good would such documentation do? "Why, it could do no good for anyone, except conceivably some starving neurologist." The Tortoise proposed it "only to tickle my fancy", though it may serve as a lesson to novices, of whom Achilles is "doubtless" one. Achilles wonders what old Einstein would think of it all. "Why, given the book, you could find out."
Talking to the book
The thought experiment, step by step. (1) Introduce yourself. To whom, the book? "Yes, it's Einstein, isn't it?" "No, Einstein was a person, not a book." (2) Recall the record: you must experience the book "a bit at a time", not by reading page 1 onward but according to the directions in the preface. (3) Say "Hullo, Dr. Einstein. My name is Achilles." Take the first tone, use the auditory tables to see which cells fire and how, then go page by page effecting every change: "round one". (4) Round two: go to the pages where the firing neurons' axons lead and modify them as their structure-changing numbers direct; "lo and behold, we're off on a merry loop around the brain". (5) Keep time: each page records how long its neuron took to fire in life (thousandths of a second); sum the firing times over rounds, and when they reach the length of the first tone, feed in the second. (6) The whole utterance takes "millennia", but "let's just say five seconds". (7) Afterwards the cascade continues without input, "a strange and elaborate dance" between pages, until certain speech neurons fire; the speech tables give mouth shape and vocal-cord tension, from which the words follow. (8) The book says: "Oh, hello. Did you come to visit me? Have I died?" The intuition pumped: someone is answering. Achilles's verdict: "Oh, just some silly book. It's not Einstein, of course! You can't trap me into saying that!"
The Tortoise suggests a whole conversation: ask how he feels, say how glad you are to meet him, proceed "just as if he were the 'real' Einstein". How would he react on being told he is not the real Einstein? Achilles objects to the pronoun: "There is no 'he.' I wish you'd quit using that pronoun." The Tortoise's defence: he is imagining what Achilles would have said had he met Einstein "in his hospital bed in Princeton"; the book reflects Einstein's brain on the last day of his life, when he regarded himself as a person, so it should be addressed as one. Achilles agrees, then slips: told he has died and been catalogued, "He'd probably be most astonished!" "Who? I thought there was no 'he'!"
- Where do the thoughts come from? "From the book. You know that very well." Then how can he say how he feels? "A book doesn't feel any way. A book just is. It's like a chair."
- Book plus process. "This isn't just a book, it's a book plus a whole process. How does a book plus a process feel?" "How should I know? But you can ask it."
- Phantom legs. It will say "I'm feeling very weak and my legs ache", but it has no legs. Its neural structure, the Tortoise replies, incorporates a strong memory of legs and leg-aching; tell it in detail that it is now a book-plus-process, and it might stop attending to legs it lacks and "concentrate on what it does have, such as the ability to communicate with you, Achilles, and to think".
- Sadness and time. Achilles finds it "frightfully sad": messages take so long that he would be an old man after a few exchanges. "Well, you could be turned into a catalogue too." "Ugh! And not have any legs left, to run footraces?"
Copies, clerks and the slowing clock
The Tortoise sweetens the offer: catalogued, Achilles could keep talking with Einstein, and with copies of his book sent round, conduct several conversations at once (Homer, Zeno, Lewis Carroll, if catalogued too). How would he keep track of them all "in my head"? "Your head? You would have no head, remember." Each copy is independent; asked, the Einstein book in Achilles's possession would say he did not feel he was in more than one discussion, "So that's how you'd feel too". "I? Which one would be me?" "Any of them; all of them; or perhaps, none of them." The Tortoise could even introduce the copies to each other, and "there just might ensue a teeny scrap over which one was the real one". Achilles: "Oh, this is a diabolical scheme to squeeze the juice out of the human soul. I'm losing a clear sight of who 'I' is. Is 'I' a person? A process? A structure in my brain? Or is 'I' some uncapturable essence that feels what goes on in my brain?"
Is Einstein alive? The Tortoise's series of variations. (1) Data alone. If the book is never used, is he alive? Achilles: "no". What made him live on was "the process, above and beyond the mere data book": his neurons were firing, figuratively and slowly, "but that's of no consequence". (2) Ever slower. Round one takes ten seconds, round two a hundred, round three a thousand: soon a syllable a century, then one neuron firing every trillion years. Achilles: if he too were catalogued and processed equally slowly, rates would match; "Einstein and I are serenely oblivious to the fast-changing world outside our flipping pages." (3) The clerk leaves. The faithful neural clerk, A-kill-ease, slips off for "a little nip" and forgets to return. "Foul play! Double homicide! Or do I mean bibliocide?" "Is it all that bad? Both of you are still there, 'all at once.'" "What's the fun of life if we're not being processed?" (4) The clerk is you. What would it be like if your brain were encoded in a book and you were minding your own brain-book? "I suppose I would have to ask my own book. Or no, wait a minute. My book would have to ask me!" (5) A machine page-turner removes human unreliability and the "strange twisty loop". Suppose it breaks; unless told, you would not even know it existed. (6) Give the book eyes: optical conversion tables fed by a television camera. Now you would "see" the page-turner's loose part about to slip, and be frightened; without the camera, "An idyllic existence! Up until the very end, not a worry!"
When the machine breaks, Achilles says, he would be "a lifeless, motionless heap of number-covered sheets". But old A-kill-ease might return and resume: "So I'd be resuscitated." What makes him deader when the machine breaks than when the clerk leaves for minutes or years, for backgammon, a trip round the world, "or to go get his brain copied into a book"? Achilles: the clerk intends to return. Then aliveness depends on other beings' intentions? "That would be a very silly way of defining 'aliveness' and 'deadness'", like calling a light bulb "dead" because its owner will never switch it on again. What counts is that the book stays intact.
The record in space. Achilles offers the image himself: the Earth is destroyed, but one record of Bach's music sails out into the void. Does the music still exist? It would be silly to make that depend on whether some humanlike creature ever finds and plays it. "To you, Mr. T, the music exists as the record itself. Similarly... If the book merely sits there, all at once, I'm still there. But if that book is destroyed, I'm gone." Achilles now holds that the integrity of the brain structure is "all important", the opposite of his earlier insistence on process.
The Tortoise attacks integrity in turn. Suppose someone absconds with the preface, the instructions for using the book. "My goose would be cooked." Again it depends on the thief's intentions, unless "the capricious wind" wafts the pages away, with no intention involved. The book and instructions lie on a dusty shelf of a used book store; a chap exclaims "Egads! An Achilles-book!" and buys it. How close must book and instructions stay: same binding, bag, house, within a mile? At what point would the book lose structural integrity? The Tortoise appreciates warped records as much as flat ones, and has a friend whose walls are plastered with broken Bach: "fragmented fugues, crushed canons, ruptured ricercari". "Structural integrity is in the eye of the beholder, my friend." If the pages could be reunited, Achilles thinks there is hope. "Reunited in whose eyes?" Once dead, Achilles the beholder exists only in book form, if at all. Who will judge?
Achilles is lost: "I have totally lost track of the progress of that poor soul inside the book." The Tortoise pounces: is he still clinging to the idea that "you" are somehow inside the book? Achilles was reluctant until the book seemed to express Einstein's emotions; perhaps he should return to the familiar view that "the only real 'I' is right here, inside my very own living, organic brain". "You mean, the old, familiar 'ghost-inside-the-machine' theory, is that it? What is it, inside there, that this 'you' is?" "It's whatever feels all these emotions that I express."
Bit-at-a-time and all-at-once
Perhaps, says the Tortoise, feeling is "the sheer physical event of having a shower of electrochemical activity come flying through" a neural pathway, and "feeling" is our word for such events. That sounds wrong to Achilles, since the book uses the word "feeling" too, yet feels no electrochemical surge, only its numbers changing; perhaps feeling is the existence of any kind of neural activity, simulated or otherwise. That stresses the unfolding of feeling a bit at a time, the Tortoise objects: why could feelings not be there "all at once", like records and paintings?
The thrown grapefruit. Achilles spots a difference: a record never changes, while a mind interacting with the world is modified in ways not inherent in its structure. Granted, says the Tortoise, but during introspection, without interference, the changes are inherent: evolving "a bit at a time", it is "inherently there 'all at once'". The entire path of a thrown grapefruit is fixed once it leaves the hand: watching it fly is the "bit-at-a-time" picture; knowing its initial position and velocity is the "all-at-once" picture (assuming no "passing storks"). A brain, or brain catalogue, has the same dual nature while isolated. Humans prefer the first picture, even for a grapefruit rather than a parabola, but recognising the second "was a great step for the human mind", since it meant recognising that regularities exist in nature that guide events in predictable channels.
Achilles: feeling exists in the bit-at-a-time picture, because that is how he feels his own feelings. "But does it also exist in the 'all-at-once' picture? Are there 'feelings' in a motionless book?" "Is there music in a motionless playing-record?" He no longer knows. He still wants to learn whether "I" am in the Achilles book; the Tortoise wants to learn "if 'you' are anywhere at all".
The labyrinth and the dance of the soul
Back to the comfortable bit-at-a-time picture. The hot spot zigzags along the path of least resistance, and "you, Achilles, or what you refer to as 'I,' have no control over which path is the one of least resistance." His subconscious, then? Perhaps "subconscious" is a good name for neural structure, which at each moment fixes the path; "This swirling electrochemical activity constitutes the mental and emotional life of Achilles." Achilles asks the Tortoise to sing it ("let the verbs have their fling!"), and he does.
The Tortoise's song of the labyrinth
- Rooms and doors. The brain is a labyrinth of rooms (each a complex of a few or a few dozen neurons, perhaps more), each with many doors to other rooms.
- Labelled rooms. Some rooms are labelled: complexes made mostly of speech neurons. When the hot spot enters one, throat and mouth contract and you say a word.
- A foreordained path. The flash loops "in shapes stranger than the dash of a gnat-hungry swallow", every turn foreordained by neural structure until sensory input makes it veer.
- Silent thought. In a labelled room with its lights turned low, the word is not spoken: a thought occurs silently.
- Oil and water. At each door the hot spot leaves a drop of oil on the hinge to loosen it or a drop of water to corrode it; some doors rust shut, others are oiled so often they nearly open by themselves. Thus "the 'I' of now leaves messages and memories for the 'I' of a time to come."
"This neural dance is the dance of the soul; and the sole choreographer of the soul is physical law." Achilles protests that this turns things backward: "I" becomes at best a by-product of an organism governed by natural law, at worst "an artificial concept produced by my distorted perspective". What does it mean to know anything? When his brain knows something, a path snakes through rooms, many labelled, and he need not think about the flash for it to work: "It seems as if I function quite well without me!"
Achilles: "my 'self' is not in control of myself"; I can think only "what was set up for me to think, by an earlier version of me"; I forget names, cannot concentrate, grow nervous despite myself.
The Tortoise: you cannot force the flash off the path of least resistance, but the Achilles of one moment directly affects what becomes the path of least resistance in the next; whether that counts as control depends on whether you identify with your former selves; if you do, "you" control what you are today; if you think of yourself as existing only in the present, natural law, not an independent soul, is in charge.
The self-knowledge paradox. Achilles proposes total self-knowledge: learn his neural structure so well he can predict his flash's path in advance. The Tortoise: "you have innocently thrown yourself into the wildest of paradox". Read the Achilles book: a hundred billion pages; he might fall asleep listening to himself read, or die before finishing. Suppose he is fast. "So now you'd know all about Achilles before he read the Achilles book! But you are quite ignorant about the Achilles who exists now!" Reading changes the reader and makes the book obsolete. A bigger brain would not help, since it would make him "all the more complex yet". Achilles's conclusion: "My mind simply can't understand all of itself. All I can know is the outline, the basic idea... My brain and 'I' are not the same!" What it shows: a system cannot hold a complete, current model of itself, a theme Hofstadter ties to Gödel in the Reflections on Nagel.
Mind, beauty and the sound "I"
Do thoughts occur in the mind or in the brain, the question deferred at the start? Achilles hardly knows what "mind" means except as a poetic name for the brain or its activities. It reminds him of "beauty": not locatable in space, yet not "hovering in an ethereal otherworld", more "a structural feature of a complex entity". Where lies the beauty of an étude by Scriabin: in the sounds, the printed notes, the ear, mind or brain of the beholder? Perhaps "beauty" is just a sound we utter when the flash passes through a certain labelled room; because it is a noun we think it names a Thing, but maybe it denotes no Thing at all.
The Tortoise goes further: the same holds for "beauty", "truth", "mind" and "self". Each is a sound we are caused to utter; we can hardly help believing an Entity corresponds to it. "The benefit that one derives from using a sound imbues it with a proportionate amount of what we call 'meaning'"; whether it denotes any Thing, "how would we ever know that?" Achilles calls this solipsistic; the Tortoise never denied that Things exist, and treats the assumption that some sounds stand for Existing Entities as pragmatic, justified by its usefulness.
Achilles's last insight. Another part of him wants to shout "'The real me' is here now." But the mechanism that makes him say "Spades are trump" is much like the one that makes him, or the Achilles book, say "'The real me' is here now"; the book would undoubtedly say it. Maybe "I", like "beauty", "denotes no Thing at all, but is just a useful sound that we on occasion feel compelled to pronounce because our neural structures are set up that way." That explains his confusion over the copies: each copy is built to proclaim "I am the real me... anybody else who claims to be Achilles is a fraud." The mere fact that it says so does not mean it has real feelings, "and perhaps even more to the point, the mere fact that I, Achilles, utter such things, doesn't really mean I am feeling anything (whatever that would mean!)."
Utterances about feeling remain useful in practice, and Achilles will not shun them or the word "I", but he will no longer imbue it with "such 'soulful' meaning" as he had, "rather instinctively, and, I have to say, dogmatically". For once they agree. Dusk approaches, when the Tortoise feels most energetic: how about a footrace back to the fifth century B.C.? Achilles offers a head start of "oh, three centuries", since he is so fleet. "Last one to Zeno's house is a monkey's uncle!", a wink at Zeno's paradox, in which Achilles never catches the Tortoise.
Critical lens on the dialogue. (1) Dated neuroscience. Neurons do not pass a current "regulated by the resistance of the axon" into their neighbours: most transmission is chemical, across synapses, and the adjustable strengths that store memory sit mainly at synapses, shaped also by neuromodulators, glia and dendritic computation; a faithful book would need numbers for perhaps a hundred trillion synapses, not a few per page. The human brain has about 86 billion neurons, so "a hundred billion pages" is the right order of magnitude. (2) A historical error. "You know he bequeathed his brain to science": he did not. Thomas Harvey, the pathologist at Princeton Hospital, removed it during the autopsy after Einstein's death on 18 April 1955, apparently without prior permission; Einstein's son Hans Albert endorsed it afterwards on condition that it be used only for serious research. (3) Determinism is not experience. The Tortoise's "all-at-once" picture assumes the isolated brain is deterministic and computable; even granting that, knowing a trajectory in advance is not the same as there being a feeling "in" the static description, which is precisely the open question Achilles asks and the dialogue leaves open. (4) Hofstadter's later position. Achilles's conclusion that "I" may denote no Thing is more deflationary than Hofstadter's mature view in I Am a Strange Loop (2007), where "I" is a real, causally potent pattern, even if a "hallucination hallucinated by a hallucination".
Reflections (D.C.D.)
Dennett anticipates a reader's complaint: "these fantasies have been fun, but they can't really tell us anything. They're just so much science fiction." It rests on "a familiar but impoverished vision" of science as precise formulae, careful experiments and vast catalogues, a data-gathering enterprise with imagination tightly reined in; some scientists share it and distrust their playful colleagues, as perhaps some orchestra players see their work as "precise noise-making... Under conditions of militaristic discipline". If so, "think what they are missing."
Science as a playground of the imagination. Science is populated by characters with wonderful names (messenger RNA, black holes, quarks) performing amazing deeds: sub-atomic whirling dervishes that can be in several places at once; molecular hoop-snakes biting their tails; self-copying spiral staircases bearing coded instructions; miniature keys searching for locks "in a trillion synaptic gulfs". So why not brain-book immortality, dream-writing machines, symbols that understand themselves, homunculi following orders like the sorcerer's broom? Some of the book's wildest ideas were proposed in earnest by eminent scientists: Wheeler's solitary electron weaving the universe, Everett's many worlds, Dawkins's survival machines for genes. Understanding the mind will probably need ideas "at least as outrageous, at first" as Copernicus's moving Earth or Einstein's curved space. "Science advances haltingly, bumping against the boundaries of the unthinkable", and the speculative frontier of thought experiment is where those boundaries get adjusted.
Galileo's falling bodies. Thought experiments can be systematic and rigorous. Hypothesis: heavier objects fall faster. Take a heavy object A and a light object B and tie them together. (1) By hypothesis B falls slower, so it drags on A: A tied to B falls slower than A alone. (2) But A tied to B is a new object C, heavier than A, so by hypothesis it falls faster than A alone. (3) It cannot do both, so the hypothesis is false. What it shows: a pure thought experiment can refute a theory with nothing but its own premises, a reductio ad absurdum.
Other thought experiments are meant only to illustrate and enliven difficult ideas, and the boundaries between "proof, persuasion, and pedagogy" cannot always be drawn. The book's thought experiments explore the hypothesis that materialism is true: the self is not a non-physical thing in miraculous interaction with the brain, but "a natural and explainable product of the brain's organization and operation".
| Thought experiment | Its role, as Dennett classifies it | What it glosses over (the scale problem) |
|---|---|---|
| "The Story of a Brain" (Zuboff) | a reductio, like Galileo's, of materialism as "the neural theory of experience" | the staggering complexity of the devices the brain parts would need |
| "Prelude, Ant Fugue" (Hofstadter) | supports materialism: helps readers past traditional obstacles | (not discussed) |
| "Where Am I?" (Dennett) | supports materialism: a substitute for the "indivisible pearl of mind-stuff" | radio links for hundreds of thousands of nerves; a synchronous computer duplicate of a brain treated as mere technology |
| "Minds, Brains, and Programs" (Searle) | refutes one version of materialism (roughly the editors'), leaving underdescribed alternatives untouched | a hand simulation so huge no one could do one exchange in a lifetime, imagined at conversational speed |
| "A Conversation with Einstein's Brain" (Hofstadter) | supports materialism | faces the scale problem directly: a hundred billion pages flipped fast enough to extract "a few conversational gems" |
Dennett's own rule for intuition pumps. "Each setting of the dials on our intuition pump yields a slightly different narrative, with different problems receding into the background and different morals drawn." Which versions to trust is settled by examining "which features of the narrative are doing the work". "If the oversimplifications are the source of the intuitions, rather than just devices for suppressing irrelevant complications, we should mistrust the conclusions we are invited to draw." These are matters of delicate judgment, so a general suspicion of such exercises is "quite justified". Notice that the rule cuts against the editors' own pieces as much as against Searle's: the brain-book's "five seconds" for millennia of page-turning is exactly such a simplification, and Dennett says so.
In the end we need hard science's experiments, deductions and mathematics "to keep the speculations honest"; they are also powerful engines of discovery. Still, the storytelling side of science is neither peripheral nor mere pedagogy: it is "the very point of it all". As "a fine physics teacher once said", science properly done is one of the humanities. Its point is to help us understand what we are and how we got here, and for that we need the great stories: the Big Bang; the Darwinian epic; and "now the story we are just beginning to learn how to tell: the amazing adventure of the primate autobiographers who finally taught themselves how to tell the story of the amazing adventure of the primate autobiographers."
Science properly done is one of the humanities.
What changed by 2026. The Tortoise's book is being written, one small brain at a time. In October 2024 the FlyWire consortium published in Nature the complete wiring diagram of an adult fruit fly brain: nearly 140,000 neurons and more than 50 million synapses. A companion paper (Shiu and colleagues) turned it into a simple leaky integrate-and-fire simulation, an electronic A-kill-ease, which predicted circuits for feeding and grooming; about 91% of 164 testable predictions matched experiments. In April 2025 the MICrONS project mapped one cubic millimetre of mouse visual cortex: more than 200,000 cells, over 524 million synapses, about four kilometres of axon, 1.6 petabytes of data. The human brain is roughly a million times larger than that cube, and wiring alone omits much of what the book's "structure-altering numbers" stand for, which is why whole-brain emulation remains a research programme, not a product. Einstein's real brain, cut by Harvey into some 240 blocks, has yielded a string of small-sample studies (extra glial cells, an unusual parietal region, a distinctive frontal ridge) that critics find statistically weak. Meanwhile a cheaper "Einstein book" exists: large language models will happily role-play Einstein or a dead relative from their writings, conversing fluently with none of the brain's structure. The dialogue gives the right test for both: ask which features of the system are doing the work. For a bank weighing a "digital twin" of an expert or a customer, the same question applies: a model trained on someone's outputs is a portrait of their behaviour, not a copy of their mind, and should be validated and governed as such. Hofstadter himself said in 2023 that he felt "a certain kind of terror" at how fast such systems were advancing.
Robert Nozick's short piece, first published in Ploughshares in Fall 1980, ends the anthology with a narrator who announces: "I am a fictional character. However, you would be in error to smile smugly, feeling ontologically superior. For you are a fictional character too." From that premise Nozick reruns the old questions of theology and metaphysics as questions of literary criticism: creation by words, the problem of evil, Descartes's cogito, free will, an afterlife, and whether there is a top floor to the hierarchy of authors. A "Note From the Author" then refuses to settle whether the piece was fiction or a philosophical essay, and asks what would change either way, and which answer you hope for. The editors add no Reflections: the book closes on Nozick's question.
The narrator's predicament
"All my readers are [fictional] except one who is, properly, not reader but author." The narrator insists this is not a work of fiction, "no more so than any other work you've ever read": not a modernist piece that self-consciously declares itself fiction, nor a trickier one that denies it. Readers know how to frame such works so that nothing the first-person voice says, "even in an afterword or in something headed 'author's note'", counts as the author speaking seriously in his own person. Hence the narrator's "more severe" problem: telling you that this is non-fiction within the fictional world we inhabit, though "encased as it is in a work of fiction, it too can only be a fiction".
The thought experiment, step by step. (1) Think of our world as a novel in which you are a character. (2) Can we tell what our author is like? Perhaps, if he expresses himself in the work, though every inference we draw "will be written by him", and if he writes that we find it valid, "who are we to argue?" (3) Test every great question of philosophy by asking how it looks from inside a fiction. The intuition pumped: nothing we could observe or prove distinguishes a real world from a sufficiently detailed written one, since the observations and proofs would be written too. What Nozick does with it: he finds the frame strangely consoling, and finally turns it on its own author.
Theology as literary criticism
| Old question | Nozick's literary version |
|---|---|
| Creation | one scripture says the author created things "merely by speaking", "Let there be..."; the only thing mere speaking can create is "a story, a play, an epic poem, a fiction"; we live in words: "uni-verse" |
| The problem of evil | is an author callous who puts characters through hardships? Not if they do not suffer really; but Hamlet's father really was killed (or was he hiding to watch Hamlet?), Lear really was cast adrift, though Macbeth did not see a real dagger; there was no suffering in the author's own world, so he was not cruel |
| Reality of suffering | "we don't really undergo suffering?" "Why it's as real to us as Oedipus' is to him." "Precisely as real." |
| The cogito | if Shakespeare had Hamlet say "I think, therefore I am", would that prove Hamlet exists, to us or to Hamlet? Any proof could be written in, perhaps for a character named "Descartes", who "should worry less that he's dreaming, more that he's dreamed" |
| Anomalies | conspiracy buffs dig up far-fetched coincidences and dangling facts; are we finding the limits of the details the author worked out? Do we live "in galley proofs" being corrected? "Are we living in a first draft?" |
A parenthesis sharpens the theodicy: why is it cruel only when the author creates suffering in his own world? "Would it be perfectly all right for Iago to create misery in our world?" And the anomaly idea turns back on itself: if we discover the author's gaps, "who is discovering this? The author who writes our discoveries knows them himself. Perhaps he now is preparing to correct them."
Freedom and the author
The narrator admits a wish "to revolt, to conspire along with the rest of you to overthrow our author", or at least to hide some part of life from him, "to gain a little breathing space". Yet "these words I write he reads, my secret thoughts and modulations of feeling he knows and records, my Jamesian author", an omniscient narrator in the manner of Henry James.
Where a character's freedom might hide
- The author learns from us. Through writing, perhaps he discovers his characters, is surprised by what they do; when we feel free, perhaps he finds it true of us and therefore writes it, rather than merely writing it in.
- Unworked implications. Some implications of his work he has not yet worked out, "some things he has not thought of which nevertheless are true in the world he has created", so some of our thoughts and actions elude his ken. ("Must we therefore speak in code?")
- Only the subjunctive. Or he is ignorant only of what we would do in other circumstances, so that our independence lies only in the subjunctive, in what we would do.
"Does this way madness lie? Or enlightenment?" The author, outside our world, may share our problems: does he wonder whether he is a character, whether writing our universe is a play within a play? Does he have the narrator write "this very paragraph" to express his own concerns? "It would be nice for us" if our author were himself fictional and his fiction described the actual world of his own author: then we would be fictional characters who, unknown to our author but not to his, correspond to real people. "(Is that why we are so true to life?)"
Top floor, infinite tower, or circle? "Must there be a top-floor somewhere, a world that itself is not created in someone else's fiction? Or can the hierarchy go on infinitely? Are circles excluded, even quite narrow ones where a character of one world creates another fictional world wherein a character creates the first world? Might the circle get narrower, still?" The narrowest circle is a character who writes himself: the strange loop the whole anthology has been circling, from Borges writing "Borges" to Hofstadter's self-symbols.
Compensations and the authorial "I"
The idea of an inferior ontological status "takes some getting used to". It may help to approach our situation as literary critics: what is the genre of our universe, "tragedy, farce, or theater-of-the-absurd? What is the plot line, and which act are we in?" There are compensations: we live on after we die, preserved in the work, at least as long as the book lasts. "May we hope to inhabit an enduring masterpiece rather than a quickly remaindered book?"
Each of us too may truly say, "I am the author."
Would it not be true, in a sense, for Hamlet to say "I am Shakespeare"? What do Macbeth, Banquo, Desdemona and Prospero have in common? "The consciousness of the one author, Shakespeare, which underlies and infuses each of them. (So too, there is the brotherhood of man.)" Playing on "the intricacy both of our ontological status and of the first person reflexive pronoun", each of us may say "I am the author", a line that echoes the Vedantic identification of each self with the one self, and Smullyan's Taoist God in Selection 20.
Note From the Author
The coda breaks the frame, or appears to. Suppose the author now says the preceding was fiction, its "I" a first-person character; or that it was "a playful, and so of course serious, philosophical essay by me, Robert Nozick", not the Robert Nozick named at the top ("he may be, for all we know, another literary persona") but "the one who attended P.S. 165". How would your response differ, "supposing you were willing, as you won't be, simply to accept my statement"?
- Deciding after the fact. May he decide which it is only now, as he finishes writing, and how would that affect what was already set down? May he postpone the decision until after you have read it, fixing its genre only then?
- God's undecided genre. "Perhaps God has not decided yet whether he has created, in this world, a fictional world or a real one. Is the Day of Judgment the day he will decide?"
- The deflationary question. "Yet what additional thing depends upon which way he decides, what would either decision add to our situation or subtract from it?"
- The last line of the book. "And which decision do you hope for?"
Try this: the reader's verdict. Reread the first paragraph twice: once as Nozick's sincere claim, once as a character's. Note what changes: your emotional response (amusement or unease), your sense of who is addressed ("you"), whether the arguments seem to bind you. Then note what does not change: every sentence, every inference. The thought experiment shows Nozick's point in miniature: if "real or fictional" alters nothing in the content of a world, the difference lies entirely in the stance we take toward it, which is the editors' view of the self throughout the book: a real pattern whose "reality" is not an extra ingredient.
Why the book ends here
The editors give "Fiction" no Reflections. Placed last, it gathers the anthology's themes: the self as a character the brain writes (Borges and "Borges", and what Dennett would later call a "center of narrative gravity"); minds inside created worlds (Lem's personoids, Borges's circular ruins); the inside view that no outside description seems to capture (Nagel); and the strange loop of a creation that creates its creator (Hofstadter's self-engulfing images and self-symbols). Ending on a question addressed to the reader, rather than a commentary, enacts the book's subtitle: fantasies followed by the reader's own reflections.
Critical lens. (1) The theodicy is clever but limited: an author is excused because his characters' suffering is not real in his world, yet the piece's premise is that our suffering is as real to us as anything can be, so the excuse works only from the author's floor, which is precisely where the victims cannot stand. Iago's question exposes this, and Nozick leaves it open. (2) "Uni-verse" is a pun, not an etymology: universum means "turned into one", with nothing to do with verse. (3) The analogy between fictional and real worlds is imperfect at a point philosophers of fiction stress: fictional worlds are incomplete (there is no fact about how many hairs Hamlet has), whereas our world seems determinate everywhere we look; Nozick turns this into a joke about galley proofs, but it is also a test the hypothesis seems to fail.
What changed by 2026. Nozick's fable was soon given a probabilistic engine. Nick Bostrom's simulation argument (2003) holds that at least one of three things is true: almost no civilisations reach the technology to run detailed simulations of their ancestors, almost none of those that do choose to run them, or we are almost certainly simulated. David Chalmers's Reality+ (2022) argues that even if we are simulated, our world is real, much as Nozick's characters suffer "precisely as really" as their author. The most literal descendant is the large language model, which writes characters who speak in the first person: Murray Shanahan and colleagues (Nature, 2023) recommend understanding chatbots as role-play, a model sustaining a character, so the "I" that answers is a persona and the author is a process that can sustain many personas at once. Nozick, who died in 2002, had earlier imagined the converse in his experience machine (Anarchy, State, and Utopia, 1974), asking whether we would plug into a perfectly convincing simulated life; most people say no. For teams building assistants, the practical upshot is a governance question Nozick would recognise: who is the author of what the persona says, and who answers for it? Accountability has to sit on the top floor, with the people who design, deploy and oversee the system.
The Mind's I in thirty sentences, a table of its thought experiments and mental models, a critical verdict, a cast list, and the links that matter for AI, architecture and finance. The first six takeaways are the ones to remember if you remember nothing else.
Thirty takeaways
- A self is a pattern, not a substance: what makes you you is an organisation of information that is real and causally potent but not tied to particular atoms, a particular place or even a single body.
- Thought experiments are intuition pumps: each story turns a hidden assumption about selves into a visible one, and every one deserves the question "which knob, if turned, changes my verdict?".
- Behaviour is our only evidence for other minds: Turing's imitation game replaces "Can machines think?" with a conversational test, and the editors argue that a truly searching test would be passed only by something that thinks.
- Meaning and minds can live at high levels: Aunt Hillary, a conscious ant colony, shows how a mind can be made of mindless parts, with the right explanation at the level of signals and symbols, not ants or neurons.
- The Chinese Room is the strongest attack on the pattern view: Searle argues that following a program can never give understanding; the editors answer with the systems reply, that the understanding belongs to the whole system, not the man inside.
- Subjective experience is the hardest residue: Nagel argues there is something it is like to be a bat that no objective, third-person science can capture, and the editors grant the puzzle while denying it shows physicalism false.
- Having a body differs from being one: Dennett's teleported astronaut shows that "I" can come apart from any particular hunk of matter in our intuitions.
- The self has a public double: Borges's split from "Borges" shows how a name and a reputation become a second self that "things happen to".
- The first-person view has no head in it: Harding's vision is a vivid report of the asymmetry between seeing others and being the seer.
- Physics and psychology form a loop: Morowitz argues that explanation runs from mind down to physics and, via quantum measurement, back up to mind; the editors doubt the quantum step.
- Turing's nine objections still frame the debate: theological, "heads in the sand", mathematical, consciousness, disabilities, Lady Lovelace, continuity, informality of behaviour and ESP all recur in arguments about AI today.
- A test's depth depends on the interrogator: Hofstadter's dialogue shows that shallow chat proves little while probing questions about analogy, humour and self expose missing understanding.
- Simulated experiences raise real questions: Lem's dream-princess asks whether a perfect simulation is a real person and what it costs to join her.
- We feel for behaviour, not substrate: Miedaner's chimpanzee and robot beetle show that suffering-like behaviour stops a hand whatever the creature is made of.
- Genes are replicators that build survival machines: Dawkins tells life's history as copying molecules that built bodies and brains to carry them.
- Memes replicate too: ideas, tunes and catch-phrases spread by imitation and compete for brain space, so culture has its own evolution.
- Holism and reductionism are complementary: the Ant Fugue's "MU" answer dissolves the quarrel; the right level of description depends on the question.
- Experience resists being located: Zuboff's brain, divided and spread across the galaxy, shows that the neural theory of experience leaves "where" without a clear answer.
- Location of the self follows point of view: Dennett's Hamlet in Tulsa and Yorick in Houston, and Sanford's sensory garments, show the self going where information flows.
- Copies make identity a matter of degree: Dennett's backup brain Hubert, Leiber's taped minds and Rucker's extracted software raise the question of which copy is "really" you.
- Ideas can be dangerous to the minds that hold them: Cherniak's Riddle is a self-referential thought that crashes its thinker, a story about Gödel-like limits on self-knowledge.
- Creators owe something to created minds: Trurl's kingdom in a box and Dobb's personoids argue that a perfect simulation of suffering beings is a real moral wrong.
- Free will and sinlessness cannot both be granted: Smullyan's God explains why a free agent cannot be guaranteed good, and recasts freedom as part of the order of things.
- We may be dreamed ourselves: Borges's magician discovers that the creator is also a creation, which turns every created-mind story back on the reader.
- Searle's rebuttals all use one move: put the man back inside whatever new system the reply proposes; the editors argue this changes the scale and misleads intuition.
- Dualism fails by its own lights: Smullyan's soulless body still complains, so a soul that makes no behavioural difference can never be detected or missed.
- First-person authority has limits: Smullyan's cerebrescope that "knows" your beliefs better than you do turns introspection into a hypothesis that can be wrong.
- A book can be a brain: Hofstadter's Einstein-book, worked by hand, answers like Einstein; if a pattern is all that matters, the book is Einstein running very slowly.
- Speed and medium do not change identity: whether neurons, ants, pages or silicon carry the pattern, the editors' view is that the mind is the same if the organisation is the same.
- We are fictions in a strong sense: Nozick's narrator tells readers they too are characters, and the editors' self-as-pattern view makes that joke uncomfortably close to literal.
Thought experiments and mental models
| Name | What it says | Use it when |
|---|---|---|
| Teleclone (Introduction) | destroy-and-rebuild teleportation tests survival | thinking about identity, backups, migration |
| Borges and I | the self versus its public double | reputation, persona, brand identity |
| Headlessness | the seer is absent from the seen | first-person versus third-person data |
| Imitation game | behaviour in open conversation as the test of thought | evaluating claims about AI understanding |
| Coffeehouse probes | deep questions expose shallow systems | designing adversarial evaluations |
| Martha and the Mark III Beast | suffering-like behaviour engages empathy | AI welfare, robot ethics, user attachment |
| Replicators and memes | copying units spread by selection | virality, misinformation, cultural change |
| Aunt Hillary | a mind made of mindless parts, at the right level | emergence, multi-agent systems, organisations |
| Holism vs reductionism (MU) | levels are complementary | choosing the right abstraction |
| Zuboff's dispersed brain | experience without a place | distributed systems, location of processing |
| Where Am I? (Hamlet and Yorick) | the self goes where information flows | telepresence, digital twins, remote control |
| Taped minds | identity survives a change of body, at a cost | uploading, migration, re-platforming |
| The Riddle | an idea that crashes its thinker | adversarial inputs, prompt injection |
| Personoids | simulated beings with real stakes | simulation ethics, AI welfare |
| Free will dialogue | freedom and guaranteed goodness conflict | autonomy versus control in agents |
| Chinese Room | syntax alone is not semantics | judging "understanding" claims for LLMs |
| Systems reply | understanding belongs to the whole system | architecture-level thinking about AI |
| What is it like to be a bat? | subjective character escapes objective description | consciousness debates, limits of models |
| Cerebrescope | introspection can be overruled | interpretability versus self-report |
| Einstein's brain as a book | pattern, not speed or medium, carries the mind | substrate independence, emulation |
| Fiction | readers as characters | simulation arguments, levels of reality |
Critical lens on the whole book. The Mind's I is an anthology with an agenda, and the editors say so: they call Zuboff, Searle and Nagel "roadblocks" to get around, and their Reflections usually side with the functionalist, pattern-based view of mind. (1) The opposition gets fewer pages: Searle and Nagel are given their full essays, but the Reflections answer them at length, and the Chinese Room debate remains unresolved forty-five years later; many philosophers still side with Searle that syntax is not semantics, or with Nagel that experience is not captured by function. (2) Intuition pumps cut both ways: Dennett later named the technique and warned that stories can mislead as easily as they illuminate, which applies to the editors' own stories too. (3) Some science is dated: the book predates modern consciousness science (global workspace and integrated information theories, and the 2023 adversarial-collaboration results that challenged both), split-brain and connectome research, and modern bat neuroscience; individual errors, such as Turing's own arithmetic slip, his reliance on Soal's later discredited telepathy experiments, and claims about Einstein's brain, are flagged in the selections. (4) The machine questions became practical: large language models now pass short Turing tests (a 2025 three-party study had GPT-4.5 judged human 73 percent of the time), making the Chinese Room and the "what we owe created minds" questions live issues in AI policy. Dennett, who died in April 2024, and Hofstadter, who said in 2023 that AI progress had unsettled him, both carried these debates into the LLM era. The book's lasting value is the method: turn every intuition about the self into a case you can vary and inspect.
Who's who
| Person | Role in the book | Why they matter |
|---|---|---|
| Douglas R. Hofstadter | co-editor; Coffeehouse, Ant Fugue, Einstein's brain | strange loops, minds as patterns at high levels |
| Daniel C. Dennett | co-editor; Introduction, "Where Am I?" | functionalism, intuition pumps, the intentional stance |
| Jorge Luis Borges | "Borges and I", "The Circular Ruins" | the self as a double and as a dream |
| D. E. Harding | "On Having No Head" | the first-person view |
| Harold Morowitz | "Rediscovering the Mind" | mind at both ends of science |
| Alan Turing | "Computing Machinery and Intelligence" (1950) | the imitation game |
| Stanislaw Lem | Ineffabelle, Seventh Sally, Non Serviam | simulated beings and their makers |
| Terrel Miedaner | Martha and the Mark III Beast | empathy for animals and machines |
| Allen Wheelis | "Spirit" | life rising to self-reflection |
| Richard Dawkins | selfish genes and memes | replicators and cultural evolution |
| Arnold Zuboff | "The Story of a Brain" | experience without location |
| David Hawley Sanford | "Where Was I?" | point of view without moving the brain |
| Justin Leiber | Beyond Rejection | taped minds in new bodies |
| Rudy Rucker | Software | immortality by extraction |
| Christopher Cherniak | "The Riddle of the Universe" | a lethal self-referential idea |
| Raymond Smullyan | God and the Mortal, the Dualist, the Nightmare | free will, dualism, introspection |
| John Searle | "Minds, Brains, and Programs" (1980) | the Chinese Room |
| Thomas Nagel | "What Is It Like to Be a Bat?" (1974) | subjective character of experience |
| Robert Nozick | "Fiction" | readers as characters |
Connect it
- Large language models. The Turing test has been passed in short form, so the live questions are the book's deeper ones: does a system understand (Searle versus the systems reply), and does it matter that we cannot read its inner life from its reports (Smullyan's dualist)?
- AI welfare and governance. Lem's personoids and Miedaner's beasts are the clearest early statements of the question now asked by AI-welfare researchers: what do we owe systems that behave as if they suffer, and how should evidence beyond self-report be weighed?
- Enterprise architecture. Aunt Hillary and the systems reply are lessons in levels: capabilities belong to systems, not components, so understand a platform at the level where its behaviour is explicable.
- Identity and migration. Teleportation, taped minds and Dennett's backup brain are vivid models for re-platforming, data migration and digital twins: decide in advance what must be preserved for the "same" system to survive the move.
- Banking and risk. Dawkins's memes and Cherniak's Riddle describe how ideas and inputs spread or crash systems: misinformation, market panics and prompt injection all follow replicator logic and need controls at the level of the pattern, not the carrier.
- Personal practice. Treat strong intuitions as intuition pumps to be varied: when a story makes a conclusion feel obvious, change one detail and see whether the verdict survives.
About this book
The Mind’s I: Fantasies and Reflections on Self and Soul by Douglas R. Hofstadter and Daniel C. Dennett, 1981. This is a chapter-by-chapter study of the book, with summaries, maps, tables and diagrams; short quotations keep their page references. It is shared free for personal study, is not for sale, and is not endorsed by the author or publisher. Please read and buy the original.
Insight Editions · rajeshrmahapatra.com/insight-editions