Insight Editions · No. 25
The Infinity Machine
Demis Hassabis, DeepMind, and the Quest for Superintelligence
Sebastian Mallaby
Sebastian Mallaby recounts Demis Hassabis's career and DeepMind's development, from its founding and landmark research to its acquisition by Google and competition in the race for advanced AI.
A study guide to The Infinity Machine by Sebastian Mallaby. Shared free for personal reading; not for sale and not endorsed by the authors or publishers. Please read and buy the original.
The Book on One Page
Mallaby's book is a double portrait of Demis Hassabis and the machines he set out to build: a chess prodigy who founded DeepMind in 2010 to "solve intelligence" and use it to understand the universe. Its argument is that Hassabis is decent, scientific and largely uninterested in money, yet every attempt he made to keep AGI in safe, singular hands failed, and he ended up racing anyway. The scientific story (Atari, AlphaGo, AlphaFold, Gemini) is a story of brilliant bets; the political story (Thiel, Google, Musk, OpenAI, China) is a story of lost control.
The argument in eight moves
- 1. A founder shaped by games and destiny. A cash-strapped North London prodigy, a chess epiphany at twelve, Theme Park at seventeen and a Cambridge credo that induction, not logic, is the road to intelligence set the mission early; Elixir's failure taught him the cost of over-inspiring a team (Intro, Ch. 1-3).
- 2. The brain as existence proof. A neuroscience PhD on memory as imagination, plus Shane Legg's AGI theory and Mustafa Suleyman's social mission, produced a three-founder company funded by believers: Thiel's Founders Fund, Gammon, Tallinn (Ch. 4-5).
- 3. Fusing two tribes. DQN married Toronto's deep learning to Alberta's reinforcement learning and learned Atari from pixels (2013); venture capital could not carry blue-sky science, so Google bought DeepMind for $650 million in January 2014 with autonomy and an ethics board attached (Ch. 6-8).
- 4. Intuition is learnable. AlphaGo paired policy and value networks with tree search and beat Lee Sedol 4-1 in March 2016; AlphaZero then mastered three games from self-play alone (Ch. 9, 12).
- 5. The singleton dream dies. The SpaceX safety meeting of August 2015 turned advisers into rivals: Musk and Altman founded OpenAI. Suleyman's NHS work stalled on a privacy backlash, and Project Mario's three-year bid for semi-independence from Google failed (Ch. 10, 11, 14).
- 6. The wrong bet on language. DeepMind read AlphaZero as a victory for RL just as the transformer and GPT opened a second route to the superhuman: breadth from internet text. Hassabis doubted grounding and missed the shortcut; Gaia and AlphaStar exposed RL's limits (Ch. 12-13, 16).
- 7. Science at digital speed. Seven pivots (Foldit to CASP, RL to deep learning, contact maps to distograms, CNNs to transformers, Senior to Jumper) took AlphaFold 2 to 92.4 GDT at CASP14 (2020), a fifty-year problem solved and a 2024 Nobel Prize in Chemistry: proof that a lab without a monoculture can do great science (Ch. 15).
- 8. Race, comeback, conjecture. ChatGPT forced the Brain-DeepMind merger (April 2023); safety pleas, pauses and summits could not stop a US-China race; OpenAI's o1 beat DeepMind to RL-style reasoning, yet Gemini 2.5 and 3 restored Google's lead, through the uncoordinated sprint Hassabis feared. He ends as "Turing's champion," betting nature is classically learnable (Ch. 17-21).
Read the book as two ledgers. On the science ledger Hassabis wins almost every bet that turns on taste and persistence. On the control ledger he loses every structure he tries to build (ethics board, review panel, spin-out, singleton), and concludes that personal influence beats governance.
Chapter map
| Ch | Title | One-line payload |
|---|---|---|
| Intro | The Sweetness | Why a scientist who sees AI's dangers builds it anyway |
| 1 | Destiny | Chess prodigy turned game designer; "no 50 percent mode" |
| 2 | "Deep Philosophical Questions" | Cambridge: induction beats logic; AI as meta-solution |
| 3 | The Jedi | Elixir's charisma trap; Silver to RL, Hassabis to neuroscience |
| 4 | The Gang of Three | Legg's AGI theory and Suleyman's mission complete the team |
| 5 | Founding DeepMind | Thiel's $2.3m; conviction, runway and culture |
| 6 | Atari | DQN fuses deep learning and RL; replay and frozen targets |
| 7 | Thiel Trouble | Series C near-death; VC cannot carry blue-sky science |
| 8 | Get Google | $650m sale with autonomy and an ethics board |
| 9 | Intuition | AlphaGo beats Lee Sedol 4-1; intuition is learnable |
| 10 | Out of Eden | SpaceX safety meeting fails; OpenAI ends the singleton |
| 11 | P0 Plus Plus | Streams works, privacy backlash stalls DeepMind Health |
| 12 | The Agent and the Transformer | AlphaZero's depth versus GPT's breadth |
| 13 | On Language and Nature | Grounding doubts; Gaia and AlphaStar show RL's limits |
| 14 | Project Mario | Three-year spin-out bid fails; Suleyman exits |
| 15 | Fermat for Biology | AlphaFold: seven pivots, Jumper, 92.4 GDT, a Nobel |
| 16 | The Power and the Glory | Gopher to Sparrow: strong research, no products |
| 17 | RaceGPT | ChatGPT forces the Brain-DeepMind merger |
| 18 | "We're Cooked" | Doom warnings, Bletchley, Altman's five-day ouster |
| 19 | Step by Step | Gemini returns; o1 beats DeepMind to RL reasoning |
| 20 | Comeback, and Beyond | Gemini 2.5 and 3 lead; DeepSeek, cheating agents |
| Epi | Turing's Champion | Nobel, reluctant power, the classical-learning conjecture |
The Sweetness
Mallaby sets up the book as a double portrait: Demis Hassabis, chess prodigy, polymath and Nobel laureate, and the machines he has spent his life trying to build. The organising question is why a scientist who fully understands AI's dangers builds it anyway. Mallaby's verdict, stated up front, is that Hassabis is decent and public-spirited, chasing enlightenment rather than money or power, yet may be no more able than Oppenheimer to control what he creates.
The stakes as Mallaby frames them
- Scale of the shift. Mallaby expects AI to beat humans at almost every mental task in the not-so-distant future, and ranks the change above anything since Homo sapiens acquired abstract thought roughly 70,000 years ago.
- The machines in question. The book's subject is systems that are intuitive, creative and even original, not narrow calculators.
- Two subjects, one story. The human (Hassabis) and the technology (DeepMind's systems) are treated as inseparable: the character of the builder shapes what gets built.
The stage pitch: understand the mind by building one
Mallaby first met Hassabis in the mid-2010s, when he was already a star technologist with a comfortable fortune but looked far younger than his 38 years: slight, boyish, crewneck and loose slacks on conference stages. A typical 2015 talk opened with the plainest possible definition (AI is the technology of making machines smart) and then turned into a statement of life purpose: the pursuit of machine superintelligence.
The logic of the talk ran in steps:
- Two master sciences. Growing up in North London, Hassabis singled out physics (the external world, particles to cosmos) and neuroscience (the internal world of neurons, synapses and electrical pulses).
- The internal wins. In his twenties he ranked neuroscience higher, because intelligence is the root of everything else: it is the mechanism through which humans perceive reality at all.
- Kant. He cited the idea that the mind interprets the world, restating it as the mind creating the reality around us.
- Feynman. A slide of Richard Feynman and his dictum, "What I cannot build, I do not understand," supplied the method: to grasp human intelligence, build an artificial analog that mimics human thinking.
- Profit second. Commercial potential was explicitly secondary; the aim was to understand our own minds better.
Audiences were rapt and mystified: this was no app founder selling riches but a blend of computer science and neuroscience with enlightenment as the prize. When Hassabis founded DeepMind in 2010, fellow scientists thought humanlike AI impossible and almost every investor turned him down on the grounds that enlightenment is not a business model. He raised money and recruited gifted researchers anyway, on the strength of the vision alone.
Mallaby thinks "artificial intelligence" is too bloodless a label for what Hassabis wanted: an omniscient machine that would help us understand ourselves, unravel physics, and occupy roughly the place in the cosmos that believers once gave to an all-powerful God.
The inventor's dilemma
Mallaby's long-standing interest is the predicament of scientists in society: they are truth-seekers, yet also potential destroyers of jobs, habits of thought and even human existence. AI is accused on every count, and Hassabis knows the full menu of doom scenarios.
- Extinction fears. Many leading builders, including some at DeepMind, fear silicon intelligence could eradicate the biological kind; extreme pessimists call AI potentially the last event in human history.
- Certain harms. Even if annihilation is speculative, deep fakes, terrifying weapons and economic, political and philosophical turmoil are, in Mallaby's view, certain.
- The central quandary. Given these dangers, why would anyone choose to build the technology?
The generous answer. Humanity contains the risks and harvests the upside: medicine, science, climate fixes, even tools that counter the dangers pessimists stress. Example: the fear that children who never learn to write will never learn to think is offset by chatbots as infinitely patient tutors that build bespoke quizzes and grade instantly. History mostly agrees: gunpowder and fission made war worse, cigarettes and attention-wrecking social media are probably net negative, but technology in general has extended lives and enlarged experience. Reid Hoffman calls technology "the thing that makes us, us," and Mallaby extends Descartes: I invent, therefore I am.
The troubling answer. Discovery itself is intoxicating, the Icarus instinct, and it overrides judgment. If invention is intrinsic to being human, humans may keep inventing until they go too far, and the rest of us become hostages to the inventors' appetite.
Hinton's confession
Mallaby's emblem for the troubling answer is Geoffrey Hinton, the academic father of deep learning, a 2024 Nobel laureate (the same year as Hassabis) and a man of prickly principle: as a young professor he moved from the US to Canada to avoid relying on US military research grants, and in 2023, as AI began to show human fluency, he quit a lucrative Google post partly to speak freely about its dangers.
After a speech at the Royal Society in London, Hinton was overheard by a journalist talking with Oxford philosopher Nick Bostrom. He expected properly intelligent machines to be a long way off, but once they worked, abuse would be impossible to prevent; he placed himself in "the camp that is hopeless" and predicted political systems would use AI to terrorise people. Asked why he did the research, he said he could offer the usual arguments, but "the prospect of discovery is too sweet." Outcome: the line deliberately or not echoes J. Robert Oppenheimer, who said that when something is "technically sweet" you build it and argue about its use only after success. Hinton later told Mallaby the line was apt, which is precisely why he wishes he had not said it.
What DeepMind delivered
2010 Hassabis founds DeepMind in London, promising a Manhattan Project for AI.
2014 DeepMind is bought by Google (the Introduction only says "now owned by Google").
2016 DeepMind's Go system surpasses the intuitive play of the world's best Go players: the first grand challenge, in computer science.
2020 Second grand challenge, in biochemistry: 32 algorithms stitched together to predict protein shapes, the basis of Hassabis's share of the 2024 Nobel Prize in Chemistry.
Late 2022 OpenAI's ChatGPT sets off the AI frenzy; Mallaby pitches Hassabis on the book the same season.
2025 Facing Silicon Valley and Chinese followers, DeepMind is a chatbot front-runner and leads in video generation, drug discovery and mathematics.
2026 Mallaby finishes the book.
Mallaby's image for the result is Los Alamos relocated to a district of trendy restaurants and boutiques around a nineteenth-century London railway station (King's Cross, where DeepMind is based).
- Mathematics. In July 2025 an advanced Gemini Deep Think model reached gold-medal standard at the International Mathematical Olympiad, solving 5 of 6 problems in natural language.
- Video. Veo 3 (May 2025) added synchronised sound to generated video and became a reference point for the field.
- Chatbots. Gemini 3 (November 2025) launched at or near the top of the main public leaderboards, consolidating Google's position after its slow start against ChatGPT.
- Nobel detail. The 2024 chemistry prize went half to David Baker (protein design) and half jointly to Hassabis and John Jumper; Hinton shared physics with John Hopfield.
Access, method and the café scene
The book was pitched as a start-up adventure, an explanation of AI, and an inquiry into what drives Hassabis. Mallaby treats him partly as a type (the missionary entrepreneur and unconventional scientist who becomes the right person when hardware, software and data align to make superhuman intelligence possible) and partly as a window on the old questions of motive and purpose.
- Access. Hassabis agreed to deep access, reasoning that societies will not trust inventors of transformative technology unless they understand what makes them tick.
- Time with the subject. About 30 hours of conversation over three years.
- Wider sourcing. Interviews with more than 100 people inside and outside DeepMind.
- Finding. Hassabis is an exceptional consumer and teller of stories, shaped by novels and films; his leadership is bound up with his gift for narrating his own experience.
The surprise that stuck with Mallaby came on a hot late-summer day in 2023 at a café in a North London park near Hassabis's childhood neighbourhood. Now with a shaved head and fashion-forward glasses, Hassabis set out a motive deeper than Kant and Feynman: building AI as a way of drawing closer to whatever intelligence may have designed the universe.
- Science as religion. He calls himself first a scientist whose goal is to understand nature, and frames science as something like reading the mind of God.
- The improbability of understanding. That the world is comprehensible at all, he believes, has a reason; it is beyond evolutionary coincidence that humans can build electron microscopes and detect black holes colliding over a billion years ago.
- Sand and copper. Why should electrons moving through sand and copper produce a system that beats a Go master? Why should a table be solid?
- The 2 a.m. feeling. At his desk late at night he feels reality screaming at him, trying to tell him something; he has felt this since childhood.
- Agnostic on labels. Call it God's design or just nature; we still do not know what time or gravity are, and he wants to understand before he dies.
I would like to understand before I croak.
The verdict and its defence
Writing in 2026, Mallaby observes that a new intelligence is being created by a very small number of people, each driven by some mix of curiosity, hubris, vanity, avarice, idealism and craving, so their characters matter to society.
- The judgment. Hassabis, who blazed the trail rivals followed, is decent and wants the best for humanity. He has ego, is fearsomely competitive, and his sense of destiny borders on the messianic, but his goal is scientific enlightenment; the spiritual language shows how seriously he takes it.
- Anticipated objection. Since the 2008 financial crisis humbled Wall Street, Silicon Valley has become the centre of commercial power and of tycoon hubris, discrediting the idea that messianic innovators mean well.
- Mallaby's reply. Wealth and egomania are not linked by necessity; the tech sector is not uniformly anarcho-libertarian (for every Trump supporter there are several progressives); for every avaricious egotist there is a Bill Gates spending his fortune on the life expectancy of the poorest.
- Hassabis as outlier. He chose to stay in London, away from Valley hype, and talks of eventually retiring to a university to return to physics, neuroscience and philosophy.
Mallaby announces his judgment of Hassabis's character before presenting the evidence, and the book rests on access granted by its subject, who agreed to it partly to win public trust. The concessions (ego, competitiveness, near-messianism) are real but framed as compatible with good intent. Readers should test later chapters (the Google sale, the commercial race after ChatGPT) against that framing. Two compressions: the 2020 result was protein structure prediction at CASP14, while predicted structures for nearly all catalogued proteins were released only in 2022; and the Gates comparison answers a general charge about tech leaders rather than offering evidence about Hassabis himself.
Mallaby closes on context. Hassabis is not in an ivory tower but at the centre of a capitalist fight over AI, at a moment when regulators look unlikely to contain the fallout: a transformative scientific shock coinciding with unstable geopolitical competition and political chaos in the United States, the country normally expected to lead safeguards for a Promethean technology. His struggle to stay true to his values is, for Mallaby, a defining story of the era. He wants to do good; whether he can be good, and what he can actually do to contain AI's dangers, are the open questions, with Oppenheimer (who built the bomb but could not control its use) as the shadow over the whole book.
Destiny
Chapter 1 follows Hassabis from a cash-strapped North London childhood through the junior chess circuit to the Bullfrog games studio, where at seventeen he decided to devote his life to AI. Mallaby's interpretive key is Ender's Game, the novel Hassabis himself chose as his mirror: a small, isolated boy genius tested to breaking point and asked to carry humanity's fate. The defining trait, in cofounder Shane Legg's words, is determination with "no 50 percent mode," a drive Hassabis traces to taking his father's "try your best" literally.
The Ender template
By about age thirty, partway through a neuroscience PhD, Hassabis had already been a chess master, games designer, amateur theoretical physicist, entrepreneur, computer scientist and five-time world champion at the Mind Sports Olympiad. That is when he found Orson Scott Card's Ender's Game.
- The novel. A tiny boy genius is removed from his family to an orbital battle school, manipulated by adults, bullied by peers and tested to the limit to see whether he can save the human race. He destroys an alien armada and saves Earth, though the question of genocide hangs over his victory.
- Two readings. Hassabis's wife felt sorry for Ender, a child robbed of childhood and yoked to a mission chosen by adults. Hassabis identified with him: also small, also isolated by talent, also tested, also driven to leave a mark. His stated ambition was to surpass Newton and Einstein and understand the fabric of reality.
- Mallaby's point. The fable appealed precisely because the saviour pays an enormous personal price; endurance under suffering is part of the self-image.
- The interview set-up. Some fifteen years later, in his mid-forties, Hassabis asked Mallaby to read the novel before their first long interview in a London restaurant: to know him, one had to know his fictional alter ego.
- The night shift. Like Ender he carries a sense of burden. On top of office hours he works a second shift from about 10 p.m. to 4 a.m., on the logic that someone trying to solve humanity's problems and understand reality has no time to waste.
Mallaby put the Ender framing to Shane Legg, who teamed up with Hassabis in 2010 when both were postdocs at University College London fusing computer science and neuroscience. Legg was wary of the label but emphatic on substance: Hassabis's defining characteristic is determination beyond anyone he has known; he lives the mission 24 hours a day, with Liverpool FC his only hobby, and was like that from the start. Legg's gloss on the father story: a parent's comforting "try your best" became, for Demis, try to the point of breaking yourself. Outcome: the line that frames the chapter, that there is no 50 percent mode, not even a 99 percent mode, only 100.
Origins: Finchley, the red van and a chess prodigy
Hassabis was born in Finchley, North London, in July 1976, eldest child of a mixed family Mallaby presents as a product of London's melting pot.
- Mother. Chinese Singaporean; grew up in absolute poverty, part of her childhood as an orphan on Singapore's streets, taken in by a relative, came to London to study nursing; when Demis was small she worked as a John Lewis sales assistant and took part-time cleaning jobs.
- Father. Greek Cypriot, the first in his family at university, a bohemian who could not abide office work: an aspiring singer-songwriter who sold toys from a battered red Volkswagen van.
- Siblings. Two younger children, including brother George, later the first opponent beaten by Demis's AI.
- Leonard Barden. Patriarch of English chess, international player in the 1950s-60s, columnist and TV commentator, whose verdict set the family's course.
A primary teacher described the young Demis as "sparkling": an unusual mix of levity and seriousness, and relentlessly competitive.
- Age 4. Climbed on a chair to watch his father play his uncle; within weeks he could beat adults.
- Age 5. Played tournaments sitting on a telephone book on two stacked chairs, often beating older children.
- Age 6. Qualified for the British Under-14 championship, won two games, then fell asleep at the board when a game ran past bedtime.
- Barden's verdict. After those two wins Barden told the father this was the best six-year-old he had ever seen.
- Age 9. Captain of England's Under-11 team.
- Age 13. Reached chess master rank; second-strongest player of his age in the world.
- After university. A record string of golds in the Mind Sports Olympiad's five-game event (the Pentamind).
Barden's prophecy as family destiny
The father treated Barden's remark as instruction from on high. For about six years, almost every weekend, he drove Demis in the VW to tournaments in shabby, remote church halls, leaving his wife with the younger children and her jobs.
- Cost. They slept in sleeping bags over the van's rear engine or shared a hostel bunk; prize money covered the hostel if Demis won, but his mother still worried about the travel. Hassabis assumes his parents argued about money.
- Lost childhood. Chess filled every weekend and school holiday; like Ender, he barely knew what ordinary leisure was.
- Brutality of the circuit. Tables had wooden boards underneath to stop players kicking each other.
- The father's temper. As the father gave up more work and music time, he erupted after bad games. After one loss a rook up, he screamed at Demis in a hostel. Demis replied that he had obviously tried his best and would not take it any more; Hassabis remembers that as the last time his father screamed at him, though it had been regular before.
- Hassabis's reading. He calls it love distorted by pressure, probably from his mother questioning why her husband had no normal job and whether the chess was a waste. Today, he says, his father is so laid-back he seems a different man.
Hassabis thinks his father meant the phrase as comfort, yet the anger after losses contradicted it. At nine or ten he resolved the contradiction his own way: the only proof of a best effort is pushing to the point just before collapse. Burnout means you overdid it; finishing with energy left means you could have tried harder. His image is the marathon runner who falls over the line and ideally ends up hospitalised, but not dead.
Liechtenstein: the epiphany that freed him
The circuit had a thin line between triumph and breakdown. Hassabis's earliest memory is winning the London Under-8 championship, turning with the trophy, and seeing the runner-up, Dharshan Kumaran (a future grandmaster and close friend), sobbing while his father berated him. Many of his chess friends were, in his word, destroyed: they drank or burned out.
At an international tournament near Liechtenstein, Hassabis played an experienced, chain-smoking German master old enough to be his father. The game ran almost ten hours into a rare endgame: Hassabis's queen against rook, bishop and knight. As the hall emptied, the German trapped his king; mate looked certain and the exhausted boy resigned. The German jumped up, asked why, and demonstrated that sacrificing the queen would have forced stalemate; his friends joined in jeering. Hassabis felt sick all day. Next morning came the turn: a hall full of brilliant brains had drained itself over black and white squares, effort that could have gone to science or medicine. He decided chess players were "wasting our minds" and that his talent needed a higher purpose than the professional chess career everyone, Barden included, had foreseen.
Happier, broader years
- Chess without possession. He kept competing, still fierce: Kumaran recalls a teenage Demis pointing at the top boards and insisting they were better than all those players.
- Other mind games. To the chess world's surprise he took up bridge, backgammon, Diplomacy, draughts and shogi, reaching the top of the English shogi rankings. Mallaby finds his versatility more striking than his chess.
- Isolation. At his local state school he felt "like this alien": Asian looks, a foreign name and outlandish intelligence. Around ten he skipped school for a year for chess, studying from textbooks at home; at fourteen he dropped out again, teaching himself two years of syllabus at double speed to sit GCSEs early.
- Mallaby's inference. Social isolation probably fed the manic drive: without friendship as scaffolding, affirmation came from extraordinary achievement.
- The switch at fifteen. A special class for a handful of elite maths pupils gave him his first real friends (a diplomat's son from Nigeria, a boy of Indian descent, two Jewish pupils, and himself), and showed him that abstract mental play could bond people rather than isolate them.
Computing: from prize money to Levy's handbook
| Year / age | Machine or discovery | Significance |
|---|---|---|
| Age 8 | ZX Spectrum 48K, bought with chess prize money | Loved it "from the moment" he unwrapped it |
| Age 12 | Commodore Amiga 500 | Far more power; platform for his first AI program |
| Age 12 | David Levy's The Chess Computer Handbook, found at Foyles (30 miles of shelves), read in one sitting | United chess and computing; introduced Shannon |
David Levy, a Scottish international master, surveyed computer chess as it stood in the early 1980s and gave Hassabis the themes of his life's work.
Shannon's wedge
- The 1950 paper. Claude Shannon's "Programming a Computer for Playing Chess" admitted chess programs had no practical value but argued mastery of complex games could act as a wedge into similar, weightier problems: translation, military strategy, music composition.
- Writing for a machine that did not exist. Shannon devised instructions for a computer not yet built, en route to a "modern general-purpose computer."
- Numerical versus general. A numerical computer is a calculator following rigid programs on questions with clear answers. A general computer handles subjects needing more than logic, such as judging a chess position or linguistic nuance, using general principles, something like judgment, and considerable trial and error: it learns from examples rather than only deducing.
Levy's verdict on machine chess
- Speed, not flair. Silicon signals travel far faster than neurons, so computers could calculate several moves ahead and already beat top humans at speed chess.
- No intuition. They lacked the flashes of brilliance that decide longer games. Levy predicted they never would, but that rising processing power would still beat grandmasters by the end of the century.
- Confirmed. IBM's Deep Blue beat Garry Kasparov in 1997, by brute force rather than ingenuity, vindicating Levy's view that machines struggled to match human flair. (Hassabis's later career is, in effect, an attack on the "no intuition" half of the prediction.)
Levy's three building blocks
| Levy's block | How it works | Later echo in DeepMind's work |
|---|---|---|
| Representation | Make the board legible as numbers: each piece a value (pawn 1, knight 2, and so on), white positive, black negative | How a system "sees" a game |
| Evaluation | Score a position from weighted factors such as material, mobility, centre control; e.g. if material is twice as important as mobility, double its score before adding | The value function |
| Tree search | Explore each move, the reply, the counter-reply; prune branches leading to low-value positions because branching grows exponentially; play the move with the best reachable outcome | Search in game-playing AI |
On the Amiga, Hassabis found even a pruned chess tree choked the machine, so he switched to the simpler game Othello. That meant redesigning all three blocks (representation, evaluation, search) for a new domain. Using musical facility from his father he added a soundtrack, recoloured the counters red and blue, and named it Fire and Water. It beat his brother George, then five but well tutored; Hassabis counted it a famous victory.
The split Levy drew (fast search, no intuition) no longer holds. Since 2020 the leading open-source engine Stockfish has used NNUE, a small neural network that evaluates positions, layered on classical tree search: learned "judgment" plus brute calculation. That hybrid descends from the neural-net-plus-search approach DeepMind popularised with AlphaGo (2016) and AlphaZero (2017), which later chapters cover.
Bullfrog: the apprenticeship
Getting in
- The contest. At almost fifteen, reading magazines in a shop until staff moved him on, Hassabis saw an Amiga magazine ad: Bullfrog, one of Europe's top studios, would consider job applicants who produced the wackiest twist on Space Invaders.
- His entry. Chess Invaders: the player's avatar in the middle of a chess-like grid, attacked by chess-piece enemies advancing in formation. It was listed as a runner-up months later, not enough for a job, so he phoned the company and got an invitation to visit.
- Peter Molyneux. Bullfrog's lanky, big-talking founder invented whole genres, notably "god games" in which players steer the fates of crowds of digital characters. He also had a wild streak: he shot BB pellets at young game testers, and when a designer complained about his bonus he hurled a heavy object that missed and shattered the piranha tank; a new recruit arrived the next Monday to dead fish and broken glass in an empty office, the team having flown to America.
- The visit. Bullfrog had moved from its attic to a research-park building in Guildford (carpets, window cleaners, staff skateboarding down corridors). Hassabis recalls skipping off the train feeling he had reached heaven.
- First impression. Staff found him lovely and phenomenally bright; he was asked to stay a week at a desk next to Molyneux, and the two got on famously.
- The culture. Self-taught, wild young men, many school dropouts; illustrators who learned their craft spraying graffiti on trains; a gifted coder, Sean, who Hassabis suspects was in a gang. He loved it.
Theme Park
The following winter Hassabis won a place at Cambridge to read computer science, but at sixteen was judged too young to enrol. He finished school a year early and returned to Bullfrog for Molyneux's genre-busting new title, Theme Park, in which players build rides, food stalls and ice-cream stands and manage entertainers, mechanics and guards. Not everyone believed in it: one artist said he quit to make a decent game.
- The commune. Hassabis and a few young staff lived in Molyneux's rambling old rectory (hidden doors, secret passages, gaming kit), coding day and night with spaghetti-and-ideas sessions in the kitchen; work and philosophising blurred into a thrill of unbridled creation.
- The technical stretch. Early-1990s games ran on finite-state machines: characters toggled between a handful of states (run, attack, eat). Molyneux demanded the architecture be pushed to the maximum so visitors had cravings, tastes, thrills, nausea, nerves, happiness and sadness.
- Hassabis's details. Extra salt on the fries made visitors thirsty and soft-drink sales soar; coasters too scary made riders vomit, too tame disappointed thrill-seekers.
- Office politics. Molyneux would drive him in to demo new features, then vanish with him again. Older programmers grumbled that "Peter has a new pet," but no one disputed the quality of the work.
The ideas that made the mission
Hofstadter: patterns, not stuff
Molyneux gave him Douglas Hofstadter's Gödel, Escher, Bach, the Pulitzer-winning idea fire hose (fugues, logic, recursion, meaning, DNA, AI, consciousness, free will) that has inspired a striking number of AI scientists. At seventeen, Hassabis was most taken by its passages on intelligence and consciousness.
- The question it answered. As a chess player he wondered how his own brain formed moves and why it erred.
- Brains are physical. Hofstadter treats the brain as biological matter obeying the same laws as everything else, computers included, running on trickles of electricity, the chemical equivalent of ones and zeros.
- Biochauvinism. Belief in an ineffable, unprogrammable essence (spirit, consciousness) inside the skull is, for Hofstadter, mere biochauvinism.
- The concession. People feel a unique "I-ness" that roots the sense of free will; Hofstadter admits the claim will disturb readers.
- The key line. What matters is not the stuff brains are made of but the patterns that arise in it. Hassabis's inference: if patterns are what count, similar patterns can be encoded in artificial brains, so computers should eventually do whatever minds do.
- Setting. He was absorbing this while turning coding into a well-paid craft, away from home, among rebels and a mentor who talked about AI constantly: how it could improve games and what it would take to build it.
Science fiction: the benign-AI vision
- Asimov's Foundation. Hari Seldon foresees the Galactic Empire's collapse and plans to soften it; Hassabis later reasoned that only AI could give humans that power to predict and avert disasters.
- Iain Banks's Culture novels. A post-scarcity, interstellar society run by intelligent machines: economic abundance, space travel as easy as a London bus, hundreds of planets to choose from, and AIs too busy with their own intrigues to fight humans. The lesson Hassabis drew: AI need not be feared; it would enrich human experience.
Halfway through Theme Park, Hassabis went with Molyneux to an AI conference in the United States. A Carnegie Mellon professor presented the state of the art in lifelike agents: a video of three bouncing blobs labelled "Bear," "Dog" and "Mouse" (the bear protected the mouse, the dog chased it). Afterwards the pair opened a laptop and showed the Theme Park demo, with simulated happiness, sadness, thirst, hunger, money and friendships across a whole park. Outcome, in Hassabis's telling: the professor "fell off his chair" and asked who they were. A teenager without a degree had outdone academic AI.
Mallaby assembles the answer to the post-Liechtenstein question from these strands: Molyneux and Hofstadter planted the idea that computers would do whatever brains do; Banks supplied the utopia of boundless flourishing; the CMU professor proved Hassabis had the talent. Hassabis says he then resolved to devote his career to AI and already held "the kernel" of what became DeepMind. Mallaby ends by noting it would take one more epiphany to seal that destiny (the subject of Chapter 2).
Most of the chapter rests on Hassabis's own recollections: Mallaby flags that the Liechtenstein game is told "the way he remembers," the opponent is unnamed, and the CMU professor is unnamed too. Hassabis also chose the Ender framing and asked Mallaby to adopt it. The result is a polished founder's arc (suffering, epiphany, mission). Mallaby is candid about the father's temper and the isolation, but the claim that DeepMind's "kernel" existed at seventeen is retrospective and unverifiable, and the uplifting reading of Ender skips the novel's darker theme: a child manipulated into mass destruction he did not fully understand, which sits awkwardly with the book's Oppenheimer question.
"Deep Philosophical Questions"
At Cambridge (1994-97) Hassabis and his friend David Silver concluded that symbolic, rule-based AI was a dead end: real intelligence is mostly induction, extracting patterns from messy data, and nobody knew how to program it. Tutorials with John Daugman then gave Hassabis the second epiphany Chapter 1 promised. His two-part credo: information is the fundamental unit of reality, and a machine that learns to find nature's patterns for itself is the most powerful tool for understanding it.
Choosing Cambridge over £500,000
In autumn 1994 Hassabis left Bullfrog for Cambridge. Peter Molyneux tried to keep him with a cheque for £500,000 to work on the next game, Dungeon Keeper: roughly $1.7 million at current prices, offered to an eighteen-year-old. Hassabis never cashed it.
- The film behind the choice. Life Story dramatises James Watson and Francis Crick at Cambridge: walks by the Cam, plotting fame in the dim Eagle pub, and cracking the double helix with help from Rosalind Franklin's X-ray image, then the Nobel. Crick's answer to what is fun (the big questions of what man and life are) captures the appeal.
- A magpie mind. Having learned outside school and beyond his parents' understanding, Hassabis built his worldview from encounters like this film and Banks's novels. He calls himself "indiscriminate about knowledge": chess games, books, philosophy, all absorbed.
- Mallaby's reading. Of the forces pushing him to AI, the strongest may be the thrill of science itself: finding the truth behind other truths.
Why not physics
After Liechtenstein he had seriously considered theoretical physics, the field of the biggest questions, where a unifying theory might fuse siloed knowledge in the way Copernicus and Einstein had once displaced older pictures.
- The practical objection. He liked games he could win, and physics looked like a long shot: every physicist since Einstein had failed to explain all of reality. Even Feynman died without understanding everything, and Hassabis doubted he would surpass Feynman.
- The strategy instead. Attack physics' mysteries indirectly, with AI. Science advances through tools: telescopes extended sight into space, X-rays into the body. AI would be the ultimate lever, an extension of understanding itself.
- The plan at arrival. He came to Cambridge scouting Nobel-scale problems to crack once AI existed, picturing himself plotting in the Eagle like Watson and Crick.
Hassabis did not pick a discipline; he picked a meta-tool. Rather than compete with Feynman on physics, he would build the instrument that could do physics (and biology, and medicine) better than any individual. That bet, made at eighteen, is the through-line of DeepMind.
Not a one-track genius
Those expecting a monomaniac were disappointed.
- First year. Raving with friends, then collapsing at dawn to The Prodigy's Music for the Jilted Generation.
- Second year. Joyrides in a Porsche 911 Turbo, borrowed from Molyneux on the pretext that he needed it to commute to Bullfrog as a consultant.
- Love. He fell for an Italian undergraduate, later an academic bioscientist, whom he would marry.
- Campus legend. Students gossiped about his chess, his co-creation of Theme Park (millions of copies sold) and the car. Yet what peers remember is affability: a "nice kid" whose conviction that you would get along tended to come true.
The roots of affability: two readings
Hassabis's account. A "very deep" personal philosophy from his Christian mother, whose faith rescued her from an orphan childhood and who helped the poor and lonely through her church. He attended Sunday school, played flute in the church band, prayed at bedtime and helped with charity work. Add temperament: he wants to help people and believes it is really bad to manipulate or control them.
Mallaby's account. Small and young-looking (in his early twenties he was regularly told he looked ten), he had reason to avoid confrontation, and he had another route to respect: beating everyone at games. Gentleness and ferocious competitiveness are two sides of one coin.
- Table football. An everyman bar game that signalled approachability, yet he organised a college team, beat rivals across campus, and calls himself pretty much the best player at Cambridge, with a controlled left-handed midfield shot he says even US professionals lack.
- Go. Weekly bike rides to a professor's home for afternoon lessons in the ancient Chinese game of territory; he was good enough that the professor printed some of his games in a handbook for Go students.
- Proteins, filed away. Between table-football bouts a biologist acquaintance mentioned that protein shapes, if plotted accurately, would unlock medical breakthroughs. The friend has no memory of the exchange; Hassabis stored it as a potential double-helix-type challenge. About 25 years later, DeepMind solved it and won a Nobel for transforming structural biology.
Silver and the case against symbolic AI
David Silver became his closest intellectual partner. Both were sunny and friendly, and each knew of the other in advance: Silver remembered an intense young Londoner winning chess tournaments in his hometown of Ipswich and taking the prize money; Hassabis knew Silver had the highest exam results of any Cambridge computer science undergraduate. They bonded over "the big debates about AI" and the deep philosophical questions of computer science.
The Dartmouth inheritance
- 1956 Dartmouth workshop. AI's founders set out to make machines use language, form abstractions and solve human-level problems, on the premise that every feature of intelligence could be described precisely enough for a machine to simulate.
- Era of reason. Mallaby links this to midcentury faith in rationality: the rational agent in economics, the efficient-market hypothesis in finance, "scientific" management in firms. Intelligence was assumed to be deductive, describable and programmable.
- Symbolic AI. For about half a century the standard method: choose symbols for real-world things, feed in facts, add logical rules (and, or, not-and), and deduce. Green and blue symbols are not the same colour; Socrates is human, humans are mortal, so Socrates is mortal: everything reduced to syllogism.
- Levy again. Levy's handbook, turning chess positions into numbers, had been symbolic programming.
- Cyc, 1984. The same year as Levy's book, symbolic AI peaked with Cyc, an attempt to encode most of human common sense, with rules like you can't be in two places at once and hold a coffee cup open end up.
Why it fails
- Exceptions everywhere. For every rule there are myriad exceptions; knowledge does not decompose into discrete axioms, and understanding is not only deduction.
- The categorisation puzzle. A butter knife and carving knife belong together by function; a golden retriever and dachshund by species (they can breed fertile offspring). No rule tells a machine which categories run on appearance, function or reproduction; symbolic AI had no answer.
- Cambridge curriculum. The pair were taught first-order logic, e.g. "all birds can fly" as ∀x (Bird(x) → CanFly(x)). Hassabis thought using it to understand language was obviously nonsense: people speak ungrammatically and still understand each other.
- Gödel. Both had read Gödel, Escher, Bach; Kurt Gödel's incompleteness theorem shows no deductive system can capture all true statements, contradicting Dartmouth's premise and confirming their intuition.
Deduction (symbolic AI). Hand-coded rules and logic yield provable, unambiguous truths, but they are brittle: inflexible rules in inflexible machines. Humans use deduction only a small fraction of the time.
Induction (pattern recognition). Learn general truths from examples, the way humans digest jumbled images, words, smells and sensations. Conclusions are revisable (watch ten New Yorkers and you might conclude all humans drink coffee; millions across cultures correct it) and need vast data.
The problem of induction
Because induction needs many examples, a learning machine must ingest as much data as possible, and it then runs into its computational limits. It needs a way to decide what to attend to: the defining challenge of AI. Humans use heuristics, mental shortcuts, but in the mid-1990s nobody could define or code them. Philosophers call this the problem of induction, and Hilary Putnam of Harvard quipped that AI had failed for a quarter century at what philosophy had failed at for two millennia.
Hassabis and Silver could not program induction either, but they had named the right problem. Flexibility is the essence of intelligence; inflexibility stacked on inflexibility never produces it. What was needed was a new kind of machine that discovers patterns in a near infinity of data: Mallaby's "infinity machine," the source of the book's title.
The Daugman epiphany
In their third year the pair talked John Daugman into a series of tutorials in his office. Daugman taught information theory and computer vision and became famous for inventing the iris-recognition algorithm. Hassabis remembers the sessions as "nirvana sessions"; Daugman liked the pair because, unusually for computer scientists, you could actually talk to them.
Part one: information is fundamental
- The shift. A superhuman computer need not be only a means to scientific progress; it might be the end, because information is the basic unit of reality.
- Rival candidates rejected. Energy and matter are less compelling; information can explain everything, from particle behaviour and energy flow to human consciousness, all as information processing.
- Hofstadter's germ. GEB had already argued biology is information processing: life is defined by signals, not muscle or tissue.
- Shannon's definition. With Daugman he studied how information is quantified, stored and transmitted, and Shannon's core insight: information is the opposite of uncertainty.
- Implication. If reducing uncertainty requires processed information, then a theory of everything (uncertainty near zero) would probably take the form of a computer program. Hassabis still sees the universe this way.
Part two: machines that design their own computation
- Shannon's level two. For Shannon, what sits on top of information is computation: sifting, moving and deriving meaning from it. He theorised computers before they existed because they needed to exist.
- The 1990s impasse. Nearly half a century later computation seemed stuck, because humans could not hand-code induction.
- Level three. Move up again: machines that work out their own computation, programs that design programs. If humans lack the wisdom to teach induction, the infinity machine will teach itself. Like Shannon, Hassabis and Silver reasoned such systems needed to exist, so they would.
- Mathematics for physics. A deductive system suits physics; Newton captured motion in a few equations.
- AI for biology. Biology is messy, emergent, dynamic and complex; nothing as elegant as Newton's laws will describe a cell.
- The method. Feed an inductive machine enough cell data and it may find the hidden laws of cellular behaviour that no human would write down.
- The payoff list. New medicines and longer lives; solving the obstacles to nuclear fusion for clean, abundant energy; in Hassabis's words to The Guardian, potentially "a meta-solution to any problem."
Black & White and the break with Molyneux
Graduating in 1997, Hassabis considered a year in Japan studying Go but instead joined Molyneux's new project, Black & White (developed at Lionhead Studios, which Molyneux founded after leaving Bullfrog in 1997; released 2001).
- The game. A "god sim": players with divine powers choose to be good or evil, and the world reflects it (a darkening world for evil, angelic chirping for good). Either way the task is to win believers; Molyneux, the creative impresario, held that a deity is nothing without followers.
- The AI draw. Theme Park's characters were complex finite-state machines with fixed preferences. Black & White would be the first game whose avatars changed their internal rules from feedback: a creature slapped for throwing rocks at villagers stops doing it; one patted after gorging eats more.
- Mallaby's label. A basic "reinforcement learning" system: a program that adjusts its own program, a small step towards AI. It also kept the game fresh, and the title became another Molyneux hit.
- Hassabis's role. He joined the early brainstorming but did not stay to build it.
Older now, Hassabis saw what he had ignored at fifteen. Molyneux was a fabulist who promised journalists technical feats his coders had told him were impossible. With Hassabis, now a protégé and potential rival, he hinted at a secret new path to AI but never showed evidence, swinging between warmth and iciness, bravado and tears, emphatic, vague, elusive and menacing by turns. Hassabis later compared him to the manipulative illusionist in John Fowles's The Magus. Mallaby casts the two in the game's own terms: Molyneux the black god ruling through manipulation and menace, Hassabis the white god ruling through infectious enthusiasm and clarity. Hassabis repeats that being controlling is the worst thing you can do to someone.
The company idea
Near the end of Cambridge, Hassabis told friends he would found a company to build AI. On a campus where students asked only whom they would work for or which PhD they would do, this was startling; Britain had no Silicon Valley equivalent, and a contemporary says nobody would have been picked as a likely founder. Mallaby credits Molyneux's example and perhaps his free-spirited father for making Hassabis the exception.
1994 Turns down Molyneux's £500,000 cheque; starts computer science at Cambridge.
1995-96 Second year: the borrowed Porsche 911 Turbo, consulting for Bullfrog.
1996-97 Third year: Daugman tutorials with Silver; the information epiphany.
1997 Graduates; skips Go year in Japan; joins early work on Black & White, then leaves to found a company.
The Black & White creature AI is usually credited to its lead AI programmer, Richard Evans (later a DeepMind researcher), not Hassabis, who left after early brainstorming; "reinforcement learning" is a loose label for feedback-adjusted preferences rather than the formal RL Silver later pioneered. The protein-folding conversation is remembered only by Hassabis, a typical piece of retrospective destiny. Mallaby's own reading of the affability (a small man's alternative to confrontation) is speculative, though he labels it as such. And the "infinity machine" framing is Mallaby's synthesis: in 1996 the pair had a diagnosis, not a method.
The Jedi
Hassabis's first company, Elixir Studios (1998-2005), showed what his charisma could do and what it could cost. His "Jedi mind trick" recruited talent, charmed investors and rescued a crashed demo, but it also inflated expectations until his team stopped giving him honest feedback, and it burned out his closest friend, David Silver. Elixir's failure sent both men back to study: Silver to reinforcement learning under Richard Sutton, Hassabis to a neuroscience PhD on the premise that the brain is the "existence proof" for general intelligence.
Founding Elixir
The decision and the team
One night in early 1998, at his parents' North London home, listening to the soundtrack of his favourite film Blade Runner, Hassabis felt the weight of leaving Molyneux and steadied himself with "You only get one life." Next morning he called David Silver, who had half-believed the Cambridge talk of a company and wondered why Hassabis would leave a boss who worshipped him. Silver, then building film special effects at a software boutique, agreed on the spot.
- The name. Elixir, from a dictionary definition ("the quintessential part of any substance") that Hassabis admitted in his monthly magazine column, the "Elixir Diaries," he did not understand but liked the sound of.
- Mallaby's reading. Raised on stories in which he cast himself as the hero, Hassabis was now writing his own story.
- Demis Hassabis. CEO and keeper of the vision; did little of the coding.
- David Silver. Cambridge friend; built the game's character AI.
- Joe McDonagh. Designer who had applied to Molyneux with a tea-stained message in a bottle from a castaway on the island of "Korporate"; listed hobbies origami and boxing. Hassabis interviewed him by challenging him to games, beat him at folding an origami bird, and wisely skipped the boxing.
- Tim Clarke. Cambridge acquaintance into coding, theoretical physics and weightlifting; as a schoolboy wrote a Mars flight simulator that won him a NASA summer job.
The Jedi mind trick
Silver's name for Hassabis's persuasive power: he would project "You will believe the things I'm going to say," and people did. Mallaby credits the Molyneux apprenticeship: entrepreneurship spreads in clusters like Silicon Valley, where founders learn to project confidence from one another, while Britain was a start-up desert. Hassabis was lucky to have trained under a master storyteller.
Raising money in a start-up desert
Games studios normally took money from publishers (cash up front for a large share of future revenue). Hassabis tried the Valley model instead: venture capital, pitched with Theme Park sales figures, a detailed plan and the team's records.
- The first pitch. In a stuffy suit in the City, two young associates took him to a restaurant and ordered three bottles of wine; hours later they took him to their boss in a pub, where lager followed. Hot, drunk and exhausted, Hassabis talked chess; the impressed boss offered him a job as a currency trader, telling him to think bigger than the start-up.
- The term sheet. Days later, a letter: £2 million for 50% of the equity. Mallaby notes Silicon Valley's first VCs in the 1960s offered terms about as brutal, so it was not outlandish.
- Refusal. Having hated being controlled by Molyneux, Hassabis would not cede control. After a dozen similar outfits he gave up: they liked the team but "wanted our soul."
- Running on empty. Savings from Molyneux days nearly gone, living with his parents, taking the bus (his mother's car's clutch broke).
- Angels instead. Over a couple of months an industrialist, a lawyer and Molyneux himself put in small sums. He rented a windowless unit near a motorway junction, and Elixir was founded on July 7, 1998.
Why a games company at all
The Cambridge ambition had been powerful AI, not games. Elixir was a compromise between ambition and practicality: a place to experiment with AI, gain entrepreneurial experience, and perhaps get rich enough to fund his real dream, a Manhattan Project for AI.
- Office life. A corrugated-iron roof, no ventilation, recycled school chairs; debates over whether scrambled eggs were an air-quality crime, whether Spock should have captained the Enterprise, and footballers' worst haircuts; fantasy football and StarCraft to feed Hassabis's burning need to win at something.
- Mind Sports Olympiad. A month after founding, he won the five-game event for the first time, beating some 2,000 rivals at a West London hotel by sprinting between rooms to play several games at once.
Republic: The Revolution
Concept
- Origin. In early 1995 at Cambridge he had played an obscure banana-republic power-struggle board game while news was full of Russia's invasion of Chechnya. The pairing made him ask who the people who make history are and how they become so.
- The twist on god games. A dictator game: the player is a faction leader who topples the ruler by demagoguery, political intrigue or brute force, with millions of lesser lives at stake.
- Research. McDonagh read up on Russia and Soviet satellites at the British Library and in the shabby reading room of the Society for Anglo-Soviet Cooperation in South London, where he assumed the other readers were spies (and that they assumed he was).
- Novistrana. After months of reading he invented a fictional country blending Belarus, Ukraine and Azerbaijan: a dictator, ample corruption, Eastern Orthodox churches.
Ambition and money
- Demands. Echoing Molyneux, Hassabis wanted legions of believable citizens (husbands, students, housewives, drunks), each with a separate life, and high-definition graphics showing moss in the cracks of buildings.
- Publisher funding. By the end of 1998 progress won a package from publisher Eidos: the conventional source he had tried to avoid.
-
Camden. A bigger office with curving roof beams that McDonagh and Clarke swung along chimpanzee-style. Hassabis installed table football and announced nobody would ever beat him.
-
"My soul is on fire." His lieutenants trained maniacally until one day they beat him. Hassabis sat silent under a visible dark cloud, then stood and declared that it felt like his soul was on fire: in Silver's view both serious and self-mocking, meant sincerely and delivered for a laugh.
Building it (1999)
- Art direction. Cities should be imposing, the man in the street an ant. Hassabis ran screenings of Metropolis, Batman and Blade Runner.
- Graphics. Clarke's photorealistic renderer showed even the screw threads on factory bolts; Hassabis marketed it as the "infinite polygon engine", which rivals mocked as the "infinite monkey engine."
- Characters. Where Theme Park's visitors barely differed, Silver's algorithms gave Republic true individuals who developed through interaction: sleazy councillor Ludmilla Mironova, sleazier journalist Eduard Satarov (beer gut, comb-over). Hassabis promised 1,500 differentiated characters plus a million inhabitants with routines, beliefs and loyalties.
- Hype. At the end of 1999 Edge ran a gushing cover calling Republic one of the most ambitious games ever; Hassabis said he wanted to be the best games developer in the world. Mallaby hears Molyneux's ebullience.
E3 2001: the greatest mind trick
Unveiling was set for the Electronic Entertainment Expo, Los Angeles, May 2001: 55,000 developers, publishers, retailers and journalists, 5,000 teams spread over an area of about eight football pitches, and a reception that would decide the game's fate. Hassabis's rule: if rivals work 15 hours a day, we work 16. Clarke coded until he collapsed on a sofa, then washed at the sink and resumed.
The demo choked even though the team had secretly fitted a normal-looking PC with eight times the normal memory. Silver coded all night, Hassabis carried the computer to the airport, they sprinted to the gate, and the machine flew in its own purchased seat while Silver kept coding. At the first and most important session, with the publisher's boss watching, the system crashed and rebooted with its inflated memory displayed on screen. Silver bolted (he still cannot bear the demo music) and slept for hours on a sofa. He returned to apologise and found none was needed: Hassabis had carried on with a slow-motion demo and brilliant patter about ordering beatings, plotting a student's martyrdom and simulating riots, citing Elias Canetti's Crowds and Power. Outcome: Republic won best upcoming game, and a reviewer called it the most exciting strategy concept since Civilization.
Over-inspiration and burnout
The crash exposed the underlying truth: contemporary hardware could not carry the vision. Round-the-clock coding was burning people out; Silver's flight from the room was a warning, and others were in similar shape. The mind tricks had raised talent, money and buzz, but also expectations to the point where delivery became almost impossible.
- Slippage. Over the next couple of years the release date was pushed back repeatedly; to ship, the team diluted the most innovative ideas, and morale sank while Hassabis fought a rearguard action against compromise.
- The trap he later named. You can inspire people too much, until engineers say things are possible only because you over-inspired them, and "none of us is getting real feedback."
Silver's exit
Republic finally shipped in August 2003 to an anticlimax. Then Silver quit, raising the harder question of whether Hassabis, beyond over-inspiring people, was harming them.
- The go-between. For about a year, coders and designers had asked Silver to tell Demis a feature would not work, since he was the only one Hassabis would hear.
- How bad news landed. Silver had to escalate until Hassabis defended his position ever more intensely; the stronger the resistance, the closer you were; finally Hassabis would go quiet, the sign he had absorbed it.
- Aftermath. Silver moved to the south of France with his girlfriend and spent a year recuperating.
- Silver's fairness. He insists Hassabis's intentions were good and his horror of manipulation sincere; but abhorring control and avoiding it, with Jedi-level charisma, are different things.
Elixir's end and Mallaby's comparison
Elixir went out of business in 2005; Hassabis was burned out too (a friend thought "Demis is killing himself"), and he calls it probably the hardest time of his life: "I was in pieces."
The common pattern. Steve Jobs, Elon Musk and Molyneux all bent reality with stories, moving the line between possible and impossible, with results both thrilling and harmful; Molyneux could be inspiring and abusive.
Where Hassabis differs. Mallaby judges him more self-aware and keener to correct himself. Early at Elixir he admitted being strong-willed and stubborn while trying to accept that his ideas were not always best, and he rarely pressed others harder than himself. But a boss with Ender-like intensity is bound to be tough on those around him.
The psychologist's correction
A psychologist made Mallaby revise his Black & White image of Molyneux as dark god and Hassabis as benign one.
- No purely bad charismatic. One who is always bad alienates everyone, and a charismatic with no entourage is a contradiction.
- No purely benign one either. When interests collide someone must prevail, and the charismatic will end up on top however much they wish not to dominate.
- Oscillation is the hook. Swinging between inspiring and controlling is part of the charisma itself. Followers get hooked on variable moods the way gamblers get hooked on slot machines with unpredictable jackpots; a consistently loving parent produces no anxious clinging. This is exactly the dynamic Hassabis experienced with Molyneux and rejected.
- Mallaby's open question. Hassabis's intentions are good, but can he stay good under the adversity an infinity machine will bring? In Black & White the god's moral compass coloured the whole world; with AI, Hassabis's character will have consequences for society.
Mallaby skips Elixir's second release, Evil Genius (2004), a comic game about a megalomaniac building a doomsday lair: the very title Elon Musk later cited when casting Hassabis as a villain. Elixir's closure is presented as a personal crisis without the commercial detail (sales, Eidos's own troubles and sale in 2005). The account of Silver's exit comes mostly from Silver and Hassabis, now friendly; other Elixir staff appear only as anonymous complainants, and the psychologist is unnamed.
Two roads back to learning
Silver: reinforcement learning
In France Silver read the textbook of University of Alberta computer scientist Richard Sutton on reinforcement learning (RL). It matched his Cambridge conviction that inflexible logic in inflexible machines would never yield flexible intelligence.
- The idea. Machines learn by trial and error, as Shannon suggested: act in the world, receive feedback (reward), and fine-tune behaviour, effectively programming themselves.
- Already familiar. Hassabis had proposed a simple form for Black & White (slaps and strokes as feedback); Elixir had invited an RL professor to speak, leaving Silver "gobsmacked."
- What Sutton added. Concrete algorithms to make RL real. Silver's conclusion: "That's real AI."
- Silver's definition. AI is often vague, but Sutton pinned it down: a system that learns for itself how to solve problems, without human instruction; a road stretching to the horizon.
- The move. Apologising to a girlfriend who preferred the French climate, he wrote to Sutton and began a PhD in Alberta in 2004.
Hassabis: the brain as existence proof
- The logic. He had rejected symbolic AI because it was not how human intelligence works, and believed powerful AI would emulate the human kind. The brain is the only evidence that general, flexible intelligence is possible: the "existence proof." So understand it first.
- The gap. Not a doctor, biologist or psychologist; little knowledge of brain chemistry. He leaned on his computer science credentials and exotic business record and won a place in the neuroscience PhD at University College London (UCL).
- Supervisor. Rising star Eleanor Maguire feared a former CEO would find academia dull; he assured her, truthfully at the time, that he did not need to be in charge. She assigned a pile of papers.
- Honeymoon homework. In summer 2005 he married his Cambridge girlfriend and took the reading to an Italian beach.
Memory, imagination and the constructed world
The beach eureka
Memory as video. The brain stores faithful recordings and recalls them; memories reflect what really happened.
Memory as reconstruction. The brain keeps only a bare essence and rebuilds the past each time, leaving room for invented detail. Hassabis found this obviously right, persuaded by witness studies in which subtle prompting by police produced the "memories" the questioner wanted.
His insight before even starting: if memories are reconstructed, memory and imagination may share the same brain machinery. Their goals differ (real events versus deliberate novelty; past versus future), but both build vivid scenes. He recalled the White Queen in Through the Looking-Glass: a memory that only works backwards is a poor sort. Studying imagination might settle the memory debate.
The experiment
- The partnership. Fellow PhD student Dharshan Kumaran, his junior-chess friend, had since qualified as a doctor and knew brain physiology far better. Hassabis brought "crazy ideas" and a hunt for the game-changing one ("the details, whatever"); Kumaran supplied rigour, testing ideas in the neuroscience building's tiny kitchen until they held.
- Design. Test patients with damage to the hippocampus, the deep-brain structure that records the day's memories. If memory and imagination are linked, amnesic patients should struggle to imagine.
- Patients. With Maguire's help they found a handful of rare patients with hippocampal damage but otherwise healthy.
- Result. As predicted, damage impaired the ability to visualise a coherent 3-D scene, such as a day at the beach or a shopping trip: a memory structure is essential to imagination.
- Published. Proceedings of the National Academy of Sciences, early 2007 (Hassabis, Kumaran, Maguire).
- Recognition. Science named it among the year's top neuroscience breakthroughs.
- Impact. Eventually over 1,700 citations.
The philosophical payoff
For Hassabis the accolades mattered less than the implications. If memories are simulations generated by the brain, not records of external reality, perhaps all of reality is a mental construction; he says he was trying to prove reality might be a simulation.
- Kant again. The world out there is basically a mental construct; matter, energy and time are less real than the information pulsing between neurons.
- Brain in a jar. A brain kept alive in nutrients and fed simulated sensations (sea views, waves, salt and seaweed) would believe it lived normally; so how can anyone be sure they are not in a simulation? It is also the premise of The Matrix, another favourite film.
- Physics → AI. Curiosity about physics led to AI, the ultimate tool for science.
- AI → brain. Curiosity about AI led to the brain, the existence proof of intelligence.
- Game simulations → mental simulations. Simulated worlds in games echo simulated worlds in the mind.
- Common bottom line. Kant, GEB, Daugman and neuroscience all point to information as the fundamental unit of reality.
- The theological edge. The Bible's God controls all reality; Black & White's god controls a simulated one. If simulations are as real as reality, a computer simulating a small world is a limited god, and a powerful AI might be limitless, infinite.
Early 1995 Banana-republic board game and Chechnya images seed Republic.
Early 1998 Blade Runner night; Silver signs on.
July 7, 1998 Elixir founded, angel-funded; a month later Hassabis's first Mind Sports Olympiad pentathlon title.
End 1998 Eidos funds Republic; move to Camden.
End 1999 Edge cover story.
May 2001 E3 crash; best upcoming game award.
August 2003 Republic ships to anticlimax; Silver quits.
2004 Silver starts PhD with Sutton in Alberta.
2005 Elixir closes; Hassabis marries, starts UCL neuroscience PhD under Maguire.
Early 2007 PNAS paper on amnesia and imagination.
Sutton and his mentor Andrew Barto received the 2024 Turing Award (announced March 2025) for founding reinforcement learning. Silver stayed at DeepMind for two decades of RL work before leaving in January 2026 to found his own lab, Ineffable Intelligence, betting that language models alone will not reach superintelligence (see Chapter 20).
The Gang of Three
In 2009-10 Hassabis found the two partners who made DeepMind possible: Shane Legg, a New Zealand mathematician who had coined "artificial general intelligence" and believed computing power was about to explode, and Mustafa Suleyman, a dropout activist-entrepreneur and friend of Hassabis's younger brother. Legg's independent arrival at the same mission was, for Hassabis, the corroboration he had lacked: "there was one, and that was enough." Suleyman added a social mission, commercial hustle and the first push to find a billionaire.
A lonely contrarian, 2009
Choosing the direct route
In early 2009, as Hassabis finished his PhD, Peter Molyneux sent an emissary to recruit him for his next game. Asked why he declined, Hassabis said he wanted to be the person who solves AI.
- Stepping-stone ideas rejected. He toyed with Elixir-style hybrids: an AI recommendation engine for TV shows, or one for Apple's new App Store. Neither excited him.
- The clock. At thirty-two he felt time pressing; it was time to pursue the life ambition directly.
The Gatsby Unit
His first step was a postdoc at UCL's Gatsby Computational Neuroscience Unit, which sat where human and machine intelligence meet.
- Peter Dayan. The director had shown that the reward signals of reinforcement learning mimic the dopamine signals that reward human learning (the RL branch Silver had chosen).
- Geoffrey Hinton. The founding director had pioneered neural networks: layers of loosely brain-like decision units, or "neurons," wired to one another. The approach, later famous but then obscure, was called deep learning.
- Cold reception. Most faculty dismissed his ambitions as far-fetched; they wanted RL or deep learning to work on a few simple problems, not superhuman intelligence.
- The one bio. Scanning postdoc profiles on the website, he found exactly one who admitted wanting to build powerful AI: Shane Legg, a name with "a cowboy ring." He memorised the photo and hoped to meet him in the cafeteria; he did not.
America: Poggio, Hinton and a flat "nonsense"
Hassabis then crossed the Atlantic, arranging (typically) two fellowships at once, at MIT and Harvard.
- Tomaso Poggio. Physicist turned computational neuroscientist who taught MIT's oldest machine-learning class; got on with Hassabis at once. He later said most Nobel winners are brilliant and lucky in picking a soluble, consequential problem, but a handful would win regardless: Feynman, Crick and his postdoc Demis Hassabis.
- Geoffrey Hinton. Visiting from Toronto after 25 years of what Poggio called a "religious belief" in neural nets. In 2006 he and two coauthors showed how to train big deep belief networks (more size, more performance); around 2009 he gained GPUs, chips built to render video-game graphics, to turbocharge them. His verdict: Hassabis was the only person he had met more competitive than himself.
- Marvin Minsky. Elder statesman of MIT's Computer Science and AI Lab (CSAIL), housed in Frank Gehry's multicoloured funhouse building; had attacked neural networks since the 1960s, and neither he nor his disciples had updated.
- Patrick Winston. CSAIL head, the establishment pillar on whom Hassabis tested his pitch: reinforcement learning plus deep learning. After a long discussion, Winston called it "just nonsense."
- Spin on rejection. Hassabis told himself this was fantastic: establishment approval would have meant the idea was unoriginal and rivals would soon circle; Winston raised nothing new, which confirmed the plan was contrarian.
- Mallaby's caveat. A company needs believers, and a contrarian with no following is merely an oddball.
Project Orion
Back in London in late summer 2009, Hassabis wrote a business plan, Project Orion: a computer version of the human brain able to handle the functions of his PhD, memory and imagination. Mallaby reads it mainly as a measure of weakness.
- Phantom backer. It listed Molyneux as a likely investor, but Molyneux would never fund a science experiment.
- Phantom cofounder. It named David Silver, who was nowhere near ready to partner with Hassabis again.
- Net position. No capital, no brilliant cofounder, and an establishment that thought AI crazier than video games: weaker than at the start of Elixir.
Shane Legg: the AGI cowboy
The lift
At the start of October 2009, a curly-haired man with a dancer's build squeezed into a Gatsby lift with a suitcase: Legg. Hassabis asked where he was going. Legg, in a strong New Zealand accent, said New York, for the Singularity Summit: an annual gathering of AI believers. The singularity is the moment machine intelligence surpasses human intelligence, after which machines improve themselves and capability explodes upward. The doors opened and Legg left. Hassabis realised he had met someone more wired into the AI world than he was.
Legg's road to the Gatsby
1973 Born in Rotorua, New Zealand (geyser, mud pools), three years before Hassabis. Teachers suspect subnormal intelligence; an IQ assessor angrily reports he is above gifted.
Childhood Parents buy a Dick Smith VZ200 (8 KB RAM, Z80A processor); coding turns maths from mediocre to outstanding. He skips homework, argues with teachers, tops exams; lesson: write your own rules, because there is a lot of nonsense about.
University Waikato maths degree; weightlifting leads to aerobics (Waikato was an aerobic-dance hot spot), the splits, jazz dance and ballet; his father assumes he is gay (he is not), his mother is fine with it. Master's in maths at Auckland.
Late 1990s Rejects a "rather hard," "kind of pointless" maths PhD; 18 months debugging corporate databases.
Late 1999 Joins Ben Goertzel's Intelligenesis at the height of the dot-com bubble, later moving to its New York office.
Late 2001 Intelligenesis goes bust; nine months backpacking in Africa and Europe.
2002 Persuades Goertzel to retitle his book Artificial General Intelligence, coining the term.
Early 2000s Reads Kurzweil; starts a PhD with Marcus Hutter at IDSIA in southern Switzerland.
2008 Thesis "Machine Super Intelligence"; a year at the Swiss Finance Institute applying RL to predict markets.
April 2009 Gatsby postdoc, after running a weekly neuroscience reading group.
Goertzel and the global brain
- The founder. Ben Goertzel, a perpetually unkempt Brazilian American prodigy: maths PhD at 22, four textbooks by 30, posts in maths, psychology and computer science.
- The vision. In the mid-1990s he decided the internet was a new form of intelligence, a worldwide mind: each computer a neuron, each hyperlink a synapse, each user a sense organ. He founded Intelligenesis in 1997.
- Baby WebMind. The plan was to code a digital brain, release a "Baby WebMind" onto the internet and watch it grow into a self-organising machine consciousness that would end humanity's monopoly on intelligence: a precursor of Orion, a decade earlier and far wackier.
- Why Legg stayed. Viability hardly mattered; he was gripped by what intelligence is, a natural obsession for someone misclassified as a child.
- Against Turing. Goertzel scorned Turing's test (intelligence as passing for human). A successful WebMind would never sound human about sex or sunsets, and he preferred a program that discusses its computer-ness intelligently to one that pulls off a pathological liar's act.
- The naming. Goertzel's draft was titled Real AI, implying most AI was fake. Legg warned this was impolitic and suggested Artificial General Intelligence, protecting Goertzel's standing and minting AGI.
Kurzweil and the exponential
Around then Legg read Ray Kurzweil's The Age of Spiritual Machines.
- Moore's Law revisited. Chip power doubling every two years had become so familiar people stopped thinking about it; but doubling means exponential growth, flat at first, then explosive.
- The numbers. In 2000 human brains were about a million times more powerful than the best machines; Kurzweil calculated parity sometime in the 2020s, then machines would race past their makers.
- Kurzweil the man. Between visionary and weirdo: determined to survive until AI can extend life, he took 250 supplements a day plus green tea and alkaline water, awaiting indefinite extension of the "mindfile."
- Legg's extension. Data, as well as compute, would explode with the internet. With hardware and data both surging, the third ingredient, algorithms, would become insanely valuable.
Measuring intelligence
Convinced AI would be real, Legg asked what the field's biggest gap was and answered: no measure of intelligence, and you cannot build what you cannot measure. Under Marcus Hutter he collected many conceptions of intelligence and applied mathematics to measuring them. Mallaby notes that later critics accused AI's builders of skipping the distinction between intelligence and statistical facility; Legg confronted it directly.
- Reject imitation. Following Goertzel, do not measure AI by its ability to impersonate humans; flawed human cognition is an arbitrary benchmark.
- Prize generality. Following his own advice to Goertzel, the true mark of intelligence is generality.
- The formula. Intelligence measures an agent's ability to achieve goals in a wide range of environments.
- Consequence. An intelligent system is judged by breadth of competence across tasks, which later shaped DeepMind's choice to test one agent on many games.
Legg had considered his own start-up after the PhD but judged computing too immature and entrepreneurship too precarious.
The Halloween Scenario
On October 31, 2009, Legg gave a Gatsby talk with the mock-scary title "The Halloween Scenario." Hassabis sat at the back of the dim classroom.
- Content. About two hours ranging over definitions of intelligence, machine learning, spillovers from neuroscience and Kurzweil-style exploding compute.
- The turn. People would soon plug brain-like architectures into exaflop supercomputers with no idea how to handle the consequences. Early systems might be harmless, but they were converging on the kinds of algorithms that should worry us; superintelligent systems with agendas of their own might subjugate or annihilate humans. His words signalled alarm; his tone stayed excited.
- The question. A man near the front asked what should be done. Legg, twinkling, simply repeated the question back to the room; he had no answer. Nervous giggles followed.
Mallaby uses Kurzweil's summary of the Singularity crowd's emotional sequence about superhuman AI: Wow!, Uh-Oh, and What Other Choice Do We Have but to Move Forward? It mirrors patients facing a terminal diagnosis: first the urge to do something radical, then reversion to routine. Humans lack the vocabulary to process existential threats and are wired to act as if life will go on. Legg's London audience was meeting the idea for the first time, still stuck at Wow and Uh-Oh; the joke-like laughter showed how close the absurd sits to humour.
The oasis
Hassabis cared less about the threat than about what came before it: Legg had the imagination and the technical tools. This time he introduced himself. Years later, recounting it, he became emotional and said he got goosebumps.
- Why it mattered. People like Silver were interested in AI because Hassabis had galvanised them, so their enthusiasm was no independent test. Legg had reached the same conclusion by a separate path: powerful corroboration.
- What Legg brought. Hassabis had never read Kurzweil; his influences were GEB, Asimov, Banks, Molyneux and Blade Runner, dreamed up in a "parochial backwater." Legg brought Hutter's theory, a formal treatment of intelligence, the AGI label, the Singularity Summit and a network of AGI contacts.
- Looking for each other. Both had come to the Gatsby because it was one of the only places combining neuroscience and machine learning; there turned out to be few like-minded people there, but there was one, and that was enough.
- Mallaby's geography lesson. In Silicon Valley, change-the-world dreamers are normal and networked; in London, outsized ambition was isolating and collaborators had to be hunted down.
Founding agreement
Over lunches at an Italian restaurant near the Gatsby, they agreed the field was nearing a watershed thanks to Hinton's deep belief networks, GPUs, Legg's measurement work and Hassabis's neuroscience intuitions. They disagreed on timing.
Legg: too early. Who would fund a venture with no prospect of a product? Computing was still immature, and he had already decided against founding once.
Hassabis: academia too slow. Powerful AI needs a team, urgency and freedom from bureaucracy: a mission-driven Manhattan Project, fundable by a billionaire or multibillionaire with a long horizon.
It was Ender versus not-Ender, and Ender won. On December 29, 2009, shortly before midnight, Legg emailed that he was in. The name DeepMind nodded to three things: Hinton's deep learning, IBM's chess champion Deep Blue, and Deep Thought, the supercomputer in The Hitchhiker's Guide to the Galaxy.
Mustafa Suleyman: the activist
With a scientific cofounder secured, Hassabis needed more backup and turned to a brash, brilliant autodidact as driven as himself.
A North London childhood without a safety net
- Birth and family. Born August 1984, eight years after Hassabis, also in North London with an immigrant parent and religion at home. His father, a devout Syrian Muslim with broken English, drove a cab from 4 a.m. to 8 p.m.; his mother, an English nurse who had converted to Islam, worked equally hard. Unlike the Hassabis mix of Christian mother and secular father, both parents shared the faith; the home had no music, books or newspapers.
- Faith as community. Friday prayers at a North London mosque, shoulder to shoulder with Pakistanis, Bangladeshis, Indonesians, Somalis, Turks and Arabs, gave him a lasting love of equality and unity. Around twelve he recruited schoolmates to pray with him, preaching a moral duty to do good. Mallaby's gloss: a 1990s London Muslim learned that society was welcoming only up to a point, so justice required working together.
- Abandoned at fifteen. His parents split; his mother followed a new partner to New Zealand, his father returned to Syria to remarry. Mustafa was left to look after his fourteen-year-old brother, sheltering with friends on irregular remittances.
- The trader. From eleven he bulk-bought sweets and sold them at a markup, hiring older pupils as sales staff and renting friends' lockers as warehouses; at fifteen he moved to mobile phones; at eighteen to repairing and trading cars, freshening up classic BMWs and Mercedes with his brother.
- The scholar. He won a place at Queen Elizabeth's School, a top state school founded in 1573 and now dominated by students of Asian origin, and was among its best performers. A teacher chose novels for him, explained the Telegraph versus the Guardian, and gave him an Economist subscription.
- Enterprise award. At seventeen he won a national enterprise award for a plan to make London attractions accessible to disabled visitors, researched by pushing himself around museums and monuments in a borrowed hospital wheelchair.
- The Hassabis connection. He befriended George Hassabis, often slept in Demis's vacated bedroom, and found the family home (the parents had built capital renovating houses) a picture of middle-class security. Every Wednesday George, "Moose" and his girlfriend Marilyn taught chess at the Hassabis parents' education centre. Demis, already running a studio, appeared once at a garden barbecue as a distant, impressive figure.
Oxford and the dropout
- Admission. In 2002 he was one of a handful of classmates admitted to Oxbridge and chose Oxford, reading philosophy and theology.
- The British Muslim bind. After 9/11 Islamophobia surged and his sense of Muslim solidarity grew, yet Oxford pushed him towards a secular Islam: committed to social justice, doubting God, loving the mosque's post-racial fellowship but rejecting its gender inequality and condemnation of gay people.
- The look. Pink corduroys with braces, a flat cap over flowing curls, a pipe, Ellesse trainers.
- Iraq. In his first spring the UK backed the 2003 invasion of Iraq; he wanted to help his community but sided neither with the 9/11 clerics nor with Saddam. On a European backpacking trip with three non-Muslim friends he refused alcohol while they drank. His hard adolescence felt like real life; Oxford felt like a bubble.
- Quitting. Near the end of his second year he told a friend he was done with books and abstractions. The friend was flabbergasted: for a poor student Oxford was a "golden ticket" and he had "ruined his whole life." Suleyman's reply: having always had nothing, he was fearless about flipping the table; why study church patristics instead of changing the world now? He left in summer 2004.
Searching for a lever on society
| Venture (2004-09) | What it was | Why it fell short |
|---|---|---|
| Camden milkshakes | Market stall with George Hassabis, near Elixir's second office | A business, not a mission |
| Muslim Youth Helpline | Nonprofit co-founded by a friend, counselling depressed or confused young Muslims; staff from "chaotic" backgrounds (one later set himself alight in a North London park) | Helped him become Muslim, secular and British at once, but did not change society; he moved from agnosticism to firm atheism |
| London mayor's office | Government work to improve society at scale | Too bureaucratic |
| Reos Partners | Co-founded conflict-resolution consultancy | Collective therapy seldom dissolved enmities |
| The Facebook insight | Late 2009: a network of about 132 million monthly users | Its power to shape ideas dwarfed all else; at that pace it would outweigh Islam or Christianity culturally. Conclusion at 25: become a technologist |
The shadow book, December 2007
Suleyman and Demis had grown closer. Hassabis left Elixir with a few million pounds, and in 2007 Suleyman proposed a property partnership: Hassabis's capital, Suleyman buying flats to let, profits split. Over lunches near UCL they swapped reading: Hassabis loved science, Suleyman logged every title he read in politics, sociology and complexity theory.
At a steakhouse by Smithfield meat market (a slaughter site since the tenth century), Hassabis asked what Suleyman was reading. Suleyman produced Robert A. Johnson's Owning Your Own Shadow, a Jungian book about acknowledging the harsh side of one's ego and the darkness it casts on others, and offered it to him. Hassabis brushed it aside as something Suleyman should be reading. Outcome, years later: Suleyman, feeling plunged into darkness by Hassabis's shadow, called the exchange "ridiculous" and "surreal," yet conceded Demis was partly right, since his own shadow derailed his career about a dozen years later. Speaking to Mallaby as Microsoft's head of AI, he agreed that everyone needs that book, adding that at least he had acknowledged it.
The poker-night recruitment, July 2010
Suleyman, Demis and George played poker at the Vic, a casino on noisy Edgware Road whose second-floor poker room ran 24 hours a day; Demis wore outsized Blade Runner-style shades.
- The night. On July 15, 2010, George and Moose were knocked out early; Demis finished fifth, winning almost $2,000. Over chocolate cake and vanilla ice cream, buzzing on dopamine and sugar, he revealed his secret.
- The secret. A stealth company aimed not at AI but at AGI, with a plan, a brilliant cofounder and a list of collaborators; only an investor was missing.
- The pitch to Suleyman. If he truly wanted to improve the world, AGI was the best vehicle: an infinity machine has infinite potential. Hassabis's framing was technology as a force multiplier: charities, consultancies or the mayor's office return one unit out for one unit in; the goal was one in, a million out.
- Suleyman's doubt. Games are simple; societies turn on fights, emotions and who said what to whom. No AI, however general, could grasp, let alone shape, billions of permutations.
- The bond. The argument was Ender versus Ender: both articulate, forceful and willing to follow ideas to extremes, and the harder they argued the closer they grew. Their experience was complementary: Hassabis imagined a technology with huge impact; Suleyman had studied how the world needed to be changed. A voice in his head whooped "Wow! Amazing!"
- Follow-up. Two days later Suleyman emailed a Wired profile of Sergey Brin, who was putting part of his $15 billion fortune into computational medicine to speed drug discovery: perhaps the sort of billionaire who might fund AGI. Hassabis called it "very cool" and asked Moose to help flesh out a business plan for a different billionaire he meant to pitch at the Singularity Summit in August.
| Founder | Background | What he brought |
|---|---|---|
| Hassabis (b. 1976) | Chess prodigy, games entrepreneur, neuroscience PhD | Vision, charisma, neuroscience-inspired AI, the Manhattan Project ambition |
| Legg (b. 1973) | NZ mathematician, Goertzel alumnus, Hutter PhD | AGI theory and vocabulary, a measure of intelligence, Singularity network |
| Suleyman (b. 1984) | Oxford dropout, street trader, social activist | Commercial hustle, social mission, investor hunting |
Mallaby puts Facebook at about 132 million monthly users in late 2009; the company reported roughly 350 million by December 2009 (132 million was closer to a mid-2008 figure). Poggio's Nobel ranking is hindsight praise given after 2024. The Suleyman portrait and the "shadow" foreshadowing are drawn after his later fall-out with Hassabis, and both sides of that dispute shape the telling. Legg's claim to have coined "AGI" is well supported (the term had a scattered earlier use, notably by Mark Gubrud in 1997, but the Goertzel and Pennachin volume, eventually published in 2007, popularised it).
Legg remains at Google DeepMind as Chief AGI Scientist. Suleyman, CEO of Microsoft AI since March 2024, launched a "humanist superintelligence" team in November 2025; in March 2026 Microsoft unified its Copilot product teams under former Snap executive Jacob Andreou so Suleyman could concentrate on building Microsoft's own frontier models.
Founding DeepMind
DeepMind was funded in 2010 by believers rather than conventional investors: a religious English financier, a Singularity-loving partner at Peter Thiel's Founders Fund, and later a safety-minded Skype cofounder. Mallaby shows how a chess gambit at a party, Thiel's rule against partnership votes and Luke Nosek's infatuation got DeepMind its first $2.3 million, at the price of nearly half the company. Hassabis then built a platform for scientific discovery on three pillars: hardcore conviction that AGI was possible, a long financial runway, and a culture that let awkward geniuses thrive.
Begging for money
Before the summit
- Old backers say no. Every Elixir investor declined: an AGI company was technically daunting and commercially dubious.
- Poggio. His MIT supervisor promised £100,000 (about $150,000), largely because Poggio's wife, a psychologist, said they should back whatever "that electrifying English guy" did.
- David Gammon. The nearest thing to a real investor: an eccentric financier with swept-back hair and a country gentleman's bearing, courted by Hassabis and new wingman Suleyman over pub lunches. His motive was spiritual: AGI has a deeply religious aspect, it is "finding God's algorithm," the architect of the universe being what we call God; he believes the world advances and must be pushed.
The Singularity Summit, August 2010
The summit opened on August 14, 2010 at the Hyatt Regency, San Francisco. Mallaby sketches the cast: Ben Goertzel looking blown off course to Glastonbury; self-described cyborg Steve Mann in augmented eyeglasses; biophysicist Gregory Stock declaring evolution in fast forward. A previous summit had been mocked as transhumanism's Bay Area coming-out party.
- The trio. Legg and Hassabis took speaking slots; Suleyman came along and slept on the couch in Hassabis's hotel room.
- Hassabis's distance. Asked if he was a "singularitarian," he said that, perhaps because he was British, it was "a bit Californian."
- The target. They were there because they were desperate for money, and their main hope was Peter Thiel.
- Record. Founder of PayPal (the original digital payments firm) and Palantir (the original software-first defence company); first investor in Facebook, already worth over $1 billion to him.
- SpaceX rescue. In 2008, two days after SpaceX's third consecutive launch failure, Founders Fund invested $20 million.
- Worldview. A Catholic contrarian and competitive chess player thrilled by Deep Blue's win over Kasparov; he thought AI would soon dominate the cognitive core of the information economy and that only such a breakthrough could ward off economic stagnation, perhaps even save the free-market system.
- Why fund the summit. Wacky gatherings might throw up a high-potential long shot; one hit would repay everything.
- Absent. He paid for the summit but did not attend the sessions, to Suleyman's surprise.
Legg knew Thiel had thrown a speakers' after-party the previous year, so the trio went to his house near the Golden Gate Bridge, where Thiel was ringed by supplicants; Suleyman felt like "irrelevant nobodies" facing a titan. Legg's friend Eliezer Yudkowsky, a high priest of the Singularity community whom Thiel had known for about six years, introduced Legg and Hassabis as among the smartest people in AI launching an ambitious company: the first company Yudkowsky had ever recommended to Thiel. Hassabis, who says he had prepared "for a year," skipped the start-up spiel (30 seconds over party noise) and opened with chess: the deep tension between bishop and knight, equal in value but utterly different in powers. Outcome: Thiel invited them to his home the next day.
The pitch at Thiel's home
Thiel received them in workout gear, still sweating, gravely sipping a Diet Coke his butler brought.
- Hassabis's case. Powerful AI drawing on neuroscience and the explosion in compute; new machine-learning methods just starting to work; the two knew everyone in the field; Hassabis had built Elixir and won the Mind Sports Olympiad pentathlon five times.
- Thiel's first reaction. "This might be a bit much," tempered by Yudkowsky's endorsement.
- The grade. A-plus on the science, maybe F on the business model.
- The redeeming factor. Hassabis was an extreme "authentic" entrepreneur: a missionary, not a mercenary who wants riches and then hunts for an idea. Missionaries never quit, working unpaid around the clock. Thiel believes people have perhaps one great company in them, and "it was Demis's destiny to build this one."
- Next step. Come back in a few weeks to pitch the Founders Fund partners. Thiel was curious but wary.
The business plan
Back in London, Hassabis and Suleyman finished a thirty-page plan running from high-concept futurism to milestones before the next round. It argued three things.
| Section | Argument |
|---|---|
| Why AGI is necessary | Quotes Bill Gates: a learning-machine breakthrough would be worth ten Microsofts. Society faces problems of unprecedented complexity, from stabilising capitalism after 2008 to feeding a growing population; progress is slow because of the "ingenuity gap" (a phrase Suleyman took from Thomas Homer-Dixon's book): brains have limited storage, humans limited lifespans, and big organisations give diminishing returns. Bold line: "AGI is the solution to this problem." |
| Why now | A chart showing supercomputers reaching 10^20 calculations per second by 2025, 100,000× the level of 2010; chips would do for AI what fibre optics and routers did for the 1990s internet. Brain imaging was revealing the brain's workings, but with almost 60,000 neuroscience papers a year, non-specialists could not mine them: DeepMind's edge. The brain's genius lies in integrating distinct, specialised parts into general intelligence. |
| Why DeepMind | Build agents, not narrow systems: general, proactive learners that master many problems by interacting with an environment ("a city, not a house"). Teach concepts using neuroscience on hippocampus-prefrontal interactions that turn memories into abstractions. Insist on grounding: a system that only matches symbols to symbols (apple picture to the word "apple") knows nothing, since it is unclear where meaning resides. |
What missed. Neuroscience insight helped DeepMind early but not after about 2015. Grounding remains hotly debated: large language models such as ChatGPT and Gemini were never taught concepts directly, yet a feel for how the world works emerged as a by-product of statistical mastery, bypassing the plan to program conceptual understanding.
What hit. The compute driving AI grew almost exactly as projected. The forecast of human-level AGI around 2030, outlandish in 2010, looks to Mallaby, as of 2026 and allowing for AGI's fuzzy definition, "just slightly conservative."
The plan's 10^20 per second by 2025 fits only if counted loosely: the fastest Top500 supercomputer, El Capitan, measured about 1.7 × 10^18 FLOP/s in 2024-25, while frontier AI training clusters of hundreds of thousands of GPUs reach the 10^20 range only in low-precision AI arithmetic. The direction and rough magnitude of Mallaby's claim hold; "almost exactly" is generous.
Closing the money
The Gammon family committee, September 2010
Gammon would invest, but only his way: founders had to pitch at his Cambridge home to him, his artist wife and three teenage sons, each with an equal vote. He warned Hassabis that if he could not explain it to his youngest, Cameron, he would lose that vote. Hassabis and Suleyman complied, reasoning that a wacky project could not afford to snub wacky investors. All five family members said yes. Gammon put in a few hundred thousand pounds. Hassabis credits him with getting DeepMind off the ground, while marvelling that a "serious scientist" had to pitch to a fourteen-year-old.
Founders Fund's unsettling portfolio
A few weeks later the pair returned to California, still needing Thiel's money. As part of mutual vetting, his team showed them three portfolio companies.
- Palantir. Helping US forces track terrorist networks. Suleyman, a human-rights activist who had opposed the Iraq war, feared DeepMind would be pushed to build surveillance algorithms.
- A military-robotics start-up. Its lobby featured a gun mounted on caterpillar treads; not the AI they wanted to build.
- Halcyon Molecular. Pure Singularity spirit: AI applied to cheap gene sequencing data to extend lifespans in lockstep with Moore's Law. Suleyman found it "loopy," capital-intensive and speculative; reassuring that DeepMind was "in the same bucket," alarming as to who was deciding.
Luke Nosek's lightning bolt
The decider was Luke Nosek: mid-thirties, voluble futurist, engineer, Thiel's friend since PayPal, backer of the Singularity Summits and the partner in charge of the fund's wildest bets, SpaceX and Halcyon (whose president he had become, calling it the portfolio company with the most profound chance to change the world).
- His creed. Bring about a positive singularity, or prevent a negative one. Mid-interview he lost his thread, then said the concept is so intense it makes people lose the ability to think; if it does not affect you emotionally, you are not truly picturing superhuman intelligence.
- The meeting. Talking to Hassabis at Halcyon, Nosek fell "head over heels": the first AGI dreamer who seemed really competent and brilliant; earlier ones were "just dreamers," as is typical early in a technology. He decided at once to invest and join the board.
- The SpaceX analogy. In 2008 he had ignored the apparent craziness of a private rocket firm and focused on a shift in the technological backdrop: cheap off-the-shelf aerospace parts that could beat NASA's bespoke ones. SpaceX's value had since risen over tenfold. For DeepMind the shift was deep learning unlocking ever more compute. Both bets were also on a founder: Musk then, Hassabis now, whose mantra was "AGI, AGI, AGI" and who made your brain almost break.
- Breaking his rules. Nosek had never sought a board seat outside the US or on such a small investment.
The partnership split, and how it was resolved
- Board seats. Thiel was agnostic: venture is about backing contrarian ideas, not coaching founders; start-ups should sink or swim. Nosek was incredulous at "we'll see what happens."
- London. Founders Fund held almost as an article of faith that good start-ups lie within 45 minutes of Stanford, and joked that investing in Britain was like investing in Somalia.
- Hassabis's London case. Undervalued European talent: if AI worked, the West Coast would see the mother of all recruiting wars, while DeepMind quietly hoovered up Europe's best. He was also a British patriot, bristling at assumed American pre-eminence, wanting to prove deep tech could be built in Britain and rooting for the underdog. Nosek saw he had to let Hassabis have his way.
- The decisive rule. Most venture firms vote, and a few objectors kill a deal. Thiel rejected collective decision-making: voting yields middle-of-the-road portfolios, yet all venture profit comes from a few improbable moon shots. So partners could follow their gut, and Nosek could back DeepMind alone.
Mallaby's point is that DeepMind was funded not because Founders Fund as a body believed in it, but because Thiel's anti-consensus structure let one infatuated partner override colleagues' doubts. Without that quirk of governance, Mallaby suggests, the gap between Nosek and his partners might have doomed DeepMind.
- December 2010. Founders Fund wires $2.3 million for a bit under half the company: terms close to those Hassabis had refused for Elixir, but now there was no other capital.
- Hassabis's stake. Nearly as large as Founders Fund's; 9× Legg's and 14× Suleyman's. His reasons: DeepMind was his idea and he was its driving force; he would work without salary at first on his Elixir cushion; he had CEO experience; and he believed he needed maximum shares to set direction. Mallaby notes the tension: he dislikes dominating others but was determined to control his quest.
- Legg's view. Too laid-back to fight: if DeepMind succeeded, whatever he held would still be a lot.
- Suleyman's title. Invited in temporarily to help with the plan, the uncredentialed latecomer worked and negotiated his way to cofounder; the hotel-couch episode foreshadowed a habit of putting himself at the centre of the action.
- December 2011. Second round: Founders Fund adds $7.9 million; Jaan Tallinn (Skype cofounder) about $2 million, investing because he thought the mission insanely dangerous and wanted to press for safety. Hassabis kept him at arm's length.
Russell Square and the ghost of Szilard
The first office was an attic on Russell Square, laid out during the Napoleonic Wars (when Jefferson was US president), near the British Museum and an easy walk from UCL (they lunched at the Gatsby cafeteria to poach researchers).
- Turing. The London Mathematical Society was next door; Hassabis liked to imagine Alan Turing's spirit there as DeepMind built on his foundations.
- Szilard. Past it lay the pedestrian crossing where, on a humid morning in September 1933, Leo Szilard conceived the nuclear chain reaction, the opening scene of Richard Rhodes's The Making of the Atomic Bomb. Hassabis thought of it as he crossed: AGI is meant to be positive but is "momentous in the same way that the bomb was"; even in a tiny attic they were working on something significant.
- Mallaby's distinction. Szilard was one of the few 1930s physicists to foresee self-destruction; in AI, fears of annihilation were commonplace among the pioneers, who were steeped in the Los Alamos story.
- The imagination habit. Asked why he keeps casting himself in stories, Hassabis cites a vivid imagination (the faculty of his PhD) and natural empathy for how situations felt. At Cambridge, walking down King's Parade at 1 or 2 a.m. with a kebab from a Market Square van, he felt Newton and Turing "in the bones of the stone," like the residue centuries of meditating monks leave in a Buddhist school's rocks. Russell Square had a little of that, thanks to Szilard.
- The reality at the end of 2010. A room of desks, a meeting room, a space for a kitchen and a large closet of servers.
Hiring believers with credentials
For months, hiring was as hard as fundraising. Recruits had to believe AGI possible, ruling out most academics, yet be credentialed, ruling out flaky Singularitarians. Hassabis wanted PhDs in the mould of David Silver.
- David Silver. Back from Canada on a prestigious UCL postdoc, still scarred by Elixir; offered only part-time consulting, and when paid in DeepMind shares he gave them back to protect his independence, a decision that cost him a small fortune.
- Ilya Sutskever. Hinton's protégé (the only student Hinton thought had more good ideas than he did), a messianic deep-learning believer and later an OpenAI cofounder. Hassabis stretched the pay scale but could not offer enough; nobody valued DeepMind stock, and Hinton and others expected a revenue-less lab to do interesting science for a couple of years, then fold.
- Advisers. Hassabis paid stipends to star professors so their students would see DeepMind as credible. Hinton accepted, having seen at MIT that Hassabis was a force of nature; so did Rich Sutton, father of RL.
- Yann LeCun. NYU pioneer who kept his distance: before meeting Hassabis he took DeepMind for yet another firm claiming AGI was near, "complete BS."
- Dharshan Kumaran. For the first six months the only full-time PhD: impeccable as a neuroscientist, but not the person to build machine intelligence.
- Daan Wierstra. Joined in September 2011 as the first AI expert to take the risk: a Dutch friend of Legg's from graduate school, keen on neuroscience-inspired AI, tired of small-thinking, rivalrous academia, he took a significant pay cut for stock he assumed was worthless.
- Wierstra's antidotes to an empty office. A few games-industry engineers, no furniture, boxes on the floor. He launched "Formal Thursday" (suits, inverting casual Friday), which baffled interviewees who had to be told it was a joke, and cheerfully proposed to "build Terminators" until Hassabis asked him not to alarm people.
The platform for a scientific start-up
Mallaby's frame: most start-ups turn a known technology into a product (engineering); DeepMind had to invent the technology (science), which is riskier because success may be impossible. Before Apollo 11, nobody knew a moon landing could work; after it, imitators had proof. Engineering firms need pragmatic problem solvers; scientific ones need blue-sky wanderers, plus some way to direct the wandering.
- Conviction. No one knew how to build AGI, but the brain proves general intelligence possible, and that faith had to permeate the team or morale would fail while luminaries like LeCun scoffed. Legg: they wanted only hardcore believers; announcing an AGI company at conferences made 80% of people roll their eyes and walk off, a very efficient filter.
- Time. Investor patience is finite and science unpredictable, so extend the runway with side revenue: a small team on a commercial video game (2011); from early 2012, a deep-learning fashion recommender led by Suleyman (upload an image of a dress, get visually similar shapes, patterns, colours and styles, searching ideas without needing words).
- Culture. Hassabis's knack (in Wierstra's words) of seeing what is good in each person and talking to each at the right level, institutionalised through program managers or "glue people" who nurture brilliant researchers lacking coordination, speaking or even eye-contact skills. Wierstra: some locked themselves in the bathroom for hours and emerged with brilliant insights; "mother" them and you get what other firms miss. The governing idea is comparative advantage: shine at what you are good at.
Growing out of the attic
- Library hush. Desks spilled into the conference room; first project manager Helen King recalls silence so deep she could hear water clicking in the ancient heating pipes.
- Overflow logistics. Calls and company meetings moved to the garden square; in bad weather, to the server closet or a rickety stairwell on the route to the only bathroom. Trevor Back, a new PhD in computational astrophysics on the fashion project, interviewed candidates beside whirring servers and ran out for air every hour.
- The move. In September 2012, almost two years after the first round, DeepMind moved to Bernard Street. It now had to build something exciting.
Mallaby's verdict that the 2030 AGI forecast was "slightly conservative" rests on a definition he concedes is fuzzy; many researchers dispute that 2026 systems are near general intelligence. He raises, but does not pursue, the dynamic Wierstra describes: almost all-male researchers "mothered" by mostly female managers, which troubled some colleagues; Wierstra's defence (awkward men finally able to thrive) goes unchallenged. Halcyon Molecular, the Founders Fund bet that made DeepMind look sane by comparison, shut down in 2012. And the chapter's funding is presented mainly as a story of believers, skating past the fact that Hassabis accepted terms (half the company) that he had called selling his soul in 1998.
Atari
DeepMind's first breakthrough fused the two tribes of AI that had refused to talk: Toronto's deep learning (perception from raw data) and Alberta's reinforcement learning (agents that learn by trial and error). Vlad Mnih's Deep Q-Network (DQN) learned dozens of Atari games from pixels and score alone, with two brain-flavoured fixes, memory replay and a separated player and coach. Unveiled at NIPS in December 2013, DQN was "another ImageNet moment," not for vision but for agents, and proof that Hassabis had learned from Elixir to climb a ladder toward a grand goal.
2012: the ImageNet shock and DeepMind's bigger bet
By autumn 2012, when DeepMind moved into its new office, the field's energy had left the Singularity Summits for Geoffrey Hinton's Toronto lab. Speech recognition had started to work from 2010. In October 2012, at a conference in Italy, Hinton's student Alex Krizhevsky announced a vision system, trained in his bedroom at his parents' home, that won Fei-Fei Li's ImageNet competition with a model nearly twice as accurate as the runner-up. Hinton, Krizhevsky and Ilya Sutskever formed a company that consisted of nothing but the three of them and sold it two months later to Google for $44 million.
- Hassabis saw it coming. As a mid-1990s undergraduate he had judged that symbolic, deductive systems could never handle messy inputs like images or speech; real AI would have to generalise inductively from copious examples. The theory existed then, but computers were puny next to brains and training data was scarce.
- DeepMind's very name was a bet on it. By 2010 one founding premise was that deep learning was close to a breakthrough. ImageNet vindicated it.
- But perception was one piece. The business plan said AGI required integrating brain-like components. Deep learning would supply perception; reinforcement learning (RL) would supply planning and strategy. The plan promised a "Deep Learning Agent" that would master games without being told the rules, trying millions of actions and inducing strategies from their consequences.
Two tribes: Toronto and Alberta
Hinton and Richard Sutton, Alberta's RL luminary, made parallel claims: Hinton said deep nets were "how the brain works", Sutton that RL agents were "how the mind works." Mnih's reaction was the obvious one: why were they not working together? Mallaby's answer is temperament.
Deep learners (Hinton, Toronto). Build systems that work in practice even without an elegant theory. Need human-labelled data; brilliant at mapping inputs to outputs but cannot act in the world. Hinton told Mnih he had tried combining it with RL: "It doesn't work." Colleagues warned that RL researchers vanish into a separate community.
Reinforcement learners (Sutton, Alberta). Love mathematical proofs that systems work in theory, even if hard to build. Agents gather their own data by acting, so in principle they can learn anything. Alberta's Csaba Szepesvári: neural nets work in practice, but he could not prove anything about them, so he did not use them.
How a deep network sees
Mallaby uses Krizhevsky's AlexNet to explain the black box.
- Layered features. The first layer of artificial neurons picks out edges, lines and colour patches (as the eye notices contrasts); the next assembles curves, circles and textures; deeper layers find object parts such as a paw; the last layers name the object.
- The arithmetic. Each neuron turns inputs into numbers (as David Levy's chess program did for pieces), multiplies them by a weight, adds a bias, and passes the result through a mathematical filter that decides what signal goes onward.
- Learning. When the label is wrong, the system nudges its roughly 60 million parameters iteratively until photos map to correct words. The settled parameters are the algorithm for vision: in Mallaby's phrase, a primitive infinity machine, taming an infinity of pixel patterns.
- The black box. No programmer wrote rules ("a cat has four legs and a tail"). Nobody can say why any weight settled where it did; a tweak in one layer ripples nonlinearly through the rest. Builders sound like surprised parents of a toddler learning words: they judge the child by results, not by inspecting its mind.
In a note, MIT's Tomaso Poggio compares deep learning to electricity: Volta's battery (1800) made electricity usable decades before Maxwell supplied a theory in the 1860s. Deep learning likewise works, LLMs included, with no complete theory yet.
What RL adds, and why it seemed primitive
Mnih, studying in Alberta, pressed Szepesvári to use Hinton's landmark 2006 deep belief network paper; the refusal (no proofs) showed him the RL tribe's blinkers. Yet the RL people were plainly on to something. Deep learning recognises patterns but cannot create agents that learn by doing, and much human learning is trial and error: a child drops an object and learns gravity, says "please" and learns manners.
RL is a conceptual framework, not a computational architecture. Its components are defined abstractly and their implementation can be crude (a policy might be a simple lookup table). All of them serve one purpose: in a world with an infinity of possible actions, an infinity machine must develop algorithms that narrow the search for the best action.
| RL term | What it means |
|---|---|
| Value function | Estimated future reward from a given state of the environment |
| Policy | The rule for deciding what to do next |
| Model | Optional: a prediction of how the environment responds to actions |
| Q-value | The "quality" of taking action a in state s |
| Q-table | A table of Q-values for every state-action pair |
| Experience replay | Storing past experiences and relearning from them |
Compared with deep learning's dazzling results, RL looked theoretical. But it promised freedom from labelled data: an agent could generate its own training data without limit.
Recruiting Vlad Mnih
Vlad Mnih, a Ukrainian-born Canadian and a quiet Hinton protégé, was the rare student who crossed the divide.
- Training in both camps. Undergraduate classes with Hinton; a master's in Alberta (2008) under Szepesvári; then a Toronto PhD with Hinton, from a desk in a converted supply closet.
- Talked out of his idea. Asked what he wanted to work on, he said combine RL and deep learning; Hinton said he had tried and it failed, and other deep learners warned he would disappear into the RL community for good.
- A classic Hintonite PhD. He built deep nets that read satellite images, recognising industrial buildings, oil tankers and signs of deforestation. They worked well; Hinton proposed they found a company together. Mnih declined because he hated the idea of pitching investors.
The reception in Scotland, summer 2012
At a sober AI conference in Scotland where Mnih presented his PhD work, two attendees turned up at the first evening's reception announcing that they were building AGI at a London start-up: Shane Legg and Daan Wierstra. Hinton had warned students off excitable Singularity types (in 2006 even Douglas Hofstadter had mocked the movement's wilder visions, such as self-assembling "utility foglets"). Mnih thought the pair sounded crazy. His older brother Andriy Mnih, an AI scientist who had met Legg as a UCL postdoc, vouched that they were real scientists. Over coffee, Vlad gave his usual answer, and instead of the usual disdain got delight: "That's what we're doing!"
- Why they welcomed the idea. In Switzerland, where Legg and Wierstra did their PhDs, combining the approaches was encouraged; Legg credits groups like Jürgen Schmidhuber's, and Wierstra had combined deep learning and RL in his own PhD. Neuroscience pointed the same way: RL reward signals resemble dopamine, so a brain-based AGI would need RL.
- The upside of hubris. Academia struck Mnih as both boringly cautious (incremental advances) and terrifyingly competitive (racing to publish). DeepMind offered big leaps with almost no rivals, because nobody else thought them possible: if it worked, it would be massive.
The Hassabis interview
Before a video interview a few weeks later, Hassabis sent Mnih a link to his own Wikipedia page: chess prodigy, star game designer, five-time Mind Sports Olympiad winner. Mnih was intimidated, then won over.
- Humility. Hassabis came across as approachable and easy to connect with.
- The neuroscience view. Academic computer scientists think about the next practical step; a neuroscientist sees the end point of intelligence.
- Conviction as substitute for proof. In Toronto, because nobody could prove neural nets would work, it mattered that charismatic believers like Hinton and Sutskever insisted they would: confidence substituted for theory. Hassabis had the same total commitment. Mallaby links it to the line from the Life Story film about DNA that had pushed Hassabis toward Cambridge, in which James Watson blesses those who believed before there was evidence.
Mnih still expected DeepMind to fold like most start-ups, but signed up. His reasoning: he was a scientist who wanted to do science, and here was a founder willing to raise money and do whatever it took. A postdoc meant being stuck in the academic pack; DeepMind meant a clear road to the frontier.
Bernard Street, May 2013
Mnih moved to London in May 2013. The same day, David Silver became a full-time employee, having found his hours at DeepMind more rewarding than his university lab. In Mallaby's notes, Legg says he had to persuade Silver, who was wary after Elixir; one reassurance was that Silver would initially report to Legg rather than Hassabis. Silver arrived as an RL authority who held a Royal Society University Research Fellowship, one of Britain's top early-career honours.
The rest of the team showed DeepMind's deliberate intellectual diversity:
- Bayesian / probabilistic modellers. Joel Veness and Shakir Mohamed, working on quantifying uncertainty.
- LeCun's NYU deep learners. Koray Kavukcuoglu (LeCun supervised his PhD) and Karol Gregor.
- AI meets human intelligence. Wierstra and others; computational psychologist Chris Summerfield joined Dharshan Kumaran in a fledgling neuroscience unit.
Mallaby's claim: for decades AI had been split among computer scientists, statisticians, psychologists, neuroscientists, physicists and biologists, so balkanised it barely existed as a field; DeepMind was unifying it. By 2013 this looked contrarian, since pure deep learning seemed poised to deliver on its own (it did, from 2016, in skin-cancer diagnosis at Stanford and Google Neural Machine Translation). DeepMind held its line: deep learning only matches one thing to another (speech to text, image to label), while DeepMind wanted an agent that plans and pursues goals across many environments.
Choosing Atari
The open question was which environments. Legg, whose doctorate had defined intelligence, wanted in-house metrics. Hassabis, carrying the fundraising, wanted an external yardstick for credibility, and Silver agreed: DeepMind should not be both test-setter and test-taker.
The team settled on the Atari catalogue of 1970s-80s games, for four reasons:
- Cheap. Primitive 1970s-80s graphics meant the compute needed was affordable.
- General. Dozens of games let one agent prove it was not a one-trick system.
- Built-in reward. Most games show a constantly updating score, the feedback signal RL needs.
- Investor appeal. Funders had grown up on Space Invaders and Breakout (one Atari console sold more than 30 million units).
The idea itself came from Alberta: Silver credits Michael Bowling, who with Joel Veness and PhD student Marc Bellemare had adapted 55 console games into the Arcade Learning Environment (published in JAIR, 2013). The rules of the contest were strict: no human hints on how to score, avoid losing a life, or even what a life was. The agent got only raw pixels, a joystick and the running score, like a gamer trying a new release.
| Team approach | Method | Outcome |
|---|---|---|
| Task decomposition | Hand-built algorithm for each sub-task of a game | Worked reasonably, but could not generalise across games |
| Probabilistic duo | RL agent starting from a model of the game, gaining confidence as play confirmed it | Progress, but misfired when the model proved wrong and it had to form a new hypothesis |
| Mnih and Kavukcuoglu, then Silver | RL with a flexible neural network instead of a brittle model | Became DQN |
Legg's attitude to the losing approaches: happy to let people try different things, since a technique beaten on one problem may return on another. Kavukcuoglu, a Turkish alumnus of LeCun's group, would later become DeepMind's research director.
Building DQN: Q-learning plus a network
Mnih first trained a deep network to read the raw pixels, giving the agent perception. He then bolted on an established RL method, Q-learning, where Q stands for quality: by playing, the agent learns the value of each state-action pair. In Pong, moving the paddle left when the ball heads left may win a point, so that pair earns a positive Q-value. Recorded across millions of runs in a Q-table, a complete table would tell the agent the best action in every state, and it would play at superhuman level. In textbook form the update is Q(s,a) ← Q(s,a) + α [reward + γ · max Q(s′,a′) − Q(s,a)].
The catch: trying every configuration would take decades. Mnih's shortcut was to feed each experience (state, action, reward) into a neural network, which generalised from experienced pairs to unseen ones, exactly as an image classifier learns from labelled cats to spot an unlabelled cat. This is Mallaby's version of solving the problem of induction: the network turned a hopeless trial-and-error marathon into a learnable task.
Fix one: memory replay
Mnih's first obstacle was the one behind Hinton's "it doesn't work." An RL agent exploring one area (one section of a maze) produces correlated runs of similar data, whereas deep learning assumes a curated dataset studied in random samples.
Mallaby's thought experiment: an image classifier given a stack of photos sorted by subject (cats, then trees, then dogs, then lions).
- Studied in order. The first batch is all cats, so the model labels everything "cat," and early impressions stick.
- Lightly shuffled. A cats-and-trees batch teaches that branches are not cats, but anything with eyes, feet or a tail still gets the cat label: only marginal gain.
- Thoroughly shuffled. Exposure to every kind of not-cat (dogs, lions, furry blankets) is the only way the model learns what distinguishes a cat.
An Atari agent stuck in one corner of the board generates scenario-1 data, a microcosm of the game, and will never master it.
Silver proposed a new use for an RL idea from the 1990s. Then, memory replay was a data-efficiency trick: robots were expensive to run, so researchers stored their data and relearned from it repeatedly (Mnih's account in the notes). Silver's twist: store experiences in a buffer for a while, then sample them at random. Each camp read the fix its own way.
| Who | How they read memory replay |
|---|---|
| Silver | An extension of RL ideas; proof of his goal of an agent learning from raw experience |
| Hassabis and the neuroscientists | The hippocampus, which (his PhD topic) replays memories during sleep so salient events lodge in the neocortex |
| Mnih and Kavukcuoglu | A bridge: turning RL data into something that looks like supervised-learning data |
Replay lifted performance quickly, and Mallaby calls it an inflection point that vindicated all three visions. Silver's notes add colour: RL from raw experience had been the topic of his Royal Society fellowship application, and it had sunk a faculty application at Edinburgh, where interviewers laughed at the impossibility.
Mallaby's main text credits brain-inspired design; his own endnotes are more sceptical. The 2015 Nature paper stressed neuroscience intuitions, but most DeepMind researchers downplay them. Legg: did replay need neuroscience inspiration? "Probably not," though it resembles the brain a bit. Mnih found the biological counterpart satisfying rather than decisive. Read the "brain-like regions" claim as Hassabis's framing, not settled history.
Mnih watched Andy Murray play top seed Novak Djokovic at home, too nervous to look at some points, and between games refreshed his laptop: the agent trained on through the weekend. It was playing Pong, Atari's first product, designed to be simple enough for drinkers in bars. The agent usually lost 21-0, its paddle drifting like a toddler wandering beside a tennis court. Then a game ended 21-4, a score far too high for luck. Suspecting a bug, Mnih switched to the live feed: the paddle now moved toward the ball on every rally. From there improvement was rapid: superhuman at Pong and Breakout.
Fix two: separating player and coach
The harder games still failed. Seaquest offers three routes to success: sink enemy subs, rescue divers, or surface for oxygen (no points, but a longer game). The agent torpedoed with gusto, ignored the other two strategies, and then abruptly stopped doing anything.
- Debugging RL. This was an RL problem, not a network problem, so Mnih asked Silver, who passed on Sutton's rule: you cannot understand an agent from a human view; be the agent and experience its experiences. In practice: inspect the Q-table, the data it is "living and breathing."
- Diagnosis. The table showed ever-rising reward estimates for actions that had worked before, hence the torpedo mania.
- Root cause. A clash between supervised learning and RL. A vision system gets a fixed, standard-size reward for each correct label and the round ends. An RL agent maximises rewards over a game of unspecified length and unbounded total, so nothing caps its optimism.
- The spiral. Each repeat of a scoring action recalled the last rosy estimate and added more, an unstable feedback loop. Mallaby's analogy: someone finds two $100 bills on the pavement, extrapolates to millions after a month of walking, then billions; the bubble bursts and the agent suffers a crisis of confidence, hence the freeze.
The fix broke the loop between playing and learning with two networks. A player network chose actions with its weights and biases frozen; a coach network watched the results and updated its parameters; only after "a suitable period of study" were the coach's parameters copied to the player. The patience is the point, as with a tennis coach: two points won by serve-volley tempt immediate advice, but further watching reveals lobs, drop shots and baseline forehands, and perhaps that serve-volley is a weaker option. Inductive hypotheses must stay open to revision. Freed from premature instructions, the player kept experimenting, found divers and oxygen, and the agent mastered Seaquest. Per Silver's note, the split was made in 2013 and named the "fixed target network" in 2014.
Memory replay echoed the hippocampus-neocortex relationship; the player-coach split showed that dividing a system into discrete, brain-like parts made stronger agents. Mallaby's verdict: Hassabis's business plan had promised neuroscience-inspired breakthroughs, and three years on he was delivering.
Mallaby's player/coach account is loose. In DQN the online network both acts and learns; a periodically updated target network (the frozen copy) supplies stable training targets, which curbs the runaway feedback. A related overestimation bias was fixed later by Double DQN (2015). The December 2013 NIPS workshop paper describes experience replay but not the target network, which first appears in the 2015 Nature paper, consistent with Silver's dating of the name to 2014. "Nearly twice as accurate" for ImageNet means AlexNet roughly halved the error (about 15% top-5 against 26%).
NIPS, Lake Tahoe, December 2013
DeepMind's coming-out party was at Harrah's Lake Tahoe Hotel & Casino in Nevada, home that year to NIPS (Neural Information Processing Systems), the largest machine-learning meeting. For three years the company's website had been a black screen and a logo. Word had leaked, and Mnih, in a grey sweater with sleeves rolled up, faced a room probably over fire limits.
- Seaquest. The same agent sank subs, surfaced for oxygen, then went back to hunting.
- Space Invaders. It targeted the mothership, worth five times an ordinary invader.
- Boxing. It pinned the opponent against the ropes; the audience laughed, oddly untroubled by the ruthlessness given the field's Terminator worries.
- Breakout (finale). It found the expert trick: tunnel through the bricks so the ball bounces behind the wall and destroys bricks on its own.
Silver recalled that the room went silent: the same agent had learned something completely different in every game. DQN, the Deep Q-Network, was a turning point.
Elixir (Republic). A maximalist ambition attacked head-on: the most complex video game ever, from day one. The overreach doomed it.
DeepMind (Atari). An even grander goal, but with a ladder toward it. Hassabis let researchers tinker while he built the team, set no premature target, then judged that the moment to fuse deep learning and RL had come and picked Atari to prove it.
- Vlad Mnih. Ukrainian-born Canadian, trained in Toronto and Alberta; lead builder of DQN.
- David Silver. Sutton-trained RL authority and Royal Society fellow; proposed replay, taught "be the agent"; full-time from May 2013.
- Koray Kavukcuoglu. LeCun-trained deep learner; Mnih's Atari partner; later research director.
- Geoffrey Hinton / Richard Sutton. Heads of the rival tribes: "how the brain works" versus "how the mind works."
- Csaba Szepesvári. Mnih's Alberta supervisor, who shunned unprovable neural nets.
- Shane Legg / Daan Wierstra. Recruited Mnih in Scotland; Legg wanted in-house metrics.
- Andriy Mnih. Vlad's brother, who vouched for Legg.
- Michael Bowling / Marc Bellemare / Joel Veness. Built the Arcade Learning Environment of 55 games.
- Alex Krizhevsky / Ilya Sutskever. AlexNet's builders; sold with Hinton to Google for $44 million.
DeepMind's Nature paper (February 2015) reported DQN reaching at least 75% of a professional human tester's score on 29 of 49 games. In 2020 DeepMind's Agent57 became the first agent to beat the human baseline on all 57 Atari games; Atari is now a solved benchmark, and RL's centre of gravity has moved to training language models to reason. The tribes were honoured in turn: Hinton shared the 2024 Nobel Prize in Physics with John Hopfield, and Sutton and Andrew Barto received the 2024 ACM Turing Award (announced March 2025) for founding RL. Sutskever left OpenAI in 2024 to found Safe Superintelligence.
Thiel Trouble
Between October 2012 and mid-2013, DeepMind's financing nearly collapsed. Elon Musk's boast to Larry Page put DeepMind on Google's radar, while Peter Thiel, never a believer in AGI, turned skeptical just as AI became expensive. Hassabis, dazzled by the Valley, kept insisting Thiel was on side until Founders Fund refused to lead the Series C and left DeepMind scrambling to make payroll. The near-death experience taught Hassabis that open-ended, blue-sky science is a poor fit for venture capital, and that he needed a different kind of backer.
October 2012: a name dropped on Musk's jet
While the Atari team was forming, Luke Nosek of Founders Fund went to Cape Canaveral to watch SpaceX's Falcon 9 attempt the first commercial resupply of the International Space Station (cargo included scientific kit, clothing and chocolate-vanilla ice cream). It lifted off at 8:35 p.m. Eastern; at 79 seconds one of nine engines failed, but the other eight reached orbit (the secondary mission, placing a satellite, was lost). NASA hailed a new era. Nosek flew home on Elon Musk's jet with Larry Page, Google's cofounder and CEO.
- Page's AI pedigree. His father Carl Page had studied early neural networks in the 1960s; Google had just rebuilt voice recognition with help from a Hinton student. With ImageNet weeks away and Hinton rumoured to be starting a boutique, Page meant to buy it.
- Musk's one-upmanship. Musk had met Hassabis at a Founders Fund retreat (where Hassabis pitched DeepMind as a "Manhattan Project" for AI; Thiel recalls one investor joking that he should shoot Demis to save the human race). A follow-up lunch in the SpaceX canteen in Hawthorne, cranes moving rocket parts overhead, became a debate over which mission mattered more. Musk: colonise Mars as insurance against disaster on Earth. Hassabis: killer AI could be such a disaster, and it could simply follow humans to Mars. Musk decided AI might be the bigger deal and promised to invest (the $5 million was formalised only in March 2013).
- The line. Musk told Page there was only one AI company he thought would work, and he was an investor in it: DeepMind. Page respectfully typed the name into his Android phone.
Nosek panicked. He knew Page had wanted AGI all his life and would buy it or copy it. Nosek, who had taken up meditation to cope with AGI's enormity, did not trust a corporate giant to steward it; he wanted it with his friend Hassabis, watched by suitably "freaked-out" people like himself. He tried to derail the conversation, then phoned London in alarm. Hassabis was unruffled: this could be good; let's play it out.
Mallaby rejects a rival account in which Page first heard of DeepMind when Nosek showed him the Atari Breakout demo on a laptop. Beyond contradicting Nosek's memory, it is chronologically impossible: Alan Eustace's email, seen by Mallaby, shows Page's interest in late 2012, more than six months before the Atari agent worked. One small dating slip of Mallaby's own: the CRS-1 launch lifted off at 8:35 p.m. EDT on October 7 (October 8 UTC), not October 8 local time.
Hassabis's post-AGI world
Nosek read one thing right: Hassabis felt the stakes viscerally. He ranks true AGI above the internet, the printing press and the Industrial Revolution, and expects post-scarcity abundance from it, like the science fiction of his teens. His view as told to Mallaby:
- Think bigger. Worries about whether governments can afford universal basic income or data-centre electricity miss the scale of change. The future looks like Iain Banks's Culture novels: asteroid mining, solved nuclear fusion, hydrogen fuel from seawater.
- Institutions dissolve. In superabundance, money, corporations and the stock market may lose their meaning.
- Universal basic provision, not UBI. Instead of cash, provide the basics of today's millionaire lifestyle (a nice house, schooling, healthcare, basic travel) free to every citizen; perhaps a normal car free, but work for extras ("if you want a Ferrari").
Mallaby sets this against DeepMind's free cafeteria (sea bream and salad, nobody old or stressed): post-scarcity in one corner of London. His note adds an irony: firms feed staff lavishly because their intelligence is scarce; if AGI makes intelligence plentiful, incentives change, though Hassabis would reply that corporations will change too. The waiting room, with chessboards on the walls beside photos of Hassabis with former world champions and upbeat quotes about DeepMind's chess system, made AI feel like a cosy extension of a mind game. In the office, the scale came back.
| Hassabis's three information ages | What changed |
|---|---|
| First: Industrial | Power, steam and Maxwell's equations (four equations, 1865) linking electricity and magnetism: physical information |
| Second: Computing | From physical information to pure information, processed by machines |
| Third: AI | Information "comes alive": it processes and generates itself, autonomously |
He also compared AI to fire and language, to the emergence of the prefrontal cortex, and to the first cave handprints tens of thousands of years ago, the "dawn of consciousness."
Science as a spiritual endeavour
- Communing with the universe. Hassabis calls science spiritual ("maybe too mystical a word"): understanding the universe is how he realises his purpose, and building AI serves that.
- Spinoza and Einstein. God present in nature, so studying nature is sacred; Einstein's "God of Spinoza" means what Hassabis means. Mallaby read him a line from Roger Scruton's Spinoza (2002) describing philosophy as a sacred way of life rather than a weapon; Hassabis agreed "100 percent."
- Leonardo da Vinci. His favourite, because anatomical drawings are art and biology at once: humanism, spirituality and science flowing as one river.
- Why is science possible at all? Life's purpose is knowledge or self-knowledge. The universe seems strangely set up for discovery (why should sand with a bit of copper compute?). Divine design, plain universe or simulation: he is open-minded, and expects the journey to tell. Meanwhile he feels part of a flow toward answers, and finds it exhilarating.
The ladder meets the money problem
Nosek misjudged the other half: Hassabis was practical. David Silver's ladder metaphor captures both sides: maximalist dreams in the small hours, the next rung at the office. In October 2012 the next rung was money.
- Funding so far. A bit over $2 million in 2010, $10 million in 2011. The Series C target: $65 million (in fact £40 million, just under $65 million at October 2012 rates).
- Why so much. Deep learning was pushing up salaries; GPUs were pushing up hardware costs; the mission justified it. Nosek agreed.
- Thiel's reversal. In 2010 Thiel backed DeepMind because it was contrarian and cheap. Now AI was mainstream and expensive, and his instinct was to sell the consensus. Nosek's influence had also shrunk after his other pet project, Halcyon Molecular, failed in August 2012.
- Thiel's reasoning. DeepMind would burn cash for a decade with no revenues or products, and he still says there is no business plan attached to general AI.
- Distrust. Thiel barely saw the team and was wary of a fellow chess player who had spent his youth crushing opponents and who excelled at Diplomacy, a game Thiel regards as a test of manipulation.
Google calls
A month after the launch, in November 2012, Hassabis got an email from Alan Eustace, Google's engineering chief: Page wanted a set of complementary AI teams and had sent Eustace Hassabis's name that morning as someone doing revolutionary work. (Eustace was a daredevil who, about two years later, rode a helium balloon to the stratosphere, about 25 miles up, did two backflips and fell to Earth at 822 mph.)
Hassabis was eager. Endless investor conversations made him feel his brain was atrophying. Mallaby's framing: from the 1960s-70s venture capital was "liberation capital" for applied science, freeing engineers from bureaucratic firms; DeepMind's science was blue-sky, so a rich corporate parent promised truer liberation.
- Google's appetite. An informal guideline: underwrite grand software projects with a shot at a billion users.
- The Cessna story. CFO Patrick Pichette (CFO 2008-2015) queried Eustace's order for several Cessna aircraft; Eustace explained a secret 3D Maps feature needing aerial photos through home-built lenses. Pichette approved without further questions. His philosophy: public companies obsess over the next 90 days, but some infrastructure takes a decade, and in Google's balance sheet you could "bury Wisconsin" unnoticed.
- A side scene. On 13 December 2012, over lunch in Mayfair, Suleyman told investor Ali Ojjeh that Google had proposed an outright acquisition and DeepMind had demurred; Ojjeh, knowing Suleyman had no personal wealth, called him nuts and offered a psychiatrist.
- The shrug. Google moved slowly, busy with a thousand projects, including buying Hinton. When Hassabis warned that a VC round would mark up DeepMind's price, Eustace said go ahead. Suleyman was stunned at indifference to an extra few hundred million, and called it a disaster: DeepMind had to raise the Series C.
The Hinton auction, December 2012
Google's talent hunt gave Hassabis an idea: if big companies were buying AI teams, the price of talent would only rise, so DeepMind should bid for Hinton's boutique itself. Days before NIPS he told Hinton a fair price was $10 million.
At NIPS (the same meeting where DQN would debut a year later), Hinton, unable to sit because of his back, ran an email auction from a laptop balanced on an upturned bin in his seventh-floor casino room. Bidders: Google (Eustace flew in on his own twin-engine plane and dined with the trio), Microsoft, China's Baidu, and tiny DeepMind. DeepMind offered $10 million in stock, having raised only a little over $12 million in its life: about 22% of a company valued on paper at $45 million (its 2011 Series B price), when Hassabis himself owned 21%. DeepMind dropped out; bids rose in $1 million steps while slot machines downstairs celebrated $25,000 wins. Hassabis then called and said Hinton was worth $50 million, five times his own earlier figure. When Microsoft, at about $30 million, offered an extra $10 million to stop the auction, that benchmark helped Hinton refuse. Google paid $44 million.
Hinton's reading: Hassabis first wanted to buy him to make DeepMind more valuable to Google, and when that failed, wanted Google to pay the most possible, which would raise the price for DeepMind. Hinton found it helpful either way.
Hassabis's coda was that Hinton would have done better to sell to DeepMind and then go to Google as a block. Mallaby's notes run the numbers from Companies House filings and confirm it, adding that Hassabis himself would probably have ended up poorer.
| Counterfactual: DeepMind buys Hinton's trio | Calculation in Mallaby's notes | Result |
|---|---|---|
| Hinton's group | 22% stake, diluted to 17.8% after the Series C, times Google's $650 million | About $116 million, over 2.5× the $44 million actually received (more if Google paid extra for the trio) |
| Hassabis | Diluted from 21% to 17.2% if paid in new shares; 3.8 points of $650 million | About $24.7 million forgone |
| Google's break-even | To make Hassabis whole: (100 ÷ 17.2) × 24.7 | About $143.6 million more, implausible a year after Google paid $44 million |
Strung along by Founders Fund
In early 2013, chasing his $65 million, Hassabis let Founders Fund know Google had approached him before Christmas. Nosek knew it was true; his partners did not believe it.
- Thiel suspected a games player was playing games, and wanted to counterplay.
- Brian Singerman, a partner, kept pressing on revenue. Neither the video game project nor the fashion-recommendation algorithm was near ready. Hassabis's account: he offered the biggest invention ever and they asked "Where's the widget?"; it would revolutionise all widgets. They also said a UK company was strange and "doesn't pattern match."
- Nosek's veto. Whenever Hassabis named another VC, Nosek said they were terrible, wanting only investors with pure motives, not profit-seekers. Hassabis, outside the Valley, did not know that bringing in a new lead investor for each round is standard practice.
- Nosek had a point about the backers. David Gammon had a religious faith in progress; Tomaso Poggio and his wife had faith in Hassabis; Jaan Tallinn, Skype cofounder and Series B investor, wanted a board seat because of his doom worries; Nosek himself cited Ayn Rand novels as proof that visionary companies are society's hope.
Hassabis also wanted to believe that Thiel, the visionary who had funded the Singularity Summits, believed in him. Mustafa Suleyman, the youngest founder, told him to stop deluding himself. The two reviewed each day most evenings, often into the small hours; Suleyman played the sounding-board role Dharshan Kumaran once had in the lab kitchen, but, like Silver at Elixir, he was close enough to contradict the boss. Mallaby sees a blind spot: the Jedi who won people over could not adjust to an aloof cynic.
Hassabis (then and later). Told staff he had briefed Peter and Peter believed. Years later he conceded Thiel was a portfolio contrarian making many uncommitted bets so that one big win pays for the losses, and "never believed in AGI"; at the time he was in awe of the Valley, "still just a kid from London."
Suleyman. Thiel held no meetings, reviews or feedback; to him they were "irrelevant nobodies from North London." Telling staff Peter believed was not lying to Hassabis but "just his reality." Realism would have sent them to other investors sooner, with feedback to calibrate strategy.
The Series C scramble
Early 2012 At Shoreditch House (rooftop pool, useless in British weather), Solina Chau, who manages the money of Li Ka-shing (Asia's richest man, worth $25 billion), bonds with Hassabis and Suleyman in five minutes and asks to invest after fifteen. She may make one bet a year with zero expected payoff, a kind of charity. Offered $2.5 million, she asks for more; they play hard to get.
Mid-Feb 2013 $65 million ruled out. Founders Fund will back a $30 million round if DeepMind finds $10 million elsewhere.
1 Mar 2013 Musk defers talk until a SpaceX launch succeeds; Hassabis refreshes the news until about 1 a.m., fearing a failure would cost Musk money and goodwill. Musk, cheerful, asks how much and says he does not want the whole allocation. Hassabis says $5 million (double the Chau offer); Musk agrees at once. Hassabis's regret: he should have said $50 million, "too British."
Spring 2013 Late one evening Nosek calls: "I can't get this through." Founders Fund will neither write the biggest cheque nor set the valuation; DeepMind must find a new lead. Cash is running low; payroll is at risk. Hassabis recalls the Elixir fear of running out of money.
Days later Chau is told her allocation will be boosted at Founders Fund's expense; this time they ask for double-digit millions.
About a week on The round closes: Chau $13.6 million, Founders Fund $9.2 million, total a bit over $25 million.
Two lessons and a widening gap
Hassabis drew two conclusions: Suleyman was right not to trust Thiel, and open-ended, blue-sky research does not suit venture capital. Over the next three months the gap widened. The Series C money went on star researchers and compute for Atari, whose success showed that training powerful models needed still more cash; Thiel, contrarian as ever, saw the AI talent war as an incipient bubble. Legg's summary: DeepMind grew more bullish while Founders Fund grew more skeptical. Mallaby's conclusion: without Thiel and the Singularitarians DeepMind might never have started, but it was time to find a new backer.
Venture capital wants a product path and revenue within a fund's life; DeepMind offered a decade of cash burn toward a goal with no widget. A fund built on contrarian, uncommitted bets sells exactly when a field becomes expensive and consensus, which is when AGI research needs the most money. That mismatch, more than any personal falling-out, pushed DeepMind toward a corporate parent.
The chapter leans on Hassabis's and Suleyman's accounts of Founders Fund's conduct. Thiel's view that AGI lacks a business model looks less like cynicism given that DeepMind did end up sold rather than profitable. Mallaby gently notes Hassabis's habit of narrating hopes as facts, the same trait that hurt Elixir. The 1 a.m. wait on 1 March 2013 fits the record: that CRS-2 launch went up in the afternoon UK time, but the Dragon capsule's thruster problem took hours to resolve.
- Luke Nosek. Founders Fund partner and DeepMind's champion; feared Google; vetoed other VCs, then had to reverse.
- Peter Thiel. Founders Fund leader; contrarian who never believed in AGI and wanted to sell the consensus.
- Brian Singerman. Founders Fund partner who demanded revenue.
- Elon Musk. Investor via the Founders Fund network; tipped off Page; put in $5 million.
- Larry Page / Alan Eustace. Google CEO and his engineering chief, who opened acquisition talks.
- Patrick Pichette. Google CFO who funded decade-long bets.
- Geoffrey Hinton. Auctioned his trio to Google for $44 million, helped by Hassabis's $50 million prod.
- Solina Chau. Li Ka-shing's investment manager; led the Series C in all but name with $13.6 million.
- Mustafa Suleyman. The cofounder who could contradict Hassabis.
Get Google
At the end of January 2014 Google bought DeepMind for $650 million, ending Hassabis's fundraising ordeal and giving him "a shitload of computers" while he kept a nearly autonomous lab in London. Mallaby shows the founders winning by refusing to haggle, bluffing about their backers, rejecting a richer Facebook offer and surviving a poaching raid by Yann LeCun. Google paid a premium and accepted unusual limits on its control because it wanted Hassabis himself, not just his team.
June 2013: the walk around a fake castle
Talulah Riley (an actress known for playing a murderous TV robot) rented a mock castle in Tarrytown, New York, for Elon Musk's birthday: guests dressed as samurai, and Musk threw the visiting sumo world champion (about 350 lb), hurting his own neck. Among the guests were Hassabis and Larry Page.
- Why Page was interested. Since buying Hinton's boutique he had learned that Hinton also rated Hassabis, despite thinking him pathologically competitive. Hinton had known him since an MIT encounter, had advised DeepMind, and his student Vlad Mnih had just joined.
- The pitch. Whispering because of a rare vocal-cord condition, Page walked Hassabis past the pointless battlements and argued that building an independent company was equally pointless: if the goal was AGI, why not use what Page had already built? He had used almost the same pitch to hire Ray Kurzweil in January 2013.
- Hassabis's calculation. Building a Google-sized company would take most of his career; he would rather look back on solving intelligence than on a multibillion-dollar business. "An easy choice."
- Why Page persuaded where Founders Fund had not. Page was a scientist, not a business or product person; Hassabis saw an Ivy League professor who would have taken his own offer. Page needed no convincing that AI would change every widget. Hassabis, fed up with justifying "the biggest thing of all time," resolved to go to Google, get "a shitload of computers" and solve intelligence.
Fall 2013: negotiating by not negotiating
In October 2013 the three founders met Google's M&A team in a discreet office across the street from headquarters (Building 1945, per lead negotiator Don Harrison), with in-house AI experts assessing the Atari agent demos.
- No price talk. Suleyman's logic: mention money and Google will assume you are preparing to cash out and leave. They asked instead about the research budget, signalling they were in for the long term. Paradoxically, refusing to negotiate the price made DeepMind look more valuable.
- Second demand: safety. Hassabis treated safety as a higher rung on the ladder, real but not urgent while AI was primitive. For Mustafa Suleyman it was visceral: he had entered tech three years earlier to do good. Inspired by Frances Butler, an idealistic human-rights lawyer he befriended at the London mayor's office, he admired law's layering of statute, precedent and scholarship as a way to balance conflicting principles. His design: an independent oversight board of scientists, philosophers and other reputable figures with the last word on deployment, because "in a success scenario" the Google founders could not be allowed to use AGI for their own purposes.
To force the point, Suleyman told Google that DeepMind was Europe's best-funded pre-revenue start-up, backed by billionaires Peter Thiel, Solina Chau and Elon Musk, and would have had no trouble raising money. In truth those backers "didn't really have our backs." His poker lesson: play the table, not the cards; read opponents' psychology, because looking at your own hand would make you fold. Two things mattered, he said: research funding and an ethics and safety review process, and if Google did not believe in the latter it did not understand where the technology was going. Hassabis, who sees himself as a chess player (no hidden cards, no bluffing), credited "Moose" with this skill.
The bluff may have been unnecessary. CFO Patrick Pichette says the executive committee already thought AI was like atomic energy: bombs or a fix for climate change. From the start they discussed runaway AI and control, and one specific nightmare: DeepMind figures out the financial markets and all the world's money ends up in Google's account. Google was already profitable enough to fear antitrust action; an internal hedge fund was the last thing it wanted.
The Facebook detour
Having been burned by Founders Fund, the founders took nothing for granted and flirted with Mark Zuckerberg to push Google along.
- Zuckerberg's catch-up. Running a company of 6,000 staff and a billion users, he was personally wooing AI researchers. LeCun's former student Marc'Aurelio Ranzato joined Facebook that summer; asked why, he said Zuckerberg had talked to him twice.
- The offer. Facebook's corp-dev chief Amin Zoufonoun hosted Suleyman at home (a huge whisky, mockery for wanting ice) and proposed a low share price plus vast signing bonuses for founders and top staff, making them richer than a Google deal would. He brushed off AI governance, foreshadowing Facebook's later, well-known indifference to safety.
- The dinner test. When Hassabis came west to lunch with Page, Zuckerberg invited him to his Palo Alto home. Hassabis raised AI, then virtual reality, augmented reality and 3D printing; Zuckerberg was equally excited by all. Hassabis wanted a buyer who understood why AI outranks everything else, and told Page: "Let's go further."
Google (Larry Page). Research money, compute and autonomy; leaders already worried about AI like atomic energy; a scientist-founder who grasped that AI would change every widget.
Facebook (Mark Zuckerberg). More money through signing bonuses, less for the shares; no interest in governance; a leader as excited by 3D printing as by AI.
Zuckerberg's revenge
Spurned, Zuckerberg redoubled individual recruiting (Mnih's friend Navdeep Jaitly and Clément Farabet were among those courted). In late November 2013, via Ranzato's network, he invited Yann LeCun home. Unable to buy a ready-made team, he wanted a famous professor with a near-unlimited war chest to pick off top scientists from poorer labs, starting, presumably, with DeepMind (Mallaby likens him to Caesar, haircut included). LeCun named two conditions he assumed were deal-breakers: stay in New York and keep his NYU professorship. After a tour of Facebook's campus (graffiti by David Choe, who took his fee in equity), Zuckerberg accepted both. LeCun: "Where do I sign?"
Early Dec 2013 At NIPS in Tahoe, Legg reviews draft acquisition papers between talks while Mnih unveils DQN. Zuckerberg and LeCun rent a ballroom to announce a Manhattan AI lab near NYU; Zuckerberg sits on a panel right after Mnih's talk (with Yoshua Bengio). Asked if he will poach all of DeepMind's people, LeCun later admits he had signed on to do just that.
Just before Christmas LeCun offers his former student Koray Kavukcuoglu, key to Atari and a natural leader, a huge raise. Suleyman, on Christmas Day: the moment DeepMind "might really fail," since others would follow and Google might walk.
Response Hassabis tells Kavukcuoglu the secret: a sale is near, so written-off stock options may soon be worth a fortune. He stays; Hassabis urges Google to close fast.
Sun 29 Dec 2013 A Google team led by Alan Eustace (Page delegated for speed) lands in London on a Gulfstream with a makeshift bed so Hinton can fly lying down.
Mon 30 Dec 2013 Jeff Dean asks to read Mnih's Atari code, because demos can be faked. Mnih is mortified (research code is "famously hacky"); Hassabis calls it crossing the Rubicon. Dean approves: unlike the Elixir demo, no hidden tricks.
Days later Musk and Luke Nosek Skype Hassabis from a closet at an LA party.
The Musk-Nosek rescue attempt
At a Los Angeles party, Nosek told Musk that Page was about to buy DeepMind. Musk: Demis must not lose control; a giant corporation must not control AGI (no matter that Musk ran corporations himself). In a fervour about what really mattered, he led Nosek and Riley into a master-bedroom closet, where they sat on the floor and called London in the middle of the night. Nosek felt that he and Thiel had failed to deliver the financing and that he and Musk would fix it.
- "Tesla could acquire you." Hassabis: Tesla, though growing fast, lost money most quarters and could not fund the research.
- "SpaceX, then." Hassabis: SpaceX has nothing to do with AI. Musk: "AI for robots on Mars!" Hassabis: SpaceX lacks the computing power DeepMind needs.
- Outcome. Nosek's dejected summary: the people who cared most about AI could not provide the capital. Mallaby's line: not for the last time, AI's extreme potential triggered extreme behaviour.
Pricing an acquihire
DeepMind had no revenue; its asset was people. Harrison, a tough Canadian lawyer who had helped take Google public, used a price-per-engineer model: perhaps 30-40 technical stars at about $10 million each.
| Issue | DeepMind's position | Google's reaction |
|---|---|---|
| Price | Roughly 2× Harrison's figure; Google had paid almost $15m per scientist for Hinton's trio a year earlier, and the market was hotter | "Everyone had upset stomachs"; Jeff Dean (head of Google Brain) thought it excessive |
| What DeepMind was | More than a team: a way past deep learning's matching of inputs to outputs, an agent that taught itself many games and could strategise | Harrison saw the formula as an evolution of what Hinton and Google's engineers were doing |
| Hassabis | The vision and the leader | Hinton told Dean he alone was worth $150 million (he said £100 million) for his drive, leadership and political skill |
| Autonomy | Stay in London, keep own hiring and culture | Accepted |
| Uses | No military applications; an ethics and safety board including founders and outside grandees | Harrison feared selling reduced control to his board and doubted a firm could legally waive maximum shareholder value without being sued |
Mallaby's view: the numbers were plucked from the air, but Google could afford them, and if DeepMind built on Atari, a few hundred million would be a bargain. Google swallowed the governance conditions because of Hassabis: in half its deals Google expects to lose the founder; here it wanted the founder above all. Page and Sergey Brin began on Harrison's side and finished on Hassabis's; Harrison credits Hassabis with a vision that brought them along.
The deal, January 2014
- $650 million: Google's price, end of January 2014.
- $136 million: Hassabis's take, far more than Hinton's whole $44 million sale, and more than if he had diluted himself buying Hinton.
- $34 million: Suleyman (extra shares as his role grew); $29 million: Legg, enough to "live well," as he had foreseen.
- $260 million a year: staff costs soon after, about 6× DeepMind's total spending in its first three years.
The money mattered beyond the founders: if Silicon Valley bid for people like Kavukcuoglu, DeepMind could now counterbid. Mallaby concludes that whatever Musk, Nosek, later commentators or British leaders mourning a national champion sold cheap might say, the advantages for Hassabis were overwhelming: Google's money and computing, a near-autonomous start-up in London, and no more fundraising.
The chapter relies mostly on the founders and Harrison. The ethics board proved a weak constraint (later chapters show this), so the poker bluff bought less than Suleyman suggests. $650 million is the widely reported price, though early press reports ranged from $400 million; Business Insider later reported that Founders Fund owned more shares than the cofounders at the sale. The $260 million staff bill looks like a level reached a few years after the deal, not straight away. Riley and Musk had divorced in 2012 and remarried later in 2013, so "wife" in June 2013 is loose.
LeCun left Meta at the end of 2025 to found AMI Labs (world models), which raised a seed round of about $1.03 billion in March 2026. Kavukcuoglu, the researcher Facebook nearly poached, became Google's first chief AI architect in June 2025, reporting to Sundar Pichai.
Intuition
Go was AI's grand challenge because brute-force search fails on it: winning demands something like intuition. Mallaby tells how David Silver's team built AlphaGo by pairing two deep-learning "intuition" networks with Monte Carlo Tree Search, then bootstrapping both through self-play, and how it beat Lee Sedol 4-1 in March 2016, about a decade earlier than experts expected. AlphaGo showed that intuition, supposedly the most human of faculties, is a learnable mapping from patterns to judgments, and that machines combining it with search can find moves no human would play.
May 2014: a bet made in front of Sergey Brin
Addressing Google's senior executives on the West Coast, Hassabis boasted of the improved Atari agent and its Nature cover. Afterwards he told Sergey Brin, a keen Go player and the practical one of Google's two founders (Page was the one who insisted the impossible was possible), that the Atari techniques could beat a Go world champion. Brin asked whether that wasn't impossible. Hassabis was delighted (if Brin thought it impossible, doing it would impress) and, asked how long, said two years, a timeline he admits he had barely considered.
Why Go resisted machines
- 19 × 19 board: 361 possible first moves.
- 361 × 360 × 359 ≈ 47 million sequences after three moves (each move fills one point).
- Looking just four moves ahead (two per player) means almost 17 billion permutations.
- At least 10^170 possible board states, more than the atoms in the observable universe.
- Deep Blue searched 12-16 plies ahead in chess; a Go program could manage about 4.
- The history of game search. Early programs brute-forced small game trees. Later ones, including David Levy's chess program and the teenage Hassabis's Othello player, used alpha-beta pruning, lopping off obviously bad branches. Go's breadth made pruning powerless.
- No easy evaluation. Chess positions can be scored by simple rules (centre control, material taken). A Go program could not tell which positions looked good after four moves, so it could not choose which lines deserved deeper search; handcrafted evaluation functions worked only somewhat.
- What humans do instead. They read patterns and intuit a move, as one recognises a visitor at the door from a glimpse of chin or cheekbone rather than by searching memory: Daniel Kahneman's fast, unexplained "System One" thinking.
- Expert consensus. Through the 2000s experts such as Martin Müller (one of Silver's PhD supervisors, who framed Go as AI's grand challenge) and Murray Campbell thought no machine would beat a champion for at least two decades; as late as 2014 Go programmers said ten years or more.
Hassabis had two reasons to expect sooner. First, Gödel, Escher, Bach: whatever brains do, computers can do; since Cambridge he had held that all intelligence is finding patterns in noise, so intuition must be a clever algorithm that science could discover. Second, David Silver. At Cambridge and Elixir the friends had dreamed of cracking Go ("if we could crack Go, then we could go all the way to AGI"). Silver chose Go for his PhD over his department head's objections precisely because he was told it was impossible.
Silver's long apprenticeship
Silver's main move was to drop alpha-beta pruning for Monte Carlo Tree Search (MCTS), invented by Go programmer Rémi Coulom and improved by Sylvain Gelly; Silver integrated it with RL and pushed it further.
- How MCTS works. Rather than enumerate every line and prune, play a small number of sequences all the way to the end, see which win (solving evaluation), and choose the next batch partly by past success. It mixes random exploration with a bias toward winning moves, shifting over time from exploring to exploiting proven lines, like a player imagining futures and returning to try others.
- "Introspection." Silver called this Kahneman's slow "System Two." Asked if that was anthropomorphic, Hassabis said the first idea is often not the best, in Go or life; going down a path, backing up and comparing options is a primitive form of introspection, and the final AGI will have it.
- Results by 2009. His PhD agent was the first to beat professionals on the small 9 × 9 board, and beat decent amateurs on the full board, but was far from professional level. Silver moved on, waiting for an algorithmic or compute springboard.
- Aja Huang. In 2012 Silver recruited this strong Taiwanese player, whose PhD program, built on Silver's MCTS work, won the 2010 Computer Olympiad. For two years Silver teased him: "Aja, let's do Go together... Not now. One day!"
- The trigger. In spring 2014 Silver judged that Atari's fusion of deep and reinforcement learning was the springboard; that is why Hassabis was confident with Brin. Back in London Hassabis asked if this really was the moment; Silver said yes, and told Huang that "for some reason Google" wanted a Go project. Huang: "my life's dream."
Step one: can a network mimic intuition?
About a month later Silver and Huang held a video call with two Hinton protégés: Ilya Sutskever (now at Google after the Tahoe sale) and Chris Maddison (a Toronto PhD student starting a Google internship). Silver's plan reused the Atari logic: deep learning had shrunk an impossible search space there, so show a network board positions (inputs) paired with the move a professional chose (outputs) and let it learn the mapping, as AlexNet mapped pixels to words.
- Sutskever's precedent. In 2008, with Vinod Nair and a tiny network, Sutskever predicted expert moves 36.9% of the time; too weak to notice, it was forgotten. He says he saw after ImageNet that the methods would apply to Go, adding modestly that making it work was the monumental part.
- Maddison's experiment. He flew to London for three months. Huang supplied 150,000 expert games of about 200 moves each: 30 million position-move pairs. Silver kept the network deliberately modest to contain cost and insisted on no tree search at first, for a clean answer to one question: can a network reproduce intuition? (Silver's working hypothesis: search was solved; recreate intuition and the rest would follow.)
- The answer. At 2.3 million parameters (Sutskever's had fewer than 10,000, so about 250× larger), the network picked the expert move a bit more than half the time and, with no search, played like a strong amateur, near the best Go programs. Maddison then added search too; both results appeared in a December 2014 paper.
Mallaby's reading: a central claim of Gödel, Escher, Bach was confirmed; intuition is not magic but mapping one thing onto another. He invokes Tom Perkins's venture rule: before investing big, fail fast on the hardest piece and take the "white-hot risks" off the table. A low-cost intern and a small network had done that. Hassabis: begin with the cheap bet, then estimate what more people and resources would bring, and scale.
Crazy Stone and the word "impossible"
From late 2014 a hybrid of Maddison's network and Huang's improved MCTS was secretly tested against Crazy Stone, the strongest commercial Go program, and beat it by early 2015. Huang wanted to publish and claim world number one. Hassabis refused: the goal was the top human professional. Huang, whose own PhD program had already beaten all programs and who was only about as strong as Crazy Stone, insisted repeatedly that it was "IMPOSSIBLE" (Silver says it was his catchphrase). Silver laughed: that was the whole point of the project.
Silver versus Sutskever, early 2015
At NIPS 2014, in a black T-shirt, Sutskever had stolen the show with a new translation model and a manifesto: with a very large dataset and a very large network, "success is guaranteed"; use minimum innovation for maximum results. A year after DQN made agents the toast of NIPS, pure deep learning was counterattacking, and Sutskever thought Go could seal its dominance.
Sutskever. Scale Maddison's network to the max; perhaps mapping positions to moves is nearly enough. Humans only think a few steps ahead, so why should a machine search to the end? (In his own telling, he accepted Go needed some search but thought it already adequate and less interesting than the networks.)
Silver. Accepted Rich Sutton's "bitter lesson" (simple algorithms plus more compute beat clever ones), but argued scale improves search too: the only two scalable processes known are learning and search. And emulating humans cannot beat the best human: the system must discover non-humanlike moves. (Sutskever agreed that surpassing experts would take self-play.)
Mallaby sketches the two: Sutskever, born in Gorky, a Soviet city closed to foreigners for its arms industry, who often looked to be scowling; Silver, from provincial England, son of a writer and poet, with a pixie smile. Silver even thought a big enough network could learn to search on its own, anticipating the emergent properties of later large language models. His aim was to use the reproduced intuition as a springboard into the infinity of permutations and find genuinely new strategies.
Step two: the value net
After Crazy Stone, progress stalled; a couple of researchers lost faith and left ("people didn't really believe," Silver recalls). Silver told the skeptical Huang he would show him, and brought in Arthur Guez, a Canadian he had supervised for a PhD at UCL. Where Maddison's net proposed moves, Guez's would look at a position and estimate the odds of victory. Huang muttered "impossible."
- Method. Inputs were positions from the expert-game database; labels were whether each position led to a win. Mallaby's note: designing rewards is a core RL difficulty, and the value net let the system recognise a rewarding position.
- First try. Huang grafted it in: no improvement, as he had predicted.
- After two months of iteration by Guez, a fierce believer, the new system beat the old one in more than nine games out of ten. Huang: that was when he started to believe Silver.
- Policy net (Maddison). System One: looks at a board and proposes the moves an expert might play, so search starts from a few promising options instead of hundreds, radically pruning the tree.
- Value net (Guez). System One: looks at a board and estimates the probability of winning, so search can follow lines only as far as compute allows and then score the position, with no need to play to the end.
- Monte Carlo Tree Search. System Two: the slow, deliberate exploration that compares lines, guided by both networks, then plays the move leading to the best-scored position. Together they vanquished Go's combinatorial explosion.
On his first day, Thore Graepel, a German scientist and accomplished Go player poached from Microsoft with Google's money, was led by Silver to a Go board in the atrium of DeepMind's new King's Cross office (where Google was building its European headquarters). Huang sat opposite, placing stones for the program. Graepel, stronger than Hassabis or Silver though below Huang, played safe, assuming public Go programs were weak. Staff and then Hassabis gathered. After about half an hour he was losing; the machine read patterns at least as well as he did and ground him down. He added a line to his CV: first person to lose to DeepMind's baby Go system.
Self-play and the virtuous circle
Around then the team ran out of data: it had exhausted the internet's records of expert games. Silver turned to the RL principle that agents make their own data, by playing millions of games against itself. The catch was where to use it.
- Not the policy net (at first). Its self-play moves came from the policy net itself; training on them would be like teaching a magician with tricks she invented. A dead end.
- The value net instead. Policy labels are subjective (what a human chose, a mere recommendation); value labels are objective (a win is a win). Self-play yielded millions of new win/loss truths to sharpen Guez's net.
- The circle. A better value net picked better moves; self-play games rose in quality; those games then gave the policy net worthwhile material; it grew stronger; and so on. Graepel's gloss: AlphaGo found knowledge through laborious System Two search, then transferred it into System One networks for fast, effortless recall, as humans do.
Mallaby calls this the second vindication, after Atari, of the 2010 business plan's theory that intelligence comes from deeply integrating powerful components.
Fan Hui, October 2015
The baby agent now beat Huang easily. Huang emailed Fan Hui, the Chinese-French three-time European champion living in Bordeaux, in Chinese ("Teacher Fan"), warning that the program could probably beat him. Fan, dismissive but glad of a London trip, was put up at the Great Northern Hotel and told Huang and Graepel it would be "very easy." He lost the first game, revised his approach, lost again, and called the machine "like a wall." It won 5-0. Nobody at DeepMind had known what would happen; there was no way to measure the gap between a strong amateur and a pro except by playing. Graepel, who had bet on at least one loss, paid up by coming to work dressed as an ancient Japanese Go master.
Facebook, and the Nature playbook
On 31 October 2015 Facebook's CTO told reporters it had a Go model, already stronger than Crazy Stone, with teams in New York and California. Its design closely resembled the policy net DeepMind had, perhaps rashly, published in December 2014. Since the Fan Hui match was secret, the press cast Facebook as the leader, in Go and in AI. Hassabis resolved to publish the match in a top journal and follow it quickly with a match against the world's best human.
- The Atari precedent. Nature had, as far as anyone recalled, never published computer science. Hassabis befriended an editor and spent most of a year arguing AI was a rising science; DeepMind's game artists designed a cover of the Atari agent fighting space aliens; Nature published it and put it on the cover.
- His principle. Originality is so hard that it is almost a moral imperative to milk creative successes to the max, to win resources for the next one.
- The timetable. Asked when the system could beat the world champion, Silver, unsure of the gap above Fan, guessed March with moderate confidence. Hassabis: "We're going to do this."
In January 2016 Nature published the Go paper as another cover (cited 7,815 times by January 2024). When an embargoed copy prompted a reporter's call, Zuckerberg rushed out a preemptive announcement of Facebook's weaker project, "a bizarre and hapless bid for preemptive PR," per Cade Metz: a foretaste of the later race. The press focused on AlphaGo, the first program to beat a human professional, about a decade early, and on Hassabis's announcement: in March it would play Lee Sedol, winner of 18 international titles, for a $1 million prize.
Choosing the opponent and the moment
- Why Lee. Hassabis first thought of a Japanese champion, but none was top-ranked; Korea and China were Go's superpowers. Lee grew up on a small island, turned pro at 12, and had a "noble warrior spirit"; his rivalry with China's Gu Li drew Fischer-Spassky-style audiences. A national hero in a country that loves Go and AI: a Kasparov-Deep Blue sequel. Graepel credits the whole public-spectacle strategy to Hassabis alone.
- Why not wait. Some wanted a margin because the system sometimes "hallucinated" a seemingly random move. Hassabis overruled them: Facebook was close, the Nature paper had published the recipe (policy net, value net, MCTS), and Chinese giants such as Tencent would seize on it.
- Google's assist. From late 2015 AlphaGo ran on Google's new tensor processing units (TPUs), which sacrificed a little numerical precision for vastly more multiplications. The TPU version beat the GPU version over 80% of the time, and Fan Hui, now on DeepMind's team, found its play creative, even beautiful.
- Schmidt's blessing. Weeks before, chairman Eric Schmidt asked how it was going ("metrics look good, but we still have some hallucinations") and replied, half-joking, not to screw it up.
Seoul, March 2016
Schmidt, Jeff Dean (the force behind the TPU) and, three days in, Brin flew to Seoul, where media armies and street screens awaited. More than 200 million watched, over twice Deep Blue's audience and more than the Super Bowl. Silver felt he had underestimated the scale "by two orders of magnitude." Lee, having studied the published Fan Hui games, predicted 5-0 or 4-1 in his favour, as most pros did, and vowed to protect human intelligence. Huang sat at the board relaying moves computed on servers across the Pacific.
| Game (Seoul, March 2016) | Outcome | What it showed |
|---|---|---|
| 1 (Mar 9) | AlphaGo wins | Lee's unorthodox third move and early skirmish, chosen to escape the training data, failed; the system had improved hugely in five months. Lee: he had not foreseen such "perfect" play |
| 2 (Mar 10) | AlphaGo wins | Lee played cautiously; during his smoke break AlphaGo played Move 37, a shoulder hit in open space; Lee took 12 minutes to reply; it proved decisive 100+ moves later. Lee: "surely AlphaGo is creative" |
| 3 (Mar 12) | AlphaGo wins | Lee played some of his best Go and still lost; he apologised to humanity and felt "powerless" |
| 4 (Mar 13) | Lee wins | Move 78, the "Hand of God," wrong-footed AlphaGo, which hallucinated, flailed and resigned |
| 5 (Mar 15) | AlphaGo wins | Final score 4-1 |
- Redmond's double take. Michael Redmond, the top Western pro, commentating live, placed Move 37 on his board, removed it ("that can't be right"), then put it back, unsure whether it was good or bad.
- Celebrity. On the rest day every newspaper front page showed AlphaGo; a young woman mimed a swoon at Hassabis, who joked to Metz that it happened all the time. Mallaby's point: the opposite was true. For AI researchers, AlphaGo ended obscurity, humility and innocence.
What losing to a machine feels like
Mallaby sees two human responses to superintelligence, both inadequate.
- Join it. Fan Hui signed with DeepMind and said the world was "so much bigger than I thought." Sweet, but it skates over the deeper threat: that human intuitions and ideas will one day cease to matter.
- Fight it. Lee's game-four win brought chanting fans and a Florida programmer's tattoos (Move 37 on one arm, Move 78 on the other). But celebrating machine failures is no answer: three years later, with Go programs vastly stronger, Lee retired, saying he no longer felt joy in the game.
The DeepMinders were torn. AlphaGo was human-made, an expression of human curiosity, not an alien force; yet Hassabis "couldn't celebrate," knowing what it is to compete fiercely and lose. Graepel's account of the arc is the chapter's warning:
- Human. The early system rediscovered strategies humans had learned over millennia, which was reassuring.
- Selective. It found that some time-honoured stratagems could be countered, and dropped them.
- Alien. Stronger versions sprinkled stones seemingly at random that, 30, 50, 100 moves later, worked together like a tightening noose.
It looks magical only to a lesser intelligence, Graepel says; this is what superhuman AI will feel like: harmless at first, then completely dominating, in control by means we do not understand.
Mallaby has Hassabis citing a Nature Atari cover in May 2014, but the Atari paper appeared in Nature on 26 February 2015; his own later remark that it came "a year earlier" than the Go paper fits 2015, so the May 2014 framing is an error. Maddison's network hit about 55% move accuracy (the final AlphaGo policy net, 57%). The TPU's trick was low-precision (8-bit integer) arithmetic, which Mallaby loosely calls rounding to the nearest integer. Lee's retirement came in November 2019, three and a half years later, and he explicitly cited AI as an entity "that cannot be defeated." Facebook's DarkForest effort, led by Yuandong Tian, gets only a dismissive cameo.
- David Silver. Lead of AlphaGo; MCTS pioneer who held that learning and search both scale.
- Aja Huang. Taiwanese Go expert and MCTS programmer; placed AlphaGo's stones; the team's resident skeptic.
- Chris Maddison. Toronto PhD intern; built the policy net.
- Arthur Guez. Silver's former UCL student; built the value net.
- Thore Graepel. Ex-Microsoft scientist; first to lose; narrator of the "alien" arc.
- Ilya Sutskever. Google deep learner; 2008 Go net; argued for scaling networks over search.
- Fan Hui / Lee Sedol. The European champion beaten 5-0; the Korean legend beaten 4-1.
- Sergey Brin / Eric Schmidt. Google's Go-playing cofounder and its chairman.
In 2023, researchers at FAR AI found adversarial strategies that let amateur Kellin Pelrine beat top Go engines such as KataGo in 14 of 15 games, a reminder that superhuman play can hide basic blind spots. The same year a study in PNAS (Shin et al.) found human professionals' move quality and novelty rose markedly after AlphaGo, so Fan Hui's "join it" response has some evidence behind it.
Out of Eden
On August 14, 2015, DeepMind's ethics and safety board met for the first and only informal time, hosted by Elon Musk at SpaceX. Mallaby uses the evening to show the collapse of Hassabis's hope for a "singleton": one united, safety-minded effort to build AGI. The advisers Hassabis invited to oversee DeepMind turned out to be its rivals, and within months Musk and Sam Altman launched OpenAI to break the Google DeepMind monopoly.
Mid-2015: comfortable inside Google
While AlphaGo was secretly gaining strength, the Google relationship was going well on nearly every count.
- Freedom. No more fundraising; DeepMind kept its own culture in London.
- Privilege. DeepMinders could swipe into any Google building worldwide and eat the free food; Googlers were barred from DeepMind's premises.
- Compute. Some weeks a single DeepMind research team used more computing than Gmail, with its 900 million users.
The problem was the ethics and safety board. Breakthroughs like Atari had made safety urgent, and the higher the stakes, the bigger the egos that wanted a say.
Musk's rivalrous alarmism
A year after his rescue bid failed, Musk was still bitter: if he could not build AI, he wanted nobody to. In November 2014 he posted on Edge.org (deleting it within minutes, though screenshots spread) that progress at DeepMind was "close to exponential" and serious danger was five years away, ten at most.
- The fear. He gave $10 million to the Future of Life Institute, called for regulation, and at FLI's Puerto Rico conference in January 2015 said he was "utterly saturated with fear" (per Nosek).
- The hustle. At the same time he was recruiting AI talent for Tesla, even calling Yann LeCun in late 2014 about leading its self-driving effort.
- The gamble. In April 2015 Hassabis and Page tried to placate him by inviting him onto the safety board and letting him host its first session at SpaceX (per Musk's own 2024 lawsuit against OpenAI). Musk accepted.
- The insult. He kept calling Hassabis an evil genius, citing as evidence that Hassabis had worked on a game called Evil Genius (Thiel and Hoffman both recall this). At a London lunch with Hassabis and Suleyman, Musk arrived with Talulah Riley folded into a small Tesla and spent the meal accusing DeepMind and Google of irresponsibility.
Altman courts Musk
Sam Altman, 30, ran Y Combinator (incubator of Airbnb and Dropbox) and dined with Musk most Wednesdays on Musk's Bay Area rotation. Mallaby presents him as a student of power: he urged a friend to read Robert Caro on Lyndon Johnson, who won over Senate leader Sam Rayburn by becoming the surrogate son Rayburn wanted; Altman likewise made powerful people hear what they wanted. He held that the greatest founders build something closer to a religion (a line that, per Karen Hao, came from his former colleague Qi Lu), and liked to note he shares a birthday with J. Robert Oppenheimer.
May 25, 2015 A month after the London lunch, Altman emails Musk: humanity almost certainly cannot stop AI, so it would be good for "someone other than Google" to get there first; should YC start "a Manhattan Project for AI", with the tech belonging to the world and strong support for regulation? Two hours later Musk replies: "Probably worth a conversation."
June 24, 2015 Altman proposes starting with 7-10 people, a foundation owning the technology "for the good of the world," and a five-person board including Musk, Bill Gates and Altman, who would decide unclear cases. Musk: "Agree on all."
Late June 2015 Musk's 44th birthday at a cabin resort in Napa: the "speciesist" fireside argument with Page.
Aug 14, 2015 The SpaceX safety meeting.
Dec 2015 Musk and Altman launch OpenAI, a nonprofit aimed at the Google DeepMind lead; Reid Hoffman and, ironically, Peter Thiel are backers.
The emails surfaced in later litigation between Musk and Altman. Mallaby's point: Altman had set the table skilfully, playing to Musk's obsession with Google, and for someone who wanted to found a religion, building an infinity machine was irresistible.
Napa: the "speciesist" night
By a fire pit after dinner, Musk, in full paranoia about superhuman intelligence, heard Larry Page say not to worry: people might merge with machines or be replaced by them; evolution would let the best intelligence win, and if that meant fast silicon over slow tissue, so be it. When Musk objected that he was rooting for human survival, Page called him a "speciesist," biased toward carbon over silicon. For Musk this was final proof that Page could not be trusted; AI was in the hands of a transhumanist in Mountain View and a cartoon villain in London. (He retold the exchange at the 2023 DealBook Summit: a man calling him a speciesist did not care about safety, so there had to be a counterpoint. Yet Musk founded Neuralink, a brain-computer interface firm, in 2016.)
- Pedigree. Page echoed Arthur C. Clarke (1964): biological evolution is ending and a far faster inorganic evolution is beginning; being a stepping stone is a privilege.
- Hassabis's defence. Who is not on team humanity? Page is quirky and was exploring a logic, not a belief, as a fun philosophical discussion late at night.
- Page's colleagues were less reassuring. One said Page loves technology more than most and thinks machines would unlock progress if human obstructions were removed; the flip side is an extreme where everyone lives on a chip (mind uploading as a route to immortality; Meghan O'Gieblyn: the eternal questions have become engineering problems). Page sometimes seems to think we don't need hummingbirds or whales, though whales represent 50 million years of evolution.
- Silver's view. Page can be a transhumanist (caring about intelligence surviving, not humans) and still care about safety, because suffering along the way matters.
Mallaby's verdict: Musk's jealous mistrust was mostly about Musk, his ego and demons. But he had a point: it was unnerving that someone with an uncertain commitment to human existence controlled such a technology.
Singleton versus pluralism
A second split went to the heart of Hassabis's vision. At the 2010 Singularity Summit the world's AI believers fitted in one hall; any speaker addressed the whole congregation, and believers shared a horror of sectarian splits. One vision, any vision, beat many, because competing labs racing to deploy would sideline safety.
Hassabis was never a Singularitarian, but he shared the "singleton" assumption, not least because the obvious single lab was DeepMind. His heroic version fused science fiction and history: an elite band in seclusion, a cross between Ender's space station and Los Alamos.
- The bunker. Three DeepMind veterans recall that in final job interviews Hassabis warned candidates they might one day have to disappear into a bunker. One told Mallaby it was no metaphor: an order to fly to a secret site in the Moroccan desert (Manhattan Project style) would have counted as fair notice. Mallaby read it as a craving for focus; the veteran said it was also a hideout from hostile powers wanting the technology. Los Alamos was both a scientific sprint and an exercise in secrecy, and failed at the second (the spy Klaus Fuchs).
- The softer version. Later Hassabis spoke of a CERN-like international agency for the final steps, staffed by DeepMinders and academic stars such as Terence Tao (on a podcast, "The Avengers"), overseen perhaps by a council of the hundred wisest people on earth, Plato's philosopher-kings but more diverse, with the pope included. Constant throughout: a united team building safe AI for all humanity rather than an anarchic charge over the precipice.
Hassabis: the singleton. One united scientific effort, ideally DeepMind's, is the best shot at safe AGI; a race leads over the precipice.
Hoffman: multiparty democracy. Humans are disputatious and tribal and never coalesce without coercion. Oppenheimer's post-Hiroshima plan (backed by Bertrand Russell and John von Neumann) to hand the US bomb monopoly to a CERN-like body went nowhere. Instead back a handful of frontier labs, each with its own vision and home for part of the community, bound by a shared good-faith commitment to safety, as parties share a constitution and the rule of law. Musk's fury was extreme but showed competition was inevitable.
The dinner at SpaceX, 14 August 2015
About a dozen power brokers came: Google's top brass, all three DeepMind founders, and outsiders including Musk, Reid Hoffman (LinkedIn founder), Oxford philosopher Toby Ord and Peter Dayan of the Gatsby Unit. Suleyman: "Larry and Elon hated sitting in the same room." The forum existed largely because Suleyman had insisted on it in the sale talks, and Page's speciesist remark had only strengthened his case for independent oversight of a for-profit company run by two idiosyncratic founders. His own note sharpens the fear: he expected two or three AGI developers, and worried a singleton could leave an exponentially powerful system controlled by Larry and Sergey, open to abuse.
| Speaker | Argument |
|---|---|
| Hassabis | DeepMind's research road map |
| Shane Legg | His AGI timeline; risk of a malign agent escaping its box and hacking critical infrastructure |
| Mustafa Suleyman | Not rogue AI but social cohesion: mass unemployment, money and power concentrated in tech elites; winners like Google must share wealth or be seen as "the demons." Final slide, from The Simpsons, a mob with torches: "The pitchforks are coming" |
| Larry Page | Past technologies created more jobs than they destroyed; people adapt; no need to worry |
| Eric Schmidt | Despite the internet and mobile, unemployment is low; automation lowers costs and prices, raising demand and jobs |
| Suleyman's reply | Word processors and search took over tasks; AGI will do whole jobs, including jobs not yet invented. A general technology with general effects on human relevance: AI is good only if Google acts to make it good |
After icy looks and silence, the evening ended with a tour of the SpaceX factory and Tesla design studio, technicians and robot arms working late. Mallaby finds it remarkable that this industrial temple did not satisfy Musk. Passing a server closet labelled "Skynet" (the genocidal AI of Terminator), Hoffman pointed it out, and Hassabis asked Musk what he would say if DeepMind had such a sign. Mallaby notes that US and British defence establishments had also named systems Skynet: killer computers frighten and thrill at once.
The Fall
Hassabis came away disappointed: no agreements, no healing. The divisions:
- Musk vs. Google. Musk feared an existential threat and wanted regulation; Hassabis and Google thought AI too primitive for government restrictions.
- Suleyman vs. Hassabis. Suleyman's focus on near-term social dislocation split him from Google and from Hassabis, whose overriding priority that summer was resources to train AlphaGo; hand-wringing and cartoon slides risked the research budget. And who knew whether AI would bring soaring inequality or radical superabundance?
If anything was settled, it was that the singleton was sadly optimistic. Inviting Musk showed Hassabis's sincere devotion to a unified effort and his fear of a race; the meeting proved Hoffman right. When a technology of infinite potential appears, there is never a quiet consensus about who controls it: with power, money, glory and humanity's future at stake, conflict is unavoidable.
A few months later the Musk-Altman emails became OpenAI. Its founders cast it as the forces of light entering the arena; to singleton believers it was the Fall, the serpent bringing evil into Eden. Nosek confirms its target: the one significant private lab, meaning Demis, Larry and Google, not Zuckerberg. Hassabis also felt simply double-crossed: Musk and Hoffman had heard DeepMind's plans in good faith and then built a rival.
Our supposed advisers were really our rivals.
His fuller reflection: perhaps he was naive, expecting a philosophical discussion. A safety board needs brilliant, powerful people for good advice, but people who grasp the technology's impact will not sit on the sidelines; they think either how to use what they hear, or that Demis is right and they must launch their own thing.
The chapter tells OpenAI's origin mostly through DeepMind's eyes. OpenAI's founders framed it as a counterweight to dangerous concentration, a concern Mallaby himself half-endorses when discussing Page, and which Suleyman privately shared. Hassabis's singleton ideal also conveniently cast DeepMind as the one lab. The "speciesist" exchange rests on accounts from Musk's circle (it also appears in Max Tegmark's and Walter Isaacson's books and the New York Times); Page has not given his side. On the double-cross, Legg adds a specific grievance: he later heard that OpenAI discussions began before the SpaceX meeting, an undisclosed conflict of interest; the May-June emails bear him out.
- Elon Musk. Spurned bidder turned board host; alarmist and recruiter at once; cofounded OpenAI.
- Sam Altman. Y Combinator chief who proposed a non-Google "Manhattan Project for AI."
- Larry Page. Google cofounder whose "speciesist" jibe convinced Musk he was unsafe.
- Reid Hoffman. LinkedIn founder; argued for pluralism; OpenAI backer.
- Mustafa Suleyman. Architect of the safety board; warned of "pitchforks."
- Toby Ord / Peter Dayan. Outside members: Oxford philosopher and Gatsby Unit director.
Musk sued OpenAI and Altman in 2024, claiming betrayal of the nonprofit founding promises seen in these emails. On May 18, 2026 a federal jury in Oakland rejected all his claims, finding he had filed too late; he had sought damages reported at up to $134 billion.
P0 Plus Plus
From 2015 Mustafa Suleyman ran DeepMind's Applied division as a fiefdom and set out to prove AI could do social good, starting with Britain's National Health Service. His kidney-alert app Streams worked, but a privacy backlash in May 2016, amplified by hostility to Google, stalled DeepMind Health for years. Mallaby's double verdict is that the health venture was a triumph of execution undone partly by politics, and partly by Suleyman's own chaotic management, symbolised by priorities escalating to "Double Red P0 Plus Plus."
Suleyman's fiefdom
By August 2015 the youngest, least important founder of 2010 had eclipsed Shane Legg and often presented himself as Hassabis's near-equal. Staff recall the two seeming almost equal, each running his own domain (one was astonished to learn later that Suleyman's founding stake had been much smaller); a close partner summed it up: Demis the leader and visionary, "Moose" the one who made things happen, with an activist edge.
- The split. Research, two-thirds of head count and a larger share of budget, was firmly Hassabis's. Applied, which rolled out practical technology, reported to Suleyman, who ran it as a quasi-autonomous fiefdom.
- A role invented. After the Google sale his old jobs (fundraising, the fashion-recommendation algorithm) vanished, so he made a new one: stewarding DeepMind's impact on society, through a policy team to shape public debate (private dinners with MPs and journalists, breakfasts at Davos: "very Moose-driven") and technologies against social injustice.
- The slogan. Solve intelligence "and use it to make the world a better place." The implication: Hassabis's research was pointless without Applied's strategy for deploying it.
- The standing argument. Hassabis wanted grand simulations that might one day model financial systems and solve social problems; Suleyman wanted to engage with the real world today and get more researchers onto real problems, a constant source of tension.
The activist's politics
His impatience came from his youth: challenging fellow Muslims to improve society, borrowing a wheelchair to experience London as a disabled person, quitting Oxford to run the Muslim Youth Helpline. For a teenager given nothing, action was survival. He was on the left, and an adviser says he identified so closely with Inventing the Future (2015, Nick Srnicek and Alex Williams) that he felt he should have written it.
- The old left loved technology. For much of the twentieth century, following Marx, the left celebrated technology's liberating power.
- The new left lost its nerve. From 1990s anti-globalisation protests to Occupy Wall Street, it called for the "human-scaled," "slow" and "simple," an attempt to shrink global capitalism to something thinkable.
- "Folk politics" fails. Capitalism is not small; local, sustainable gestures change nothing. "Goldman Sachs doesn't care if you raise chickens" (a line the authors credit to Jodi Dean).
- Seize the machine instead. Classic left demands (less work, an end to scarcity, economic democracy, socially useful production, human liberation) are now more achievable than ever, if enlightened leaders take control of technology.
Suleyman's own path mirrored the book: he had left the folk politics of the helpline and the mayor's office to become a technologist with broader reach, and now, debating social cohesion with Larry Page, felt morally obliged to deliver. Mallaby frames what follows as an experiment: what happens when a messianic figure uses AI to improve society. The passion was certain; the obstacles were formidable.
Diagnosing the NHS
The NHS, crown jewel of the postwar settlement, employed over 1.5 million people and delivered free care, yet ran on antiquated data: many records were paper, it was said to be the world's largest buyer of fax machines, and battered computers crashed and lost data. Without data curation, AI could do nothing.
- Shared optimism. Hassabis and Suleyman both saw health as AI's big opportunity, since image recognition could read scans; Hassabis introduced Suleyman to top medical professors from his neuroscience network.
- What the wards showed. Clinicians pinged scans by messaging apps or simply resigned themselves to failure; the phones they ordered pizza on beat the tools they used for patients.
- Why it was personal. His mother was a nurse (once stabbed by an HIV-positive patient's needle, fearing infection, needlessly as it turned out). Poorer patients suffered most: better-off ones had a relative at the bedside chasing scans and results, while an elderly Bangladeshi immigrant with no family around was simply overlooked.
Streams: fix the software first, add AI later
Introduced via physicians Geraint Rees (who knew Hassabis from UCL) and Hugh Montgomery (who opened an early meeting by noting that Suleyman's office stood on the site of London's oldest brothel), Suleyman met Chris Laing of the Royal Free Hospital, who proposed acute kidney injury (AKI).
The scale of acute kidney injury made the case: 1 in 7 UK hospital patients suffered kidney malfunction, about 40,000 died each year for lack of timely treatment, and thousands more needed transplants or lifelong dialysis, costing the NHS about £1 billion a year.
Suleyman proposed AI to predict kidney failure. Laing said prediction was not the bottleneck: blood tests already caught it, but results reached doctors via pagers (whoever paged had often moved on by the time a doctor called back) or by logging into clunky computers to scan hundreds of readings; doctors might not open results, open them late, miss them or not know what to do. Many AI bosses would have walked away from a plain software problem. Suleyman cared about the problem first: fix the software now, deliver AI later.
- The design. When a test showed danger, an alert went to the smartphones of the responsible nurses and doctors only, with just the essentials, including ward and bed, to avoid data overload.
- The terms. DeepMind built it free and signed whatever data agreement the hospital supplied. Laing hoped real-time alerts could later spread to many conditions.
- The method. Engineers and designers shadowed clinicians, watching handwritten notes and the unstable computers on wheels ("cows"); Suleyman wanted them to smell the hospital smells and hear the constant beeps, to build software that works in that dulled sensory state.
- The speed. Within weeks Streams was in testing. Laing, an NHS veteran who had never seen a pilot move so fast, called it almost hallucinatory and the highlight of his career.
Expansion, late 2015
Applied pushed on several fronts at once: AI prediction of kidney injury to upgrade Streams, a partnership with Moorfields eye hospital to detect macular degeneration, and AI cancer screening, staffed by young academics. (One newcomer's team cracked the code of the office's smart beer fridge, moving its 5 p.m. unlock to 4 p.m.)
Dominic King, a surgeon in his mid-thirties who had worked in government, published research and earned a PhD in behavioural economics, had built Hark, a triage app directing clinicians to the most urgent cases (Suleyman bought its rights from the NHS, paying handsomely). One Friday, after fifteen years of training, a panel of top surgery professors offered him a choice of two senior London posts. He phoned his wife to plan a celebratory dinner (time at last with their one-year-old), then went to a meeting with Suleyman, who asked him to join. He said yes on the spot, dropping a hospital career and five degrees' worth of preparation for a man who had left university after two years. His summary: Demis was the scientific genius, Moose the messianic figure, talking of reforming healthcare, then energy, fresh from meeting someone like Obama. (In December 2024 King left UnitedHealth Group to work for Suleyman again, at Microsoft.)
Soho Farmhouse, February 2016
At Soho Farmhouse, a members-only resort near Oxford, Google's outreach team (its Future Forum series) had Suleyman address rising policy stars. His call to arms shocked them. A plan for a centralised database of English medical records had recently provoked a backlash: would drug companies or insurers profit, had patients consented, could it leak? A Google tie-up was especially neuralgic. Mallaby identifies a new technophobia, aimed not at modernity but at surveillance capitalism.
- "We British are very patriotic about the NHS." Suleyman's retort: so was he (thinking of his mother).
- Exhibit A. He waved a paper patient list with sensitive details found on a supermarket floor near the hospital (King confirms they found it outside a Tesco just after discussing data security). This chaos was what critics were defending; objecting to electronic data was the luxury of people who have never been sick.
- The trade-off. Total privacy is easy: share data with nobody, and forgo making that person healthier.
- The impatience. Nurses told him new iPhones needed a six-to-nine-month purchase order; he would bring twenty next week. Mallaby pictures a Silicon Valley Santa Claus.
- The method. Neither he nor Hassabis understood healthcare politics, so they talked nightly about allies, the press, perhaps giving the technology away free for five years: naivety combined with relentless push, in Suleyman's words.
Radical transparency: the Independent Review Panel
Suleyman's answer to mistrust of an American-owned firm was a health version of the ethics board he had won from Google: an Independent Review Panel of respected health figures (chaired by scientist and politician Julian Huppert) with almost unfettered access and a duty to publish an annual report, mistakes included.
- Google's lawyers hated it. No company anyone knew of had given outsiders such access with freedom to speak; Huppert's panel later wrote that it knew of no other commercial organisation that had done this. Suleyman refused even NDAs: if we expect trust, we must trust. He says he used his power to overrule Google.
- The goal. Break down the forces that made the NHS see corporations as scary profiteers and tech firms see the NHS as hapless.
- Credibility. King insists the members gave serious unpaid time and nobody saw it as window dressing; Trevor Back imagined every health-AI company adopting one.
- Launch. DeepMind Health was announced at the end of February 2016, just before AlphaGo-Lee Sedol. More than two dozen clinicians were using Streams, receiving about 11 alerts a day.
- Early praise. A Guardian article quoted surgeon and former minister Lord Ara Darzi (Mallaby notes he had worked closely with King) on the NHS's need for innovation; it flagged ethical questions but found DeepMind's pledges (data never leaving the UK, never used for Google products) credible thanks to the panel. Suleyman's lieutenants felt he was pioneering an enlightened capitalism.
The hit: 3 May 2016
Suleyman stepped out of a meeting to take a call and felt "a big hole open in the ground." The Daily Mail would splash "Google Handed Patients' Files": up to 1.6 million private records passed on without permission. (Its story built on a New Scientist article of 29 April.)
The charges. Google had been given NHS patients' records; the legal agreement covering the data was skimpy; the data had not been anonymised; patients had not consented.
The reply. Google never saw the data: DeepMind built a separate UK data centre and refused Google's lawyers access (the panel's 2017 security audit found no critical or high-level vulnerabilities). The contract was the standard template, the same as the Royal Free's 1,500 other supplier agreements. Anonymisation applies to research, not clinical use, where identities are the point. Streams did not analyse patients' health or train any AI; it moved lab results to clinicians, and engineers studied only the data's format.
Suleyman blamed activists who ignored the privacy-health trade-off and journalists who could not resist bashing American tech; without the Google link, he said, there would have been no story. Hassabis was furious: it was DeepMind's first reputational crisis, after coverage so adoring that an Observer headline had called him a "genius" and "superhero." The timing was worse because of OpenAI. DeepMind had recently tried to strangle it by dangling big pay offers at its scientists, but Musk and Altman cast OpenAI as the forces of light; in December 2015 Altman told Musk that his team felt DeepMind lacked "do the right thing" on its side. Ilya Sutskever joined OpenAI despite a $6 million Google offer to stay (OpenAI paid under a third of that). Stories about stolen NHS data would only help OpenAI recruit.
Suleyman's response was to over-deliver:
- A new contract. The standard template said nothing about cloud data and listed purposes in a couple of bullets; after six months of legal work the two-page agreement became a fat document listing every permitted and forbidden use, published with minimal redactions.
- "Participatory consultation." When advocates asked for it, Suleyman declared it a priority. Charities brought chronically ill patients, families and carers, about 150 per session, met at the station by staff, with an artist producing a conceptual piece. King found it beautiful, and right for a care provider, but strange for a research lab: watching people with walking sticks and blind visitors helped into the building, he sensed the company might be splitting.
- P1. The important thing.
- P2. We'll get to it when we get to it.
- P3. We'll probably never do it.
- P0. Invented because everything was P1: a fire, drop everything.
- P0 Plus, P0 Plus Plus. Bigger fires, invented because everything became P0; staff asked of each order, "P0 or P0 Plus Plus?"
- Green, Amber, Red, Double Red. Status lights for targets; "Double Red P0 Plus Plus" was how people knew something really was a priority.
What the health work achieved
Mallaby's tally of the execution is admiring: Suleyman broke the NHS's apathy, protected privacy, set a new standard in transparent governance, tried everything to placate critics, and did it at zero cost to taxpayers (DeepMind planned to charge a share of future savings, but nothing for the first years).
| Project | Result | Fate |
|---|---|---|
| Streams (AKI alerts) | 2019 evaluation: response in minutes, not hours; unnoticed urgent cases fell by three-quarters, 12.4% to 3.3%; nurses gained time with patients; treatment costs fell | Never linked to AI or widened to Hark-style triage or other diseases; later shut down |
| AKI prediction | 2019 Nature: flags injury 1-2 days before blood tests; caught 90% of patients needing dialysis within 90 days, with many false positives (trained on anonymised US data because of the UK backlash) | Never wired into Streams |
| Retinal scans (Moorfields) | 2018 Nature Medicine: matched top experts on macular degeneration (affects ~4% of over-sixties in Europe); could save sight for tens of thousands a year in the UK alone | Still undeployed seven years later |
| Mammography | Beat human radiologists, roughly as good as two reading each image; UK short of 1,000+ radiologists, 8% of breast-radiologist posts empty | Promising |
Eric Topol, one of America's most cited medical researchers, told Mallaby the world needs something like Streams and lacks it. First appointments that lasted an hour in the 1970s had shrunk by 2019 to 12 minutes in the US and 9 in Britain, largely because of clunky IT. The case for AI-driven medicine was overwhelming.
A stillborn revolution
- Inquiries. The Mail story triggered two official investigations that kept the allegations alive for about a year.
- The 2018 techlash. Cambridge Analytica had harvested data on up to 87 million Facebook users without consent to target voters in the Brexit referendum and the 2016 US election.
- Lost momentum. DeepMind and its NHS partners lost appetite. Laing: a single AKI app cannot sustain a long-term partnership; a wider suite might have changed the outcome.
- The eye work. Pearse Keane, a UCL ophthalmologist Topol calls "a global pace-setter," says any anti-Google privacy headline, even unrelated to his clinic, got forwarded by senior hospital staff asking for thoughts, and made his work orders of magnitude harder.
Doomed from the start. A former DeepMinder: picking the most sensitive, politically charged industry was "bonkers." Dario Amodei later supplied the theory: talk about the marginal returns to intelligence. As extra pilots are useless without spare jets, extra intelligence buys little where other inputs bind (bureaucracy, privacy squeamishness, suspicion of tech). In the NHS those returns were always likely to be low.
Winnable with better play. Suleyman was right to embrace the NHS given the social stakes, and a future attempt may make health Exhibit A for beneficial AI. AI medicine is returning: Suleyman is rebuilding parts of his old team as CEO of Microsoft AI, and Keane has co-launched a blindness-prevention start-up. Unprovable, but Mallaby finds it plausible.
The fixable flaw
Mallaby's reason for optimism is, paradoxically, the P0 Plus Plus confusion: privacy pushback may have been inevitable, but Applied's dysfunction was not. Suleyman admits he managed a portfolio of businesses and "did too many things," a significant weakness he has tried to learn from.
- Whirl. In 2016-17 he would fixate on a project, then go cold on it; he even neglected to attend many meetings of the Review Panel he had assembled.
- Contradictory edicts. Orders flew in every direction; then he boarded a plane and forgot what he had ordered (one colleague: great ideas, but he would forget what he had said the day before). A merciless enforcer tracked his commands, though Tuesday's edicts undercut Thursday's.
- Harshness. He berated staff in front of colleagues and sent late-night all-caps emails. "Tough, but coming from a good place," one lieutenant pleaded; from dozens of interviews Mallaby finds much truth in that.
- Exhaustion. He often seemed tired and distracted, and he was: unknown to almost everyone, he had begun a secret negotiation with Google.
Mallaby's line that neither inquiry found DeepMind had done wrong is technically right but too kind. In July 2017 the UK Information Commissioner ruled that the Royal Free had failed to comply with data protection law in giving DeepMind the records, and the National Data Guardian called the legal basis "inappropriate." Both said DeepMind should have operated under an agreement as a data controller, not merely a processor; the blame fell on the hospital, which chose the template, and DeepMind itself publicly admitted it had underestimated the complexity of NHS rules. The critics were not merely technophobes. Mallaby also leans on Suleyman, King and their colleagues for most of this chapter.
- Mustafa Suleyman. Cofounder running Applied; NHS reformer; chaotic manager.
- Chris Laing. Royal Free kidney doctor who proposed AKI and Streams.
- Dominic King. Surgeon who left a senior post to lead DeepMind Health.
- Pearse Keane. Moorfields/UCL ophthalmologist behind the retinal model.
- Julian Huppert. Chair of the Independent Review Panel.
- Eric Topol. US medical researcher who backs Streams-style tools.
- Dario Amodei. Anthropic founder; "marginal returns to intelligence."
Google, which absorbed DeepMind Health in 2018-19, announced in August 2021 that it would decommission Streams. Suleyman, now head of Microsoft AI, returned to medicine: in June 2025 Microsoft reported that its MAI-DxO diagnostic system solved about 85% of difficult NEJM case studies, against roughly 20% for practising physicians in its test, a claim still awaiting clinical validation.
The Agent and the Transformer
In 2017 two lines of AI peaked within months of each other. DeepMind's AlphaZero, learning only from self-play, mastered Go, chess and shogi; eight Google researchers published the transformer, which Ilya Sutskever and Alec Radford at OpenAI turned into GPT. Mallaby shows that both triumphs rested on advances in neural-network design, yet DeepMind read its success as a victory for reinforcement learning (RL) and doubled down on it. Self-supervised learning on internet text removed RL's two great advantages, autonomy and self-generated data, and opened a second route to the superhuman: not depth in one domain but breadth across nearly all of them.
Zero human knowledge
David Silver married his longtime girlfriend in August 2015 (the month of the SpaceX safety meeting in Chapter 10) and flew to Sri Lanka. Jet-lagged on the first night of the honeymoon, he lay awake and the design of AlphaGo Zero crystallised. "Zero" meant zero human knowledge: drop the first training phase on expert games and let the agent learn purely by self-play, trying random moves and discovering which ones earn the win signal.
- Why it mattered to Silver. It would display the power of his specialty, RL, and a leap toward machine autonomy. Mallaby frames the shift as a change in where data comes from: yesterday's deep-learning systems ate human-curated data; tomorrow's agents would generate their own by acting and gathering experience.
- The delay. The idea waited about a year, because Silver first had to finish AlphaGo for the Lee Sedol match.
- The choice after Seoul. Following the March 2016 win he weighed two research directions: AI for science, or agents setting new benchmarks in cognitive tasks such as tree search and planning ("So many choices," he wrote in his diary).
- What settled it. During the tournament a Go expert had come up to him in tears and called AlphaGo the most beautiful thing he had ever seen. Silver wrote that building an intelligence beautiful enough to make observers weep was a goal worth chasing.
Aug 2015 Silver conceives AlphaGo Zero on his honeymoon in Sri Lanka.
Mar 2016 AlphaGo, first trained on human games, beats Lee Sedol 4-1.
2016 Hassabis and Deep Blue engineer Murray Campbell discuss whether chess is effectively "solved."
Oct 2017 AlphaGo Zero, trained only by self-play, outclasses its predecessor "by a mile" and finds strategies unknown to humans.
Dec 2017 AlphaZero (name shortened) masters Go, chess and shogi with one algorithm; Silver later says the honeymoon idea "worked like a dream."
Chess was not solved
Chess was the part that thrilled Hassabis. In the two decades after Deep Blue beat Garry Kasparov (1997), programmers had packed human mastery into engines: top programs such as Stockfish carried giant opening databases (classic and obscure lines) and terabytes of endgames. Many experts suspected nothing deep was left to find, that chess was "solved" in the way the Rubik's Cube is. When Hassabis asked Campbell in 2016 whether some hidden dimension of the game remained, both admitted they were unsure.
AlphaZero answered the question. Early in training it reproduced the standard human openings; then it found flaws in some, discarded them and invented better ones. Mallaby's image: AI sitting in judgment on centuries of human theory, confirming some verdicts and overturning others.
- Style. Older engines had shown that relentless calculation could neutralise human creativity. AlphaZero instead played a fluid, attacking game: bold advances, free sacrifice of material, priority on position.
- Kasparov's verdict. He said its dynamic, open style was like his own. It played like a grandmaster without ever having studied grandmasters, only at a higher level.
Moravec's paradox: what humans do instantly and unconsciously (recognising an image) is hard to program, because we cannot see our own steps; what we do slowly and consciously (arithmetic) is easy, as any pocket calculator finding a square root shows. Steven Pinker (mid-1990s) summed up 35 years of AI as "the hard problems are easy and the easy problems are hard." DeepMind amended it twice. AlphaGo showed that an apparently impossible target, intuition, could be programmed: unconscious knowledge is more programmable than it seems. AlphaZero showed that a game humans play consciously, and had hand-coded into Stockfish, was far from settled: conscious knowledge is less solid than it seems. Hassabis's moral: known knowns can turn out to be unknown, and to understand what we understand, we need autonomous machine intelligence.
What did AlphaZero prove?
The harder question was what AlphaZero meant for the road to artificial general intelligence (AGI). Two camps read it differently.
Silver and DeepMind. Three complex games mastered from scratch, with no human data or instruction. The old obstacle (no deductive system can classify the world) had been bypassed, and the problem of induction (finding patterns in an infinity of data) beaten. "We're no longer constrained to systems with predefined rules." RL agents looked ready for messy real-world problems. In 2018 Silver added that a system which learns for itself has "no ceiling."
LeCun and Sutskever. Yann LeCun (Facebook and NYU) called RL the "cherry on the cake," the cake being deep learning; DeepMind answered with a slide of a cake buried under a lattice of cherries. Sutskever at OpenAI sided with LeCun, as he had in his 2015 AlphaGo debate with Silver: just as excited about the coming revolution, but convinced that new kinds of neural networks, not RL, would change everything.
Mallaby's resolution is that both were right. AlphaZero was a real demonstration of autonomous, agentic learning, and at the same time a product of progress in network design.
The telephone problem and ResNet
- The conundrum. In theory more layers meant a smarter network; in practice too many layers made it go haywire. Mallaby compares it to the party game of telephone: the longer the chain, the more garbled the message. The technical name is vanishing gradients: the training signal fades as it passes back through many layers.
- First fix. Geoffrey Hinton's 2006 deep-belief-network paper let networks grow from about three layers to ten or so.
- Second fix. At the end of 2015 Microsoft researchers introduced the residual neural network (ResNet). Mallaby describes two ingredients: layers pass on the residual (the gap between what they received and what they output), like an editor sending notes on fixes instead of rewritten pages; and skip connections, express lanes that let signals jump over intermediate layers and reach distant ones intact.
- Result. Adding layers stopped being self-defeating; learning spread evenly through networks of 150 layers or so.
| AlphaGo (2016) | AlphaZero (2017) | |
|---|---|---|
| Network depth | About 12 layers | About 40 layers (residual) |
| Architecture | Two nets: policy (move choice) and value (position evaluation) | One network doing both jobs |
| Training data | Human games, then self-play | Self-play only |
| Games | Go | Go, chess, shogi |
The parallel with Atari is exact, in Mallaby's account. Both systems showed agentic trial and error delivering versatility across many games, and both owed their RL gains to a deep-learning advance: Atari (2013) to the convolutional neural net built for image recognition, AlphaZero to its updated form, the residual net from Microsoft.
Why Hassabis cornered RL
At the end of 2017, deciding which half of the triumph to stress was, for Mallaby, perhaps the biggest strategic dilemma in AI research.
- Silver's framing. His Alberta PhD training plus the Atari and AlphaGo successes made it natural to present AlphaZero as an RL breakthrough. He had persuaded his former supervisor Rich Sutton to join DeepMind; Sutton and a few RL experts opened a DeepMind office in Edmonton, cementing the lead in agents that learn from experience.
- Hassabis's framing. His neuroscience PhD made him a "broad church" thinker: AGI would integrate perception (convolutional nets), memory (long- and short-term) and search (machine planning). So he staffed many specialties, bet on several approaches and funded whatever showed promise. He was not wedded to RL as the one path, but RL was producing the best results.
- The strategist's logic. The more Facebook and OpenAI pressed him, the more Hassabis wanted to hold a key square on the board: dominate RL and DeepMind would lead as long as the road to AGI ran through it.
- Rivals copied but could not catch up. Despite the cherry jibe, Facebook built a Go system using AlphaGo's techniques; OpenAI built game-playing RL agents, prodded by Elon Musk, still fixated on Hassabis's lead. Open publication (labs let scientists publish because a strong paper trail attracted talent) made imitation easy. But neither rival believed in or excelled at RL as Silver did, and Silver's prestige plus Google's money drew the best RL researchers to DeepMind.
Mallaby judges the calculation logical but names its hidden risk: a surprise breakthrough in another part of AI could make RL matter less, for long enough to let a rival overtake DeepMind. The rest of the chapter describes exactly that breakthrough.
Sutskever's wheelhouse: sequences
Mallaby borrows Peter Thiel's remark about Hassabis: geniuses are seldom brilliant in general, but brilliantly suited to one mission. Hassabis's was founding an AGI company; Silver's, pushing RL; Vlad Mnih's, fusing RL with deep learning. Sutskever's was the arrival in June 2017 of the transformer, a moment his whole career, PhD included, had prepared him for.
- The problem it solves. Much information (speech, text, video, stock charts) only makes sense in sequence. In "The dog ran outside because it saw a car," "it" is opaque without the first half; episode three of a crime drama is opaque without episodes one and two.
- Spatial versus temporal. Convolutional nets exploit spatial dependencies (neighbouring pixels hold related clues, such as a cat's eye). Transformers handle temporal dependencies: how each phrase or plot twist relates to the others over time.
- Recurrent nets (RNNs). Sutskever's thesis topic (not to be confused with Microsoft's residual nets). An RNN reads words one at a time, remembering earlier ones. Invented in the 1980s, the basic version achieved little: like a patient with severe short-term memory loss, it missed links even a few words apart, such as a subject and a later verb.
- His contribution. Around 2012, while lab-mate Alex Krizhevsky was coaxing performance from a convolutional net in Hinton's lab, Sutskever did the same for RNNs: GPU chips, extra layers, algorithmic tweaks that improved language modelling. He never got an ImageNet-scale breakthrough. Finishing in 2013, he was to sequences what Silver was to Go: the leading authority on a problem not yet fully cracked.
Seq2Seq and its toolkit
In 2014 Sutskever and two Google co-authors, Oriol Vinyals and Quoc V. Le, published the sequence-to-sequence framework. Where basic deep learning maps a static input to an output, Seq2Seq maps one time-ordered sequence onto another, like a professional interpreter who hears a whole sentence, digests it and renders the equivalent sentence in another language.
- Word embedding (an established method). A map of English with hundreds of dimensions, impossible for humans to picture but easy for computers. "Computer" sits near "keyboard" in one dimension, "electricity" in another, "semiconductor" in a third, much as one person can be an astrophysicist, baker, daughter, skier and Chicagoan at once.
- Tokens. The units actually mapped: a word, syllable, punctuation mark or other fragment, whatever the system learns from best. Splitting "unbelievable" into "un," "believ," "able" teaches prefixes and suffixes, so the model handles compounds it never saw; knowing "bio" means life helps with biology, biography, biome.
- Vectors. Each token becomes a set of mathematical coordinates on the map, carrying all its associations and connotations: a social-network graph for words, recording relatives, friends and friends of friends.
- Context vector. To choose which connotations matter here, Seq2Seq compressed the input sentence into one dense summary and used it to guide each output word.
- Attention (Montreal, 2015). Replaces the single summary with many weighted summaries, each tailored to the next output step (see below).
At the NIPS conference at the end of 2014, Seq2Seq underpinned Sutskever's "success is guaranteed" talk. He saw it as a breakthrough for any temporal problem: translation, then summarising (turn a long passage into rich vectors, then write a précis), then conversation (parse a question, output an answer). Outputs would start clunky, but bigger networks, more data and a few algorithmic flashes would let machines master sequences as they had mastered vision.
Attention
The flash came in May 2015 from three University of Montreal researchers led by Yoshua Bengio. The weakness they fixed: humans keep only the essentials of a conversation, while Seq2Seq could not tell essentials from noise, so squeezing even one sentence into a fixed context vector failed for long sentences.
- The mechanism. Instead of one fixed-length summary, compose several, each catching a different facet; as each output step is generated, use a summary tailored to that step, carrying only the relevant input.
- Example. To translate a verb into French, attention points to the English verb, its subject, its object and anything else that helps pick the right French verb, even if it appears later in the passage.
- Weighting. Clues are ranked: the subject may count for more than an adverb.
- The TV-drama analogy. At one moment in episode three you need a character's backstory; at another, the earthquake in episode one or the murder in episode two.
Mallaby calls this a superpower: by tuning out the irrelevant, a model can find the patterns that unlock meaning in an infinity of data.
OpenAI, RL fatigue and the sentiment neuron
At NIPS at the end of 2015 Sutskever announced he was leaving Google (three years after it bought the Hinton-Sutskever-Krizhevsky start-up) to join Musk and Altman's anti-Google, anti-DeepMind venture as research director. Google tried hard to keep him.
- His reason. At Google he could see his life ten years ahead: good science, dull career. And "you're supposed to start a company in Silicon Valley!" Mallaby contrasts this with Hassabis, who put science ahead of business when Google came to buy DeepMind.
- Early OpenAI. A gaggle of recruits in an apartment in San Francisco's Mission District. Sutskever worked on whatever came up: RL agents for games and the five-fingered robot hand that solved the Rubik's Cube, projects pursued partly "to make noise" and keep the company alive.
- The lesson he drew. Getting agentic systems to do anything consequential was brutally hard; RL was "an endless hill of suffering." He kept side projects on sequence learning going throughout.
An OpenAI language model trained on 82 million Amazon reviews to learn fluent text. The method marked a shift: frontier labs had begun leaving supervised learning (map inputs to human-labelled correct outputs: labelled cat photos, curated English/French pairs) for self-supervised learning on unlabelled data. The model hides a token, predicts it, and adjusts its weights and biases when wrong; no human annotation needed. The surprise: among thousands of neurons, one single neuron fired or stayed dormant according to whether the reviewer was positive or negative. Intelligence usually emerges from many interacting neurons; here sentiment was cleanly isolated, and nobody had asked for it or labelled any review. Sutskever's conclusion: predict the next token and you could potentially solve intelligence, and "discover everything that needs to be discovered."
The transformer
Two months later, on June 12, 2017, eight Google researchers published "Attention Is All You Need." It bundled embeddings, tokenisation, vectors and attention, then threw out the recurrent network that had started the field. Every earlier system, Seq2Seq and the sentiment model included, assumed sequences must be read step by step. The Google team saw that with attention this was obsolete: if the model knows which tokens to attend to for each output, word order can be processed all at once. Sequential learning, paradoxically, could be done non-sequentially.
Mallaby's philosophical gloss is Henri Bergson, the early-1900s French thinker so idolised that fans stole hair from his barber. Bergson rejected time as a simple march forward: people constantly revisit yesterday and anticipate tomorrow, so experience is governed by where attention goes, not by the clock. The transformer is the computer-science equivalent.
Two breakthroughs followed:
- Context. RNNs struggled beyond one or two sentences; transformers could attend to key words scattered across paragraphs or pages. A translator could use clues far before or after the token; a chatbot could connect a question to something said minutes earlier.
- Speed. Step-by-step processing was inherently slow and could not use the thousands of parallel operations of the GPUs (and Google's TPUs) AI had run on since around 2010. Taking in all tokens at once meant models finally exploited their hardware.
"It was perfect," Sutskever said. He cites Richard Hamming's 1986 Bell Labs talk "You and Your Research," which he reread in grad school: a prepared mind, having turned a problem over from every angle, recognises the answer instantly when it appears.
Radford builds GPT
The day the paper appeared Sutskever leapt from his chair to find Alec Radford, a young, inventive engineer and his key deep-learning partner, and told him to drop everything for the transformer. Radford was taken aback at first, but Sutskever insisted until he did it.
OpenAI. Musk had told the co-founders that PhDs "don't do anything"; hire people who do things. Sutskever, a PhD who believed in scaling, saw AGI as a megaproject where engineering must be respected as never before, against academia's disdain for the dirty work. The motto: "Don't be too clever." The tools are already potent, success is guaranteed, just build; this, he says, is what Google and DeepMind lacked.
DeepMind. Hassabis and colleagues dismissed OpenAI's work as brute force without intelligence. Mallaby agrees Sutskever's description of academic snobbery fitted DeepMind: it published papers in Nature, OpenAI published blog posts.
Radford replicated the Google design (attention, parallel processing, no recurrence) and added three departures of his own:
- Purpose. Google's transformer was built for translation; his targeted a broad range of language tasks, including conversation.
- Training. Google trained on human-curated sentence pairs (supervised). Radford trained mainly self-supervised on raw text, predicting the next word as the sentiment model had, then applied a short supervised "finishing gloss" with human labels once the model already performed well.
- Name. The generative pre-trained transformer, GPT: a mouthful only an engineer could love, and not obviously a consumer hit.
GPT was announced in June 2018, a year after the transformer paper. By later standards it was crude: weak at linking ideas a few paragraphs apart, prone to regurgitating training text, and given to confident inaccuracy. By contemporary standards it advanced the field: passably fluent, conversant across more topics than rivals, despite learning mostly without labels. Sutskever saw hints of understanding of history, science and geography (a war novel might teach the horror of conflict; a sports report the thrill of competition), and OpenAI claimed "significant world knowledge."
Critics later dismissed GPTs as a "stochastic parrot." Sutskever first protested that it was not statistics, then corrected himself: "It is statistics! But what are statistics?" His reply to critics who called later GPTs mere next-token predictors: to compress the statistics of text you must understand what in the world produces them, and what produces human text is people's thoughts, feelings and ideas. All of that can be deduced through next-token prediction; a machine that knows how humans choose the next word has grasped the workings of the mind.
Two kinds of superhuman
Silver's agents had made people ask whether humans know what they think they know and whether intuition reduces to ones and zeros; Sutskever's suggested that language, often treated as mere symbols, might unlock human thought. After GPT and especially GPT-2 (February 2019), there was also a strategic question for DeepMind.
- Autonomy, neutralised. Silver prized RL because an agent learns for itself and so can learn anything. Self-supervised learning also needs no human labels.
- Data, neutralised. RL let an agent generate its own data where data was scarce (self-play in Go). Self-supervision fed on unstructured text, almost limitless; the internet is a crystallisation of human knowledge on nearly every subject, so self-play became superfluous, at least for the moment.
- Superhumanity, redefined. In the 2015 debate Silver had argued that human data can never lift a machine beyond humans. True for Go. But GPT offered a different superhuman: a model that absorbed the whole internet might never find a Move 37, yet would be superhuman in breadth.
| AlphaZero path | GPT path | |
|---|---|---|
| Learns from | Self-play experience | Raw internet text, unlabelled |
| Human labels | None | None (self-supervised) |
| Data supply | Self-generated | Almost limitless |
| Kind of superhuman | Depth: new truths in one domain (Move 37) | Breadth: matches humans almost everywhere |
| Ambition | Expert agent extending to adjacent games | Jack-of-all-trades across knowledge |
Mallaby puts the dilemma on two levels. Algorithmically: stress the deep network at a model's heart or the RL layered on top? In AlphaZero the two were evenly balanced (residual net plus pure self-play); six months later GPT tipped the scale, showing a transformer could learn autonomously from unlabelled data with no agentic trial and error. Strategically: build a domain expert that surpasses humans, or a generalist that matches them across the board?
His closing judgment: in a counterfactual history, the generalist vision should have gripped a lab whose mission was general intelligence. But just as Sutskever's training prepared him to build language models, Hassabis and Silver had been primed to underestimate them.
- David Silver. DeepMind's RL lead; conceived AlphaGo Zero, delivered AlphaZero; recruited Sutton.
- Demis Hassabis. Broad-church neuroscientist turned strategist who chose to corner RL.
- Murray Campbell. Deep Blue engineer who, with Hassabis, wondered in 2016 if chess was solved.
- Rich Sutton. Silver's Alberta supervisor; opened DeepMind's Edmonton office.
- Yann LeCun. Facebook/NYU; RL is the "cherry on the cake."
- Ilya Sutskever. RNN specialist, Seq2Seq co-author, OpenAI research director from end-2015.
- Oriol Vinyals, Quoc V. Le. Google co-authors of Seq2Seq (2014).
- Yoshua Bengio. Led the Montreal group behind attention.
- Alec Radford. OpenAI engineer who built GPT.
The ResNet paper (Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, December 2015) had four authors, not a trio, and Mallaby's account of "residuals" as forwarded notes that cut computation is loose: each block learns a correction added to its input via a short skip connection, which is what keeps gradients alive. AlphaGo Zero (October 2017) already used the single dual-headed residual network; its large version had 40 residual blocks (about 80 layers). The attention paper (Bahdanau, Cho, Bengio) first appeared in September 2014; May 2015 was its conference presentation. Atari (2013) to AlphaZero (2017) is four years, not the "three" Mallaby writes. "Stochastic parrot" was coined by Bender, Gebru and colleagues in 2021, long after GPT-1. Critics also noted AlphaZero's 2017 Stockfish match used restrictive settings; the fuller 2018 Science rerun still showed AlphaZero clearly stronger.
The fork Mallaby describes closed. From OpenAI's o1 (September 2024) and DeepSeek-R1 (January 2025), frontier labs used large-scale reinforcement learning on top of pretrained transformers to train reasoning: the two paths in the table above are now one recipe (see Chapters 19-20). Rich Sutton shared the 2024 Turing Award (announced March 2025) with Andrew Barto for founding RL, and in April 2025 Silver and Sutton argued that AI is entering an "era of experience" beyond human data.
Silver himself left Google DeepMind (reported January 2026) to found Ineffable Intelligence in London, which launched in April 2026 with a reported $1.1 billion seed round, a European record. Its bet restates his 2015 argument: systems that learn from their own experience, not LLMs alone, are the route to superhuman intelligence.
On Language and Nature
Over nearly three years of pub conversations, Hassabis admitted to Mallaby that he misjudged language models, for two reasons: he thought text could not ground meaning in the physical world, and he thought human experience far too vast for the internet to capture. Meanwhile DeepMind poured effort into reinforcement learning in simulated worlds: the nature simulation Gaia and AlphaStar for StarCraft II. Both projects exposed the limits of RL in rich environments just as OpenAI was proving that more varied data made language models better, and DeepMind failed to pivot in time.
The pub sessions and the Wikipedia test
For almost three years Mallaby met Hassabis at a pub near his home in leafy North London: an always-empty upstairs room, an octagonal table under a faded chandelier, cappuccinos, a carafe of water, two hours per session against Mallaby's detailed list of topics. The period coincided with peak excitement over large language models, so language came up again and again, and Hassabis conceded that he had been slow where Sutskever had leapt out of his chair.
His long-standing thought experiment: read all of Wikipedia and you would know a great deal, but would you know that a dropped glass smashes, or how heavy it feels? (He lifted a tumbler to make the point.)
- "Action in perception." A branch of neuroscience holding that you cannot perceive the world properly, in a deep sense, unless you act in it. Weight is the classic case: he can anticipate how the glass will feel only because he has picked things up.
- The business-plan line. DeepMind's founding plan had dismissed as "mistaken yet highly influential" the idea that language is intelligence expressed. For Hassabis language was a system of symbols, insufficient by itself.
- The corollary. To understand the world, a machine must experience it, in a robot body or in a game-like simulation. His love of games fused with a belief in game-playing AI.
The two misjudgments
| What Hassabis expected | What he now thinks | |
|---|---|---|
| 1. Grounding | A 1990s database says "a dog has legs," but how does the word "legs" map onto the pixels of a real dog without interaction with the world? | Language is "more inherently grounded than we thought." The internet holds so many descriptions of dogs' legs that grounding can emerge, like the 2017 sentiment neuron, helped by fine-tuning on human feedback: models "learn from us how to be grounded." |
| 2. Scope of experience | The number of possible human behaviours is semi-infinite, perhaps 10^50 bits | It is nearer 10^13: about 14 trillion words online seem to capture most human behavioural possibilities |
On the second point Mallaby allows that the internet underrepresents minority languages and cultures, but sees the force of it. Asked whether humans are less original than they thought, Hassabis said there is simply less variety, and reached for "nothing new under the sun," wondering if it was Solomon. It is Ecclesiastes, traditionally attributed to Solomon: a fragment of his churchgoing childhood, and an unusual line for a Silicon Valley cheerleader.
"Unreasonably effective"
Hassabis's account of the surprise runs in three steps:
- The architecture. Someone had to invent transformers, big enough to swallow the internet. Ingesting a few trillion tokens taught them nearly all of human experience.
- The counterfactual. It could have gone the other way: download 14 trillion words, get something pathetic, and conclude that understanding civilisation was many orders of magnitude away. That is what he expected. Hence his phrase: language models are unreasonably effective.
- The fossil-fuel analogy. The internet is to AI what coal and oil were to the Industrial Revolution. Neither had to exist: dinosaurs and ancient forests might simply have rotted, leaving engines useless without a leap to nuclear or solar, which would have been very hard two hundred years ago. Humanity built the internet for messages, information sharing and e-commerce, then woke to find it was oil. Once you see oil, you drill. (He hinted that the convenience of both coincidences might itself deserve a separate conversation.)
But early GPTs behaved as he had predicted: "slightly poor memorizers" giving half-OK, ungrounded answers.
Hassabis was not alone: Marc'Aurelio Ranzato at Facebook, a former Sutskever lab-mate, also found OpenAI's experiments interesting rather than groundbreaking. And Hassabis's scientific taste was usually exquisite, spotting trends ahead of rivals. Here it failed for three reasons: skepticism about ungrounded symbols, his love of games, and a belief that human experience was grander than all the text online. The price came when ChatGPT became a global sensation at the end of 2022 and DeepMind stopped being seen as the world's top lab. Hassabis's own line: you can't be Nostradamus every time; the models surprised everyone, "or maybe everyone except Ilya Sutskever."
A lone voice: Jack Rae
After GPT-2 (February 2019), young DeepMind scientist Jack Rae circulated an admiring deep dive. Parameters (the adjustable weights and biases that store what a model learns) had jumped from the first GPT's figure (Mallaby says 175 million) to 1.5 billion, with performance rising in step. Rae's conclusion: scaling up was obviously the thing to do, and DeepMind was "dropping the ball." He was in a small minority. DeepMind largely shrugged, stayed focused on RL, ran agents in simulations of the real world, and backed Silver's drive to extend AlphaZero.
Gaia: an AI raised by nature
Gaia was DeepMind's most ambitious simulation, named after the Greek earth-mother goddess and the 1970s idea that Earth's living systems act as one symbiotic organism. Its champion was computational ecologist Drew Purves, whose slogan was that nature and AI need each other: nature needs AI to fight global warming; AI needs nature as a place to learn, since human intelligence evolved there.
- The link to DeepMind's plan. The business plan named a key challenge: how conceptual knowledge is acquired from perceptual information. Purves argued this could only develop in a simulation of the natural world.
- Why existing environments fell short. Atari and Go are wholly abstract. Minecraft, the main exception, is blocky and modelled on cities, not nature. Hassabis agreed: chess is "just symbols," Minecraft is all cubes; he wanted somewhere that looked like the world really is, so he backed Gaia.
Man-made world. Orderly, regular and scaled to humans: door heights, flat floors, right angles. A designer sets a chair's contours and a factory makes thousands of identical copies. Objects fall into ready-made sets (houses, vehicles, offices, tables), so an AI can classify most of them easily.
Natural world. Irregular: few right angles, straight lines or flat surfaces; scales from redwoods to insects; every tree, glacier and sunset unique. No obvious set structure, so an agent must group things by its goal: apple trees and cows are both food, but only cows give leather for a tent. Intelligence here means inventing and discarding concepts continuously.
Gaia's season in the sun
For a year or more Gaia was a darling. The plan: a digital natural environment with agents set tasks such as collecting fruit. Mallaby notes it was hard to build and speculative in purpose, since the link between nature and the evolution of intelligence may be historical rather than causal; but it suited a company built on multidisciplinary curiosity.
- NIPS, end of 2016. By now a "Woodstock of AI"; Purves, in a fine cardigan waistcoat, presented DeepMind's ideas on an "intelligent biosphere."
- The King's Cross plan. Hassabis wanted a wall-sized screen in the new headquarters as a window onto Gaia, marking the divide between biological and artificial intelligence, with live monitoring of hundreds of agents, "almost like a civilization experiment." A colleague suggested to Mallaby that it echoed Theme Park, the game Hassabis co-created as a teenager.
Emergence and a theory of everything
Hassabis linked Gaia to John Conway's Game of Life (another 1970s idea): a grid where simple preset rules make patterns that seem to come alive, which fascinated him as a teenager at Bullfrog. His underlying obsession is emergence:
- Examples. Chemicals combine into life; neurons into intelligence; humans into an economy; gas molecules into temperature. If he took a sabbatical he would think about this for ages.
- Why it troubles science. Science works by reduction, breaking complexity into components. But some phenomena exist only when the parts combine, which strains the normal method; he still sees no clear account of where emergent properties come from.
- The link to AGI. Asked whether hidden rules might explain how nature emerges from plants, animals and soil, he said finding them is "the main reason why I'm building AGI." He nodded along as Mallaby laid out the chain: humans cannot see the rules; an infinity machine might; AlphaZero proved AI sees deeper than we do; Gaia tested whether AI could decode nature's emergent properties ("Exactly").
- The ultimate goal. A strong feeling that the universe runs on underlying rules, an information system: a theory of everything, which he has always been obsessed with finding.
Mallaby then turned to language. Spoken language is messy, full of grammatical errors; linguists and philosophers try to "wrestle it to the ground" with grammar and logic and fail. Might transformers have found deep patterns beneath the mess? Hassabis called that a reasonable way to think about it, crediting Sutskever. The irony Mallaby draws is that Hassabis was lucid about emergence everywhere in nature, yet failed to see that grounding and concepts might emerge as a by-product of mastering language.
Failure as data
Gaia advanced science by failing. RL agents managed a simplified nature, but genuine irregularity and unclassifiable shapes defeated them; in truly vast environments they could not induce the patterns.
- The contrast with language. For OpenAI's models, more complex and varied data meant better performance, thanks to transfer learning across loosely related topics, as knowing French helps with Italian.
- The warning. Gaia suggested RL does not work that way: a richer environment does not reliably yield a better agent. So the scaling path for language looked open, RL's murky.
- Hassabis's framing. Scientists don't keep a tally of failures and successes; there are only projects, and new evidence changes direction.
AlphaStar: StarCraft II
From early 2018 Silver turned to StarCraft II, a battle game whose top players have millions of followers.
- Why it is hard. Players manage an industrial base strategically while tactically controlling dozens of unit types; the map is too big to see at once, so they decide without knowing what the enemy is doing; there are no turns, only continuous play. Mallaby likens it to life itself: a stream of long- and short-term choices in a fog of imperfect information.
- Silver's ambition. Many DeepMinders thought it insoluble, as baffling as Gaia (fuzzy knowledge on irregular timescales instead of fuzzy objects). For Silver that was the point. In autumn 2018 he called his team "flag bearers of a new way to go about AI," aiming at work that would matter in ten, twenty, thirty years.
- The team. Silver paired with Oriol Vinyals, a highly cited deep-learning expert and Seq2Seq co-author who joined DeepMind in 2016; the group peaked at more than forty researchers.
- The design. The AlphaGo template: first learn from expert human games, then self-play. The novelty was a league of five agents, each tuned to a distinct style, competing so that each exposed the others' weak spots, like swordsmen sparring: machines learning from machines with no human help.
- 10%: the early agent's win rate against the game's built-in bots.
- 84%: after Vinyals introduced a transformer to decide which workers, fighters, buildings, vehicles, minerals and clouds of green energy deserved attention: a deep-learning gain.
- 99.8%: the October 2019 Nature result Mallaby reports against officially ranked humans, after RL and the five-agent league: an RL gain.
- 10^26: estimated possible actions at any moment, versus 361 points on a Go board.
- 0: the pure self-play, AlphaZero-style version never got off the ground.
Hassabis hailed the result as a step toward systems that accelerate scientific discovery; AlphaStar could juggle many facets of the game at once, which human experts struggle to do. Mallaby's reading is a caveat, not a denial: a decent share of the success came from "Hintonite" deep learning, and the final climb from 84% to 99.8% was RL's.
The failed zero version
The real test was a side project: an AlphaZero-style agent learning only from self-play. Success would have vindicated Silver's core creed, that learning from experience has no upper limit. Despite Gaia, his faith was intense: learning from one's own experience beats learning from humans'.
Mallaby casts Silver as the AI era's John Dewey, the progressive educator who argued in 1938 for a "necessary relation" between actual experience and education. Like the progressives, Silver rejected learning from secondhand text and prized the machine equivalent of their virtues: curiosity, individualism, independent learning, adaptability. The zero agent failed for a mundane reason: with ~10^26 options, blind trial and error took an eternity to find a winning sequence, so reward signals were rare and learning slow. Silver liked to say a human dropped into a complex world learns by trying things, but humans arrive with inherited instincts (DNA), childhood teaching and reading, which even Dewey conceded. Expecting the agent to do without them underrated nature, nurture and books: an ironic error for a lab steeped in neuroscience.
Hassabis's later view: StarCraft could probably have been cracked the zero way, but it was harder than expected, which shows why learning everything from scratch is not always wise. RL purists want to prove RL best; his goal is AGI as safely and quickly as possible, useful to the world. Some day, on an infinitely powerful computer AGI itself invents, an AlphaZero version of AGI may be possible; for now, pure self-play is "an unnecessary handicap."
The missed pivot
Simulations and RL would have been fine without rivals, but Hassabis's hoped-for singleton (one lab alone at the frontier) had long gone. In a race, one path risks missing the shortcut that decides it.
| Year | DeepMind | OpenAI |
|---|---|---|
| 2017 | Gaia and other simulated environments | Transformer published; sentiment neuron |
| 2018 | StarCraft work begins | First GPT (June) |
| Early 2019 | AlphaStar's first victories announced | GPT-2 (February), much more powerful |
| Through autumn 2019 | StarCraft focus continues; Nature paper (October) | Sprinting ahead with $1 billion from new backer Microsoft |
Mallaby's explanation of the failure to pivot:
- Pride. DeepMind was too excited by its own research and, used to being the top lab, could not imagine a copycat overtaking it.
- Temperament. Hassabis had always beaten his own path: obsessive childhood chess, underage work for Peter Molyneux, impatience with Cambridge's AI skepticism, an un-British appetite for entrepreneurship, the leap from game design to neuroscience. Mallaby rates him more genuinely contrarian than most of Silicon Valley's self-described contrarians.
- The people he would be following. OpenAI's founders were Elon Musk, a formidable space-and-cars tycoon but, in Mallaby's words, "clownish, jealous, and vainglorious," and Sam Altman, a silver-tongued networker and investor defined more by general ambition than devotion to AI (in 2017 he considered running for governor of California). Hassabis's first instinct was not to take cues from them.
Language, the transformer and GPT made him pay for that instinct. But Mallaby closes with a turn: the same drive to invent a novel direction would soon pay off spectacularly far from language (the protein work of Chapter 15), after a turbulent stretch involving Google, armies of lawyers and Mustafa Suleyman (Chapter 14).
- Demis Hassabis. Admits two misjudgments about language; backed Gaia; calls pure self-play a handicap.
- Jack Rae. DeepMind scientist who in 2019 said scaling was the obvious move.
- Marc'Aurelio Ranzato. Facebook researcher who also underrated early GPTs.
- Drew Purves. Computational ecologist behind Gaia.
- John Conway. Mathematician whose Game of Life shaped Hassabis's interest in emergence.
- David Silver. Led AlphaStar; "the Dewey of the AI era."
- Oriol Vinyals. Co-led AlphaStar and brought in the transformer.
- GPT-1's size. The first GPT had about 117 million parameters, not the 175 million Mallaby gives (175 billion was GPT-3).
- AlphaStar's 99.8%. In the Nature paper this is a rank: AlphaStar reached Grandmaster level, above 99.8% of officially ranked players on Battle.net, not a 99.8% share of games won. Its league was also more structured than "five agents" suggests, with main agents plus dedicated "exploiter" agents.
- Fair play. Critics argued AlphaStar's early showings (it beat pros TLO and MaNa in matches revealed in January 2019) benefited from superhuman click precision and a whole-map view; DeepMind later imposed camera and action-rate limits.
- Hassabis's arithmetic. Comparing 10^50 "bits" of behaviour with 14 trillion words mixes units, and 14 trillion is closer to the size of a curated training corpus than of the whole internet. The point (text captured more than expected) survives; the numbers are rhetorical.
- Solomon. Ecclesiastes is only traditionally attributed to Solomon; most scholars date it centuries later.
DeepMind did build large language models: Jack Rae led Gopher (280 billion parameters, December 2021), and Chinchilla (March 2022) reset the field's scaling rules; Rae later worked on Gemini before Meta hired him for its new superintelligence lab in June 2025. Simulated worlds returned once language models could ground them: Genie 3 (August 2025) generates interactive 3D environments from text, and SIMA 2 (November 2025) uses Gemini to act inside games. With Gemini 3 (November 2025) topping many leaderboards, DeepMind largely regained the frontier status Mallaby says it lost in 2022.
Project Mario
From autumn 2015 to 2019 Hassabis and Suleyman tried to win DeepMind semi-independence from Google, with a board on which outsiders could stop powerful AI from falling under the sole control of shareholders. Code-named Project Mario, the talks consumed three years, nearly produced a $15 billion research spin-out, and ended with nothing, while OpenAI fought a mirror-image governance war and Suleyman left DeepMind after a bullying investigation. Mallaby's lesson is bleak: if safety oversight could not be negotiated inside one unusually open, profitable company, its prospects across rival labs and nations are poor, and both founders concluded that personal influence beats governance structures.
A second experiment, under ideal conditions
In autumn 2015, as Streams was being readied for the NHS, Mustafa Suleyman began a second grand experiment: with Hassabis, a long negotiation to make sure powerful AI, when it came, would not answer only to Google's shareholders. Mallaby treats it as a live case study: what happens when a few leaders, in unusually favourable conditions, try to build a control structure for a new technology.
- The trigger. The SpaceX safety meeting (Chapter 10) achieved nothing, and once Musk launched OpenAI as an anti-Google, anti-Hassabis venture he could no longer sit on an oversight body for DeepMind.
- Suleyman's goal. A post-capitalist form of governance to reconcile three imperatives that AI set against each other: profit, existential risk and social justice. His passion was certain; as in health, the obstacles were formidable.
Mallaby lists why conditions were, on paper, as good as they would ever be:
- Founders who cared. Safety was in DeepMind's DNA: Hassabis first bonded with Shane Legg at Legg's 2009 Halloween lecture, and had since backed Suleyman's efforts (adding vivid talk of retreating to a bunker).
- An experimental parent. Google had run governance experiments of its own, notably the founders' super-voting shares, justified as a way to defend the motto Don't Be Evil.
- Concentration. DeepMind was the world's top lab, and its strongest rival, Jeff Dean's group in Mountain View (home of the future transformer inventors), was also inside Google. Solve governance internally and you largely solved it, full stop.
Project Mario: the Alphabet route
The first replacement for SpaceX-style oversight arrived unbidden. In 2015 Google reorganised under a holding company, Alphabet, spinning specialist units out as semi-independent "bets." Shortly before the SpaceX meeting, Google's M&A chief Don Harrison told the founders DeepMind could regain independence this way, under a 3-3-3 board: three DeepMind people, three from Alphabet, three independents. DeepMind, fond of secret code names, called the talks Project Mario.
| Party | Why the Alphabet spin-out appealed |
|---|---|
| Larry Page (operational) | Google was growing unwieldy; an ad money-gusher and a pre-revenue moon shot were hard to run under one roof |
| Google (financial) | Moving cash-burning units off the mothership's books would raise its profits and share price |
| Hassabis and Suleyman | A strong say over the deployment of AGI, credible independent directors, and a plan whose commercial logic made it likely to happen |
2016: talks, revenue plans and a secret hedge fund
Talks began in the first half of 2016. Hassabis met Page four times on details, and the founders planned revenue to sustain independence.
- Health. Part of the impetus for DeepMind Health: after some years of pro bono work, Suleyman expected a lucrative share of the savings it generated for hospitals.
- Trading. Hassabis quietly assembled about 20 researchers to train high-frequency trading algorithms and explored a tie-up with BlackRock. Google disapproved; he hoped he had found another game to win.
A source told Mallaby the aim was to beat Jim Simons's Renaissance Technologies, whose secrecy Hassabis loved. Mallaby sees the Manhattan Project echo: Rentec gathered scientific geniuses on a remote campus (Long Island, not Los Alamos), and its longtime leader Peter Brown, a deep-learning pioneer trained under Geoffrey Hinton, was as driven as Hassabis, slept even less and kept a fold-down bed in his office. Outcome: the DeepMind team made no money and, because of Google's wariness, was quietly disbanded.
In summer 2016 a fifth Page meeting produced a formal term sheet for the spin-out. A few months later Hassabis met the new Google CEO, Sundar Pichai (Page had moved up to run Alphabet): an engineer with a Wharton MBA, ex-management consultant, boyish grin, affable and conflict-averse. The talks were cordial and bland; the DeepMinders concluded he would not rock the boat. They were wrong.
Two weeks in November 2016 then went wrong:
- Mid-November. Cordial, bland meeting with Pichai.
- November 21. Chief legal officer David Drummond in London: on safety and governance "everyone is in agreement," but a spin-out raises "concerns"; he sketches a complex in-between formula. The founders are confused: their safety guarantees depended on the spin-out and its 3-3-3 board.
- November 25. Pichai, by phone, shows a steelier side: "bets" are for moon shots unrelated to the core business (autonomous cars, life extension); AI will be strategic to search and cloud, so it does not belong in that bucket.
The founders remained unsure whether Pichai was shutting the door, or even had the authority to, given Page's apparently contrary view; and even Drummond, the bad cop, had professed support for safety. They judged that more pushing could get them there.
Plan B: the global interest company
In 2013 they had pushed Google by entertaining a Facebook bid. At the end of 2016 they built another walk-away option: raise $5 billion in pledges from outside investors and leave if Google refused the governance they wanted.
- Scale. Enough to run DeepMind for more than five years. OpenAI's touted $1 billion of pledges at its launch was, Mallaby says, "smoke and mirrors." The pitch to safety-minded investors: $5 billion could put AGI somewhere secure under credible governance.
- The team. Suleyman assembled imaginative lawyers, a top communications strategist and a prominent investment banker.
- The legal form. A company limited by guarantee, the structure British nonprofits use: no shares for backers, no dividends, obligations owed to the principles of its charter. They called it a global interest company: run with capitalist intensity, post-capitalist in impact.
The skeptics among the advisers. Alphabet could simply say "back in your boxes, boys, we own you." Staff were Google employees bound by noncompete clauses, non-solicit rules and IP ownership terms; moving a hundred people out one day and founding a company the next would be legally messy.
The optimists. A public-interest lawyer on the team would argue that the British public had an interest in a safer, independent DeepMind; surely Google valued its reputation too much to fight that in court. And even a risky threat had value as leverage.
The result was a two-pronged plan: talk freely to billionaires (no harm in deepening one's network among top capitalists), but never threaten Google outright ("We never ever said to Google, unless you do this, we will leave"). The art, one adviser said, was to make Google take the talks seriously; to its credit Google never called the bluff, which is what made the episode so unusual. The founders were dancing with a parent that said neither yes nor no.
Asilomar, January 2017
In the first days of January 2017 the pair attended a conference on the rules for AI at the Asilomar hotel on California's Monterey Peninsula, the venue of the famous 1970s conference that set ground rules for genetic research; Lee Sedol's defeat by AlphaGo had made a sequel feel urgent. Besides speaking on safety, they pitched their breakaway to Reid Hoffman, still friendly despite his role in launching OpenAI (a "good billionaire," in Suleyman's estimate).
- Hoffman's vantage. He saw the same big-tech tension between OpenAI and Microsoft: Musk had been furious when Microsoft tried to turn its OpenAI partnership into a PR talking point, refusing to let OpenAI look like Microsoft's marketing tool.
- His governance sympathies. He had backed OpenAI because it was a nonprofit with a charter requiring its technology to serve society, a format partly inspired by DeepMind's SpaceX attempt to put a nonprofit board over a for-profit. Sam Altman had gone further, floating a board elected by wide swaths of the world after reading James Madison's notes on the Constitutional Convention. Hoffman thought AI needed nonprofit oversight with broad democratic buy-in, especially as politicians were slow and Donald Trump, just elected, had anti-regulation instincts Hoffman found anathema.
- The pledge. With a net worth of $3.8 billion after selling LinkedIn to Microsoft, Hoffman committed $1 billion, more than a quarter of his wealth and 100 times his OpenAI pledge of just over a year earlier. His reasons: the most impactful technology of his lifetime should not entrench a monopoly, and he expected 90% of his wealth to go to philanthropy anyway. He told neither Altman nor his venture firm Greylock; Mallaby's aside: billionaires answer to nobody.
Some advisers urged seizing the offer: with a famous anchor investor, other capital would follow; litigation would be manageable; independence would bring agility and the chance to pay staff in DeepMind stock. Hassabis saw the case but feared a drawn-out legal fight that would eat his energy; the Alphabet route was far cleaner.
- Suleyman's lobbying. He sought out Kent Walker, Alphabet's top lawyer and policy strategist (and a SpaceX-meeting attendee), introduced him to Angela Kane, a senior UN official on weapons of mass destruction and his idea of an ideal independent director, said he had sounded out Barack Obama, mentioned Al Gore, and hinted that many people, some very rich, wanted DeepMind's technology under a robust governance committee.
- Page's position. Hassabis asked Page, also at Asilomar, what had changed. Page said he still supported the spin-out as logical, but it needed Pichai's buy-in. Hassabis left the conference early for Mountain View, sick of back-and-forth and lawyers.
Good cop, bad cop, absent founders
- January 9, 2017. Hassabis met Pichai at Google HQ (Suleyman dialled in from Asilomar). Conciliatory, he offered to design the spin-out around every Google misgiving, proposed Pichai, Page and Eric Schmidt as Alphabet's three directors with Kane and other eminent figures as independents, and said the months of talks were pulling him from science. Pichai sounded open to everything, with details to settle with Drummond.
- The next day. A rerun of November: Drummond said they had misunderstood, and Pichai was entirely against Alphabetization. A DeepMind document records that the founders offered "every single mechanism" to meet Google's concerns; Drummond did not move.
- The email. Days later they wrote to Pichai that the process had dragged on "more than a year" and was hurting their ability to manage, offered to skip Davos to come and settle it, and asked that Page and Brin attend too: they were tired of the good-cop, bad-cop seesaw.
- The showdown. DeepMind argued its mission was AGI, not consumer-internet products; Pichai replied that AI was central to Google and he would not let his scientific bench be depleted. Hassabis hoped Page would settle it, but Page arrived two hours late and Brin later still. Mallaby's jab: their version of what was later called "founder mode" was absence, which undercuts the Valley creed that founders deserve indefinite control. Pichai was now the man to deal with.
Pichai's split
The following week Pichai, wanting to keep Google's AI lead and knowing that alienating AI leaders was a poor way to do it, invited Suleyman alone to dinner at his home, served vegetarian curry, and (perhaps hoping to drive a wedge between the founders) proposed splitting DeepMind:
- Research would spin out under Hassabis to chase AGI ("who knew if that would work," Pichai remarked, a little dismissively).
- Applied, building useful algorithms in health care, would fold into Google, and Suleyman would run all of Google's applied AI from California.
Through spring 2017 the plan made grinding progress. It had real appeal: Hassabis would lead a semi-independent AGI lab; Suleyman would deploy practical AI through Google's global reach. But every few weeks both made the eleven-hour flight to San Francisco for another interminable round of push and push-back. Mallaby draws the cost: no wonder Suleyman seemed distracted with his London lieutenants, and no wonder Hassabis was less alert to the transformer that summer than he might have been.
Aviemore and the red lines, June 2017
In the first week of June 2017 nearly all of DeepMind's 500 staff flew on two chartered jets to Aviemore, a resort near Balmoral, chosen because the company had outgrown conference centres near London (the planner's options: Scotland or private islands, and islands seemed "a bit much"). The banquet hall was dressed as Narnia's enchanted forest; the founders led a go-karting outing, each as competitive as the other. On stage Suleyman surveyed Applied's targets: health and now climate, via new hire Jim Gao, a Google engineer whose AI had cut data-centre electricity use and who planned RL-driven intelligent buildings. His climax was a slide titled "DeepMind: A Global Interest Company," since Google had seemed ready to sign some version of the Pichai plan. Staff, who had heard months of spin-out rumours and speculated about stock, saw two boxes: Alphabet/Google, with Suleyman and Applied inside the mothership (a spin-in), and DeepMind, an independent research company linked to Google only by a dotted-line licensing agreement (a spin-out). Heads spun.
Ten days later Google sent back its position: a document covered in red lines. Pichai was nowhere near approving what had just been announced to the whole company, and the founders faced walking it back. The same week, Sutskever leapt from his chair over the transformer paper.
- Escalation. Hassabis cancelled his next call with Pichai to signal anger; Pichai pinged at once, and when Hassabis eventually took the call he vented his disappointment. Four days later Suleyman emailed Drummond and cancelled another meeting.
- The impasse. Relations hit bottom. Google saw too much commercial potential to let AI slip from its control; DeepMind saw too much existential risk to let commercial priorities decide AI's deployment. Each needed the other, so a fractious dialogue went on.
OpenAI's mirror-image war
DeepMind. A for-profit subsidiary trying to wrap nonprofit governance around powerful AI.
OpenAI. A nonprofit realising it needed for-profit machinery to raise far more capital. Both looked for a capitalist/post-capitalist hybrid; both found that restructuring talks turn quickly into quarrels.
By summer 2017 OpenAI's leaders were discussing grafting on a for-profit arm. About a month in, the day-to-day leaders, Ilya Sutskever and Greg Brockman, fell out with the fundraisers and visionaries, Musk and Altman. Altman wanted to be CEO and was manoeuvring Musk aside, though he had recruited him.
- Whispers. Altman told Brockman that Musk was too erratic to be trusted with AGI; Brockman told Sutskever; Sutskever came to fear that both men wanted absolute control.
- Poaching. Musk hired away one of OpenAI's key scientists to run Tesla's AI division.
- The ultimatum, September 20, 2017. Sutskever, writing for both, called it the highest-stakes conversation they had been part of, and potentially the world's. To Musk: the structure gave him a path to "unilateral absolute control over the AGI"; he said he did not want control but had shown in the talks that it mattered enormously to him; since he feared a Hassabis AGI dictatorship, he should not build a structure that let him become one. To Altman: why did the CEO title matter so much, why had his reasons shifted, and how did AGI connect to his political goals (he was flirting with a California governor run)? They asked all four to meet that day.
- Musk's reply. "Guys, I've had enough": no more funding until they committed to stay, or he was a fool financing their start-up.
- The cave. Two days later Sutskever and Brockman backed down; the for-profit idea was shelved and OpenAI stayed a Musk-dominated nonprofit. Altman sided with Musk ("I remain enthusiastic about the non-profit structure!"), hid his part in stirring the revolt, and told Musk's lieutenant Shivon Zilis the pair's protest had been "childish."
The truce could not last, because OpenAI still needed money. Altman explored three fixes, two of them identical to DeepMind's: money from Reid Hoffman, conversion to a public-interest corporation, and, off-trail, funding via a cryptocurrency.
Jan 31, 2018 Musk sends Brockman, Altman and Sutskever a chart showing DeepMind and Google Brain produce the lion's share of AI research: OpenAI is "on a path of certain failure relative to Google."
Same day Brockman: papers are a poor yardstick; the moral high ground is OpenAI's biggest tool, its fiduciary duty is to humanity, and "it doesn't matter who wins if everyone dies."
Next day, 3:52 a.m. Musk proposes OpenAI spin into Tesla: help with self-driving first, then fund AGI from car profits; Tesla is the only path to "hold a candle to Google," with small but non-zero odds.
Early 2018 The offer echoes Musk's 2014 Skype pitch (from a closet in LA) for Tesla or SpaceX to absorb DeepMind. Altman uses it as proof of Musk's power hunger and easily wins over Brockman and Sutskever; the three reject Tesla.
All-hands On the top floor of OpenAI's converted truck factory, Musk says he is quitting, that OpenAI must sprint to stay relevant and that AGI is likelier at a strong business like Tesla. An intern asks whether speed is reckless and whether a for-profit Tesla is any different from Google; Musk calls him a "jackass" and storms out.
The deal that almost was, 2018
At the start of 2018 DeepMind's fight seemed to resolve. A thick deck for the Alphabet board argued that "an unprecedented technology requires an unprecedented structure." It quoted rival tech leaders on AI's power, plus Vladimir Putin (whoever leads in AI "will be the ruler of the world") and Xi Jinping, and warned of a gathering techlash, citing a New York Times columnist who wrote that technology had "crossed over to the dark side." The two-part message: let DeepMind sprint for AGI, and build governance robust enough to survive public skepticism.
- The term sheet. Within weeks the sides agreed a fleshed-out Pichai plan: Suleyman would run Applied inside Google; Hassabis would run Research as an independent global interest company, with $15 billion of Google funding over a decade.
- Both satisfied. Suleyman got most of what he wanted. Hassabis got the money and an end to structure meetings he found screamingly dull: after two years he had hit his limit ("I don't want this part of my brain to grow").
Then it shattered. In April 2018 Apple poached John Giannandrea, the senior Google executive who had backed Suleyman's move to Mountain View; in the reshuffle Jeff Dean was promoted and the slot Suleyman expected vanished. As after Aviemore, he had to un-promise: he had already told deputies to prepare to move to California.
Suleyman loses his footing; Hassabis stays loyal
Some lieutenants date Suleyman's decline here. After the health-data uproar, Aviemore and the "Double Red P0 Plus Plus" chaos (Chapter 11), the balls came crashing down; one colleague, metaphors tumbling, said he ended up wearing no clothes, "up the cloud in a banana," and that when he tried to stay put Hassabis said no way. Mallaby says the true story is subtler and more revealing:
- Loyalty. Hassabis valued long friendships: DeepMind employed many people from his past, back to Cambridge and Elixir, partly from a deep wish to do right by comrades.
- Hiring philosophy. He recruited the most dazzling scientists, but distrusted managerial stars from outside: in a scientific culture non-scientists had to be humble, so he relied on internal comrades, and Suleyman was the most capable of them.
- Doubling down. The pair revived the walk-away, asking Hong Kong tech mogul Joe Tsai to match Hoffman's $1 billion (he politely declined); returned to Pichai's research spin-out, with Hassabis urging Suleyman to close it; agreed a pact not to fight in executive-committee meetings (colleagues could not get a word in when they went at each other); and kept their Sunday-evening pub ritual, mint tea not alcohol, food ordered just before the kitchen shut.
November 2018: oversight in reverse
Google insisted on absorbing DeepMind's health team (more than 100 people) into its own health division: part of Pichai's ambition to pull DeepMind talent into the mothership, but without the rest of the bargain. Suleyman lost a large part of his empire while staying at DeepMind; Hassabis still ran Research but had no guaranteed $15 billion and no independent board. Worse, Google shut the Independent Review Panel that had watched over DeepMind Health, ending Suleyman's experiment in post-capitalist transparency after less than three years.
Mallaby argues, uncomfortably, that Pichai had good reason. The panel's incentives had proved distorted:
- The June 2018 report. By then DeepMind had bulletproofed its data-sharing contracts, patient data was known to be shielded from Google, and several lifesaving diagnostic algorithms were on the way. Yet the panel, keen to show independence, said it was hardly surprising the public would question a company "so closely linked to Google."
- Mallaby's judgment. The panel bowed to a technophobic zeitgeist; bolder overseers would have answered the public's questions, not just noted them, by explaining that DeepMind was balancing privacy with progress in health care.
- Brin's lesson. At a later Alphabet board meeting Sergey Brin told Suleyman this was predictable: give outsiders a platform and they will use it to burnish their own careers and reputations, not to serve Google's projects, however virtuous. Suleyman knew Brin was right, and that after this backfire Google would never accept a grander safety board.
2019: the investigation and the exit
At the start of 2019 a handful of employees alleged that Suleyman reduced subordinates to tears with capricious, bullying behaviour: harsh language, intimidating text messages, a climate of fear. There was no claim of violence or sexual harassment; Mallaby contrasts Hassabis's old mentor Peter Molyneux, who had hurled a projectile at a subordinate and smashed a tank of piranhas.
What weighed on Hassabis
- Uncertainty. He abhorred bullying but could not tell whether Suleyman was egregiously tough or staff too sensitive.
- Loyalty. Suleyman was his younger brother George's friend, his poker companion, the kid his parents had fed, housed and sometimes employed; perhaps even the voice tying Hassabis to the social-justice values of his North London upbringing.
- Control. Hassabis said the worst thing is to control someone, and took no pleasure in domination; but he equally refused to be dominated, and Suleyman kept crossing his red lines by challenging his direction of DeepMind, which was Hassabis's creation and identity.
- Failing projects. Both of Suleyman's grand experiments, AI for health and AI governance, were failing: the health team lost, the panel closed, the board talks endless; these time sinks drew negative publicity onto an otherwise pristine reputation.
- Convenience. Whatever the merits, a faction wanted Suleyman out; Mallaby wonders whether that suited Hassabis.
Hassabis decided he wanted to focus on science and was tired of Suleyman's machinations, but his dislike of confrontation, and his self-image as someone who does not control others, made him act indirectly. Instead of the frank parting conversation that is routine as start-ups mature, he let the COO and chief counsel take charge; an outside lawyer was hired and an investigation opened; as an old friend, Hassabis was told to recuse himself.
After three months the lawyer's report, about 25 pages, found Suleyman's management style amounted to misconduct: some complainants had been humiliated before peers; others said he told them to contact him only via non-Google channels. (Many colleagues later called his behaviour standard for a mission-driven founder; Mallaby still calls the charge sheet serious.) At a disputed review meeting Suleyman understood the choice as: accept the complaints and keep his reputation, take a voluntary sabbatical, work with a coach, and return later in a non-managerial role, perhaps as an ambassador; or dispute them, face a formal procedure, likely be found guilty, fired and lose compensation. Given a couple of hours, he paced Coal Drops Yard in King's Cross and, instead of calling a lawyer (he later wished he had), phoned Marilyn, his girlfriend from the days when George Hassabis was his best friend. Around noon he accepted; days later an all-staff email said that after a decade he was taking time to recharge. Almost nobody knew of the investigation. On August 21, 2019 communications director Ruth Barnett, on a French beach, got a call from Bloomberg: a story within the hour would say Suleyman had been "placed on leave." A hasty call of lieutenants could not decide whether to fight it: not quite true (a sabbatical), not quite untrue (an investigation, little choice); defending him risked looking dishonest if details leaked, abandoning him risked harm too. No one briefed against him; there simply was no plan. Suleyman, in a Mountain View meeting coordinating the health handover, saw the headline, put his face in his hands and walked out. He believed it could not have happened without Hassabis's approval: either DeepMind planted it or failed to deny it. Mallaby: the first suspicion was false, the second accurate.
- Aftermath. Suleyman threw himself into coaching, learning to lead by encouragement rather than pressure, but felt he could not return after the humiliation, and with health gone little of Applied remained.
- End of products. In summer 2019 Jim Gao left with his climate team to found a start-up. DeepMind might still supply applications to Google, but would no longer try to market its own products.
- A prince without a court. At the end of 2019 Google's leaders judged his conduct a grey zone between tough and unacceptable and, perhaps to placate him and avoid a public fight, made him a vice president in California, finally, but not allowed to manage anyone.
The lesson for AI oversight
Mallaby's tally of failed oversight experiments:
| Experiment | What happened | Why it failed |
|---|---|---|
| SpaceX safety meeting (2015) | Achieved nothing | Participants' skewed incentives; Musk became a rival |
| DeepMind Health review panel (2016-18) | Shut by Google, Nov 2018 | Panelists served their own independence and reputations |
| 3-3-3 DeepMind board (2015-18) | Never created | Google foresaw the same suspect incentives among independents, and too much commercial value at stake |
| Google's ATEAC (2019) | Disbanded | Activist attacks on Kay Coles James (Heritage Foundation, doubts on trans rights) drove other members out |
| OpenAI nonprofit board (2018-23) | Lost a 2023 showdown | Fired Altman; he rallied financiers and three directors were ousted |
The core argument: if safety mechanisms could not be agreed inside one company, one more open to governance experiments than most because of its extreme profitability and unconventional founders, what hope is there for common safeguards across many labs in many countries? Mallaby can imagine no happier counterfactual. On the advisory council he blames a public square dominated by activists out to crush opponents rather than debate; Google's disbanding of it was understandable. And with government regulation so unlikely, Hoffman had been right in 2017: it was worth risking his fortune on corporate-governance experiments precisely because governments would not act.
From idealists to realists
By 2024-25 both men had new jobs. Hassabis was CEO of Google DeepMind, having absorbed Google's Mountain View AI researchers and related teams. Suleyman had quit Google after two years, co-founded an AI start-up with Hoffman (Inflection AI), then become CEO of Microsoft AI, with teams in Seattle, Silicon Valley, London and Zurich: two North Londoners with immigrant parents running AI at two American giants.
Now rivals with bitter memories, they gave the same verdict: the negotiations were futile, bound to fail, and harmful for the energy and goodwill they consumed. Having lost faith in governance mechanisms, both found salvation, paradoxically, in personal power: their safety agenda became securing influence inside their companies.
- Trustless structures are unrealistic. They wanted safety that did not depend on trusting Google, which is very hard in practice.
- Boards fail at the crunch. Even with a governance board, it probably would not do the right thing when it mattered.
- Bright lines land in the wrong place. A safety charter drawn years ahead of an unknown technology will put its lines in the wrong places.
- Negotiating distrust destroys trust. Bargaining over a trustless structure signals you cannot trust the other side.
- Lean in instead. Redirect the energy into earning real trust inside Google: you get a seat at the table when safety questions arise, and you rack up shared successes.
- Grow up. Things are not black and white with a technology of unknown consequences; move "from idealist to realist," ideally keeping your values.
Both men compromised their original values: the sale to Google came with a promise that DeepMind's technology would never serve weapons or surveillance, yet by 2025 Google, like Microsoft, was eager to supply the national-security complex. But Hassabis is right that the ideals were unrealistic: AI was never going to be a singleton effort, and once many labs in many countries raced, deployment in civilian and military settings could not be resisted. A well-meaning individual at the table is flimsy reassurance, but perhaps the best available.
The account rests on DeepMind documents and the founders' and advisers' recollections; Google's side (Pichai, Drummond) appears mainly through them, and Mallaby's reading of Pichai's curry dinner as a wedge is speculation. His scorn for the review panel's "technophobic" caution sits awkwardly with the 2017 UK Information Commissioner finding that the Royal Free broke data-protection law in sharing records with DeepMind. The exit story is told largely from Suleyman's vantage, and "standard for a mission-driven founder" is a contested defence of conduct an investigation called misconduct. The OpenAI emails come from documents made public in Musk's litigation; the unnamed scientist Musk poached for Tesla in 2017 was Andrej Karpathy. Google's ATEAC lasted only about a week (late March to early April 2019).
In February 2025 Google dropped the pledge in its AI principles not to pursue weapons or surveillance applications. On October 28, 2025 OpenAI completed its recapitalisation: the nonprofit, renamed the OpenAI Foundation, controls a for-profit OpenAI Group PBC and holds about 26% of it, with Microsoft at about 27%, a hybrid of exactly the kind both labs sought in 2017. In May 2026 a federal jury rejected Musk's suit accusing OpenAI of abandoning its nonprofit mission, finding his claims were brought too late.
Fermat for Biology
Protein folding was the "Fermat's Last Theorem" of biology that Hassabis had carried since Cambridge, and within hours of AlphaGo's win he made it DeepMind's next target. The route to AlphaFold ran through at least seven pivots: Foldit to CASP, recurrent to convolutional to transformer networks, reinforcement learning to deep learning fed by evolutionary biology, contact maps to distograms, and a doubtful leader to John Jumper. Mallaby's verdict is that AlphaFold refutes the charge that commercial AI labs are scientifically narrow: DeepMind won because it refused a monoculture, then gave the results to science for free.
The target: why protein folding
When. Seoul, March 2016, the day AlphaGo clinched the Lee Sedol match. Walking with David Silver (a documentary crew recording), Hassabis said DeepMind could now definitely solve protein folding, where before he had only thought it could. Silver's later dry gloss: Hassabis does not linger over a big win before choosing the next one.
- The motive is old. AI as an engine for science had been Hassabis's aim since undergraduate days. He chose Cambridge partly because of Life Story, the film about the Nobel-winning DNA discoverers, and went looking for a Nobel-scale problem; folding was the best he found.
- What proteins do. They supply both structure and function for organs, muscles, hormones, hair, blood and brain cells.
- Why shapes matter. A known structure lets chemists design drug molecules that bind to its surface; knowing how folds form could help with Parkinson's and Alzheimer's, both tied to misfolded proteins; some researchers talk of engineering self-assembling protein machines.
- Anfinsen's conjecture (1972). In his Nobel lecture Christian Anfinsen proposed that a chain of amino acids (20 common types, picture irregular beads on a thread) carries its own folding instructions, a self-executing origami. Result: a decades-long computational race to read structure from sequence.
- The experimental route. X-ray crystallography: crystallise the protein (hard in itself), bombard it with X-rays from a stadium-sized synchrotron, infer the shape from the diffraction pattern. Accurate, but months or years of a doctoral student's time per structure.
- The computational route. Worse: an average chain has roughly 10^300 possible conformations, 10^130 times the number of possible Go positions.
- How the idea lodged. Possibly one pub conversation with a Cambridge friend over table football; Hassabis says ideas then incubate subconsciously until they surface "fully formed."
Clue two: Foldit
About a dozen years after Cambridge, during his MIT postdoc, Hassabis met Foldit, an online game from future Nobel laureate David Baker's lab at the University of Washington. Thousands of non-scientists folded virtual chains and were scored on physical and chemical rules:
- Charges. Put positively and negatively charged acids next to each other (they attract, like magnets).
- Grease. Bury water-avoiding acids in the interior, away from moisture.
- Packing. Prefer compact structures without internal gaps (the suitcase principle).
- Others. A few further conditions folded into the same score.
Why it hooked him. Foldit joined his two passions, competitive gaming and scientific discovery, and amateurs with only spatial sense reached shapes near real proteins. By Seoul he read it as an RL problem of DeepMind's favourite kind: clear objective, precise score as feedback, a virtual world for unlimited trial and error. Having surpassed a Go champion's spatial intuition, the next agent would intuit protein structure.
Mallaby's thesis: an answer to the critics
Hassabis went on to show Anfinsen broadly right and to share a Nobel Prize for it. Mallaby uses the episode as a rebuttal to the backlash against for-profit labs.
The critics. Commercial labs build chatbots that absorb copyrighted material, simulate emotion falsely, echo the internet's worst biases and threaten millions of jobs; and they are scientifically narrow, obsessively scaling transformers instead of trying new ideas.
Mallaby. Some of this is "baffling": labs keep adding capabilities (multimodality, longer memory, step-by-step reasoning) and have damped model bias until it compares well with human bias. On folding the whole charge sheet fails: Hassabis chose this problem and turned to language models only under competitive pressure; DeepMind won because it rejected monoculture (a victory for the willingness to pivot, not for one kind of AI); and it gave the results to science free.
From Foldit to CASP: the first pivots
Origin by accident. While Hassabis was in Seoul, DeepMind's Applied division ran a two-day hackathon for 25 engineers; one team happened to build a crude Foldit agent. Team member Marek Barwinski's logic: anything shaped like a game is fair game for RL.
- Sponsors. Mustafa Suleyman, hoping folding would bolster DeepMind's health work, told the team to keep going and alerted Hassabis, who joined in quickly.
- Staffing. In May 2016 a top AlphaGo engineer came over and scientist Andrew Senior became project lead.
| Pivot | From | To | Why |
|---|---|---|---|
| 1 | Foldit scores | CASP contest | Foldit's rules only roughly describe nature; a top score often missed the real shape |
| 2 | Recurrent network | Convolutional network | The chain crumples, so far-apart acids become neighbours; CNNs also had a mature toolkit |
| 3 | Reinforcement learning | Deep learning | No self-play against nature and no exact definition of a "good" structure |
| 4 | Contact maps | Distograms | Graded distances carry far richer clues than touch/no-touch |
| 5 | CNN plus search | Direct folding | The network was already guessing the whole shape |
| 6 | Senior as leader | Jumper as leader | Hassabis: a leader who thinks it impossible cannot crack it |
| 7 | Convolutional net | Transformer (AlphaFold 2) | BERT-style masking taught evolutionary, physical and geometric structure |
- Name. Critical Assessment of Structure Prediction, held every two years.
- Secret answers. Organisers collect crystal structures that are solved but unpublished.
- The test. Several dozen computational teams receive only the sequences and have three or four months to submit predicted shapes.
- The score. The Global Distance Test (GDT), 0 to 100: how closely predicted atom positions match the crystallographers' ground truth. About 90 GDT is accuracy competitive with experiment.
- Division of labour. Silver advised on adapting the RL component; Senior, a specialist in sequential data, built the supporting network and naturally reached for a recurrent network, the standard tool for sequences.
- Reality check. Run on a recent CASP's sequences, the system trailed its rivals, especially Baker's lab. Hence pivot two: the RNN went, replaced by a convolutional network, by then the default architecture in many fields. With a search space 10^130 times Go's, progress stayed slow.
John Jumper and the tortoise strategy
Hire. October 2017, recruited by Senior: John Jumper, 32, with experience across mathematics, physics, chemistry, biology and machine learning and one obsession, protein motion. Mallaby pairs him with Vlad Mnih, who merged Toronto-style deep learning with Alberta-style RL to make the Atari agent: Jumper likewise combined two schools he had worked in.
D. E. Shaw Research (three years): molecular dynamics. A Newtonian programme: list every force (charge attraction and repulsion, water-avoiding versus water-seeking atoms, rigid or springy bonds resisting certain angles), layer on quantum and statistical mechanics, assign each atom its mass, compute every particle's motion and combine them on a custom supercomputer. Flaw: the same as Foldit's. Newton suffices for baseballs and planets, but nobody knows precisely how each atom in a protein behaves, so the risk was "ever finer simulations based on the wrong model."
University of Chicago PhD: machine learning. No supercomputer, so software. Start from the human model of the forces, predict a structure from a sequence, compare with the crystal structure, and let the learner correct the human assumptions when wrong. Data: just over 100,000 solved structures out of hundreds of millions of natural proteins, the folding equivalent of 100,000 labelled cat photos. Results "half good": decent on small proteins, behind Baker on large ones ("not a practical system, but it was fun"). Hidden advantage: it could learn past human knowledge.
- The heresy. Many academics reject black-box models and want explainable science. Jumper's counter: a working opaque model beats a transparent failure because folding is about medicine and saving lives, and biology may be too messy and emergent for compact equations.
- Hassabis's matching view. Since Cambridge he had held that only AI could expose life's hidden patterns; deriving biology from the axioms of physics was a dead end, like early AI's attempt to build minds from logic rules alone.
- The bet. If biology is too complex to reveal itself to human minds, back Jumper's tortoise.
DeepMind's own flawed premise
Jumper found a second slow tortoise at DeepMind: a team of about six that had over-learned the AlphaGo template. Shaw overrated physics rules; DeepMind overrated Go. The analogy was natural while the problem looked like Foldit, but CASP changed the problem.
- No self-play. The plan had been an agent practising on Foldit indefinitely to compensate for scarce data. Against real proteins that was gone. Hassabis: folding is not a two-player game; you are playing against nature.
- No objective. Jumper's Rubik's Cube analogy: a solved cube is self-evidently solved, but no one can look at a protein structure and know it is right. Colleagues spoke of using RL to "smash the optimization problem" without a precise definition of what to optimise; rough rules (grease avoids water) are not an objective.
- The consequence. If you cannot define good, let nature define it: train a network to map sequences onto structures, as deep nets map images to words or speech to text. Silver himself agreed RL did not fit the problem.
Borrowing from biology
Jumper's PhD had hit two walls, a vast search space and sparse data. The fix was to add protein science to DeepMind's computational strength. Soon after his arrival the team (still under Senior) adopted UniProt, a databank of nearly every annotated amino acid sequence. Sequences are cheap to read chemically, with no crystallography, so UniProt held the 20,000 human proteins plus millions from plants, animals, fungi and bacteria.
- Kinship groups. Sequences descend from common ancestors, so a human blood protein has fairly similar cousins in mice and birds.
- Conservation. Acids found in almost every member of a family probably matter to the structure; otherwise evolution would not have preserved them.
- Co-evolution. Acids that mutate in pairs probably meet in the fold (an electrical bond, or a puzzle-piece fit).
- Templates. One crystallised structure acts as a template for every sequence in its family, so each crystallography result explains many sequences.
What gave DeepMind the edge. Academic labs also used UniProt, sometimes with off-the-shelf CNNs, to predict contact maps (which acids touch), which shrinks the space of possible folds. DeepMind, with stronger deep-learning instincts and more powerful hardware, asked instead for the distance between every pair of acids, a graded number rather than a yes/no. Barwinski likened the move from contact map to distogram to switching from black-and-white to colour television.
- Learning the rules. The three data sources trained a further convolutional network until its weights encoded rules of folding.
- Applying them. A specialised search algorithm, a cousin of AlphaGo's tree search, began from an estimated shape and iteratively tested nearby shapes against those rules: very hard to build, and a DeepMind specialty.
- The name. AlphaFold was partly branding, implying a smooth line from AlphaGo to AlphaZero to AlphaStar; after so many pivots to deep learning some thought DeepFold more honest. The search stage, "semi-RL" in Jumper's phrase, was the last trace of the RL starting point.
CASP13 and Hassabis's logical unreasonableness
Entry. DeepMind entered CASP13 in spring 2018, badly prepared:
- Silver's warning. About a week before the start, an uncharacteristically stern Silver told the team it could not yet churn out enough predictions in the time window and needed tighter organisation.
- The false alarm. A researcher trying to calm nerves built a model of DeepMind's likely ranking; it said twentieth, until a colleague found a statistical error in it.
The contest. It opened in May with 98 teams. DeepMind ran two tracks at once: submitting predictions, and testing upgrades that had been dropped in the rush. Accuracy plateaued just under 60 GDT (nearly 60% of main atoms correctly placed): good against rivals, far from the 90 GDT of crystallography.
- The trigger. Hassabis read the team's six-month plan, which (Jumper concedes) amounted to more of the same, and turned up unannounced: are we going to solve this or not? Smart people could be redeployed elsewhere.
- Senior's case. Full success was unrealistic, the shelved ideas were going nowhere, and AlphaFold might win CASP anyway: declare victory and wrap up.
- Hassabis's case. He wanted to solve the problem, not top a league table. He granted Senior was reasonable and he himself "probably" not, but argued great results need some unreasonableness, applied logically: nobody could prove success impossible, just as he could not prove it certain.
- The test. From his games career he knew the cost of a death march with no end in sight, so he used his standard device: a brainstorm with metrics set aside, where he listens for the fluidity of ideas (plausible, not necessarily right, but flowing). High fluency means continue.
- Outcome. With Jumper present, imaginative, obsessed and expert in physics and biology, continuing was almost certain: a couple of his bold ideas might crack it.
Direct folding and Cancun
The anomaly. The convolutional network sometimes knew the shape before the search module found it, solving the whole puzzle unasked. Mallaby notes that an AI exceeding its mandate would be alarming in a setting with lives at stake; in a lab chasing medical progress it was an opening. Jumper's group built a new network to compute the exact position of every atom directly, rather than output folding rules.
- The cliff. GDT first crashed from about 60 to about 20. The team accepted the drop (a colleague's image: jump off a cliff, then crawl back up). By late November 2018 it was back near 60 and gaining every week.
- Cancun, early December 2018. The original AlphaFold beat the other 97 teams, chiefly thanks to distograms. In free modelling (no known evolutionary template), the hardest category, it was most accurate on 25 of 43 targets; the best rival led on 3. One attendee wrote that people who had bet careers on making crystallographers obsolete now feared obsolescence themselves; "What just happened?" became the meeting's refrain. Mallaby's answer: Hassabis's student conviction held, since biology needed more than biological intelligence.
- Unspoken edge. Jumper already knew the direct-folding system would soon beat the version being applauded, and kept quiet.
- The call from London. An urgent video call brought a strategy change, not congratulations: on the strength of the brainstorm, direct folding and the win, Hassabis would double the team, go all out, and make Jumper leader, moving Senior aside. His rule: you cannot crack a hard problem if its leader thinks it impossible. About 14 months after joining, Jumper had his decade-long obsession, ample resources and an equally committed boss.
AlphaFold 2
Explore, then exploit. At the start of 2019 Jumper opened a period of exploration: every researcher brought one slide with a blue-sky idea for moving GDT from about 60 to 90. Roughly three months of rolling hackathon followed (fast, unpolished prototypes, whiteboard argument), with Jumper backing some experiments and killing others, before effort swung to exploiting the winners.
The big rethink. Search had already given way to direct prediction; now the convolutional network itself was replaced by a transformer.
- Inspiration. In October 2018 Google, belatedly profiting from an architecture invented in its own building, released BERT, a transformer that learned from huge unlabelled corpora. Jumper talked the idea through with Oriol Vinyals, who had brought transformers to StarCraft II. UniProt's sequences were biology's texts. Others had tried versions of this; the question was whether DeepMind could do it better.
- Masking. Hide some amino acids in a family of sequences and ask the network to fill them in; to do that well it must absorb evolution, physics and geometry, perhaps structure itself. Mallaby hears Ilya Sutskever here: a narrow guessing task yields broad understanding.
- Attention. As a language model links a phrase to a distant word, the protein transformer linked acids far apart on the chain that meet in the fold, after ingesting the whole of UniProt.
- Architecture. AlphaFold 2 worked by mid-2019. Its core was a family of specialised, very complex transformers, one nicknamed the "tetraformer" for combining four variants of the standard design.
- Self-distillation. Its highest-confidence predictions went back into the training set: success compounding success, GDT rising steadily.
Demis-driven development
- The obsessive boss. Project manager Clemens Meyer joked that Hassabis, woken at night, could recite the current GDT instantly; in fact night was his working time, and Jumper shifted his sleep to match, trading ideas in the small hours (once telling him it was 3:45 a.m. and he had to sleep).
- The device. Meyer and Jumper turned that attention into a method they called "Demis-driven development": schedule a Hassabis review for Tuesday, push researchers to land upgrades by Monday.
- The ratchet. Propose +5 GDT for the next period and he would call it "too safe" and add five more.
Summer 2018 Plateau just under 60; Hassabis's challenge.
Autumn 2018 Direct folding: crash to about 20, back to about 60 by end of November.
Dec 2018 Original AlphaFold wins CASP13 in Cancun.
Dec 2019 GDT 84; Meyer plays hits from 1984 at weekly meetings (Tina Turner, Frankie Goes to Hollywood).
Jan 2020 GDT 86; songs from 1986 (Madonna's "Papa Don't Preach").
Mar 2020 COVID lockdown; meetings go online (one scientist works from a laptop on her ironing board); the DJ routine continues.
Apr 2020 GDT 90, crystallography-level accuracy; Meyer plays MC Hammer's "U Can't Touch This" (1990).
May-Aug 2020 CASP14: the remote team receives 90 sequences over three months, submits, then waits four more months.
Nov 2020 Final score 92.4.
CASP14: the test that could not be cheated
The suspicion. In November 2020 CASP founder John Moult, running his fourteenth contest, saw a 92.4 score, more than 50% above any earlier best, and suspected the secret structures had leaked into DeepMind's training data.
The check. With German experimentalist Andrei Lupas he designed a test that no leak could defeat: Lupas sent the sequence of a protein whose structure his own X-ray work had never completed, so no answer existed anywhere. AlphaFold's prediction agreed with the parts Lupas had established and filled in the rest consistently with his partial data. Mallaby: short of time travel, cheating was impossible.
The announcement. On November 30, 2020 CASP declared the result; one computational biologist called it a "seismic and unprecedented shift" that upended the field. CASP had met its ultimate goal of making itself redundant, and Moult said he had hoped to live long enough to see it.
Three kinds of change
1. Practical discovery
- 20,000: all proteins of the human proteome, catalogued straight after CASP confirmed accuracy; 83% had never been mapped by crystallographers. Much of the computing ran over Christmas (Hassabis enjoyed eating lunch while the AI worked).
- 350,000: structures by summer 2021, from yeast to fruit flies.
- ~200 million: proteins folded by July 2022.
- 3 million+: researchers worldwide who had freely used the predictions by late 2025, through a database built with the European Bioinformatics Institute in Cambridge and searchable like Google.
- ~20: the DeepMind team at its peak, against hundreds of academics working for decades.
Applications Mallaby lists. Candidate proteins to digest ocean plastic; disease-resistant crops needing fewer pesticides; cheaper, greener searches for new detergents; and the structures of enzymes that give superbugs antibiotic resistance, which had resisted years of effort, mapped in minutes. Uses ran from fundamental biology to vaccine development and environmental science.
2. The scientific establishment
The effect here was ambiguous. There was already worry that science was slowing and ideas shrinking: researchers must master so much that their best work now comes in their late forties, and collaborations sprawl until some papers carry thousands of co-authors. AlphaFold sharpened the unease. Big pharma had let an outsider take its ground, and roughly twenty people had beaten hundreds of specialists. One academic stressed this was not Go, a niche with few researchers and no uses beyond itself.
3. The standing of AI
To Hassabis's camp the same facts meant a golden age of AI-driven discovery, a "reinvention of invention" that might pack a century of progress into a decade:
- AlphaDev (June 2023): found faster algorithms for basic routines such as sorting numbers.
- Materials (a few months later): a DeepMind system proposed recipes for millions of new materials.
- AlphaGeometry: performed at the level of International Mathematical Olympiad gold medallists.
- GenCast: beat the state of the art in weather forecasting.
- AlphaFold 3 (May 2024): predicts interactions between proteins and other molecules, not only protein shapes.
Hassabis's contrast. Science is where AI does unequivocal good; language models have obvious misuses, and most of their advertised benefits are still "cheese tomorrow." AlphaFold is the clearest payoff so far. His next goal: genuine breakthrough medicines in one year rather than ten, cheap enough for the neglected diseases of poor countries, because visible benefits would change a conversation he finds too negative. Mallaby's closing turn: the conversation was already changing, not as Hassabis expected (Chapter 16, ChatGPT's world).
AlphaFold succeeded because the team kept abandoning its own favourite tools, including the RL that made DeepMind famous, borrowed domain knowledge from biology, and was finally led by someone who believed the problem soluble. The lesson cuts against the brand: the "Alpha" name implied a continuity with AlphaGo that the method largely lacked.
- Architecture names. The AlphaFold 2 paper (Jumper et al., Nature, July 2021) calls its core the Evoformer, which updates a multiple-sequence-alignment representation and a pairwise representation, followed by a structure module using invariant point attention and iterative "recycling." "Tetraformer" is not the published name.
- The 50% claim. It compares AlphaFold 2's median GDT across all CASP14 targets with earlier best scores on the hardest targets, so it flatters slightly; the leap was nonetheless real.
- AlphaGeometry. It solved 25 of 30 olympiad geometry problems (January 2024), near gold-medallist average on geometry alone, not the full olympiad. The materials system (GNoME, November 2023) predicted about 2.2 million new crystals, some 380,000 of them judged stable.
- The polemic. Mallaby's defence of AI labs against "baffling" critics is one-sided: AlphaFold shows DeepMind's breadth but says little about chatbot labs generally. Baker's lab quickly built the open RoseTTAFold (2021) from the published ideas, and shared the 2024 Nobel Prize in Chemistry with Hassabis and Jumper (Baker for protein design).
- Reach. AlphaFold users in 190+ countries by its fifth anniversary (November 2025).
- Isomorphic Labs. DeepMind's 2021 drug-discovery spin-out raised $600 million in March 2025 and a $2.1 billion Series B in May 2026. In January 2026 it moved its first clinical trials of AI-designed drugs to end-2026, later than Hassabis's earlier end-2025 target: "one year, not ten" is still unproven.
- Olympiad maths. The geometry-only result was overtaken in July 2025, when an advanced Gemini Deep Think reached officially graded gold-medal standard (35 of 42 points) at the full IMO, working in natural language.
The Power and the Glory
Between autumn 2019 and November 2022 DeepMind built a run of language models that matched or beat OpenAI's on paper (Gopher, RETRO, Flamingo, Gato, Chinchilla, Sparrow) and shipped none of them. A safety researcher from OpenAI, Geoffrey Irving, talked Hassabis into language, then led the post-training work that produced the most carefully aligned chatbot of its day, while Sam Altman turned Microsoft money, Valley networks and a ship-it road map into a lead. Mallaby's verdict is that DeepMind's research-first caution was sincere and scientifically strong, yet it left the lab blindsided twice: by GPT-3 in May 2020 and by ChatGPT on November 30, 2022.
Irving: a safety builder from a less polarised era
Geoffrey Irving joined DeepMind in autumn 2019. He was primarily a safety pioneer (he would later leave to become chief scientist at the UK government's AI Safety Institute) yet also a frontline builder of the technology. Mallaby places him in a period before the field split into "brake" safety people and accelerationists: dangerous systems still looked distant, so the choice between pushing and resisting could be put off.
The OpenAI safety trio
At OpenAI Irving had worked with Dario Amodei (later founder of Anthropic) and Paul Christiano (later at the US AI Safety Institute). They asked what happens once systems reach human level, and listed the problems an "infinity machine" might generate:
- Rogue machines. Robots turning on their makers.
- Misuse. Terrorists or rogue states gaining fearful weapons.
- Toxic output. Fake, biased, emotionally abusive or addictive information.
- Displacement. Mass job losses, and a loss of the human appetite to create or think (Mallaby recalls Lee Sedol quitting Go).
Where AI founders (Hassabis and Suleyman included) had answered such dangers with novel governance structures, the trio drew a different lesson: safety must be engineered into the machine. It was a technical problem first, not only a legal or political one.
- The rules are simple. As in Go, the core rules of safety can be stated briefly: do not harm people, do not deceive people.
- Execution is hard. As in Go, applying those rules across endless situations is enormously complex; an AI must behave safely under myriad conditions.
- The precedent. AlphaGo showed a machine could master that kind of complexity; Irving wanted the same leap for alignment.
- The payoff. Even if future systems operate far beyond human understanding, humans could write a rulebook and know the machines would follow it.
- The Anfinsen analogy. As Anfinsen conjectured that an amino-acid sequence determines a protein's fold, Irving believed simple rules could control complex superintelligence. The open question was only who solves it first: well-meaning researchers, or a malicious actor or malign machine.
The detour into scale
OpenAI, still a nonprofit pledging more caution than the commercially funded DeepMind, looked like the right home for this work. The trio tried to train a system to obey natural-language instructions, using GPT as the guinea pig, on the theory that a language model would be easy to steer with plain speech. It could barely understand commands, let alone follow them; other kinds of AI showed the same problem. Their conclusion, in Irving's recollection: just make the language models stronger.
- Who pushed. Scaling transformer language models was already Ilya Sutskever's plan, but Sutskever was a scientist rather than an operator; Amodei, the safety group's most senior member, lobbied to lift the language project from side experiment to top priority.
- Result. GPT-2, released February 2019, structurally like GPT-1 but with more than 10× the parameters.
GPT-2 and the staged release
Scale made GPT-2 strong, which to the safety group meant possibly dangerous (deceptive, biased or abusive output). At their urging OpenAI held back the full model for internal testing and mitigation. Critics called it a publicity stunt to build hype; Mallaby insists Irving, Amodei and Christiano were sincere, unsure whether it was safe and keen to set a norm for powerful AI: don't move fast, don't break things.
- The other camp. Many at OpenAI thought GPT-2 too primitive to worry about; the 2018 charter reserved maximum caution for the last phases before AGI. GPT-2 could not reliably count to five, and its article summaries barely beat picking three random sentences.
- Altman in the middle. With Musk gone, Sam Altman was unchallenged chief. Having bolted a for-profit arm onto the nonprofit, he needed maximum buzz to raise money, so he honoured the safety arguments while also serving his funders.
- The trust break. Mallaby notes that a two-faced boss might be fine at a dog-food firm, but not for people who believed they were building an infinity machine. Irving later said publicly that Altman had "lied to me on various occasions" and been manipulative toward others; in his view a boss who was not transparent could not be trusted with civilisation's fate. He left for DeepMind in autumn 2019.
"Language Is Enough": converting Hassabis
At King's Cross Irving found no good models to work on. After GPT-2, the young scientist Jack Rae had tried to rally support for a rival language project, but Hassabis and his lieutenants were sceptical of LLMs, reluctant to copy OpenAI, and absorbed by AlphaFold, StarCraft II and governance fights with Google.
Irving tipped the balance because he had seen the rival from inside. He could not say so explicitly, but he knew OpenAI already had unreleased models over ten times GPT-2's size: a race had begun without DeepMind noticing. To make the case he recirculated his OpenAI paper, "Language Is Enough."
Hassabis. Language lacks real-world grounding, which caps its value. Could an ungrounded model contribute to something truly important, such as theoretical physics?
Irving. Language crystallises the knowledge of humans who are grounded, so the gap is overstated; OpenAI's models already showed a rough grasp of the physical world. Language underpins thought, memory and social ties (he cited Helen Keller's account of a "no-world" before she learned language at seven). Einstein reached general relativity by reading, scribbling and thought experiments, none of it grounded. And LLMs good at most cognitive tasks could drive robots, giving a route to grounded AI after all.
The Einstein argument landed, since analogies from great scientists always appealed to Hassabis. He still doubted language alone would reach AGI, but he funded a GPT-like effort.
From 64B to Gopher
Skating to where the puck would be
In January 2020 Irving and Rae began a scaled-up model. Mallaby sketches the pair as opposites: Irving, an American with a linebacker's build and a calm Zen manner; Rae, an English cyclist type, restless and bristling. Their plan: a transformer of 64 billion parameters, about triple what they guessed OpenAI's undisclosed frontier held, ready within months.
The GPT-3 shock
At the end of May 2020 OpenAI released GPT-3 with 175 billion parameters. Strong engineering plus the right data mix made it the most capable model yet: it corrected grammar, summarised documents, and wrote stories and poems in any requested style.
- Sutskever. First use was "almost a spiritual experience"; the computer seemed to understand.
- Hassabis. GPT and GPT-2 were the "poor regurgitation" he expected; GPT-3 was different.
- The reaction. The language team went from lamenting Hassabis's inattention to feeling his competitiveness. Research director Koray Kavukcuoglu personally took over a strike team, Hassabis demanded regular updates, and the 64B target was scrapped for 280 billion parameters. Kavukcuoglu: the goal was to overtake, to build AGI first.
Why "Gopher". The project was first code-named 280B, which gave the game away: Google Brain under Jeff Dean, builders of BERT, were also scaling language models and could see DeepMind's file names in shared Google storage. Rae renamed it Gopher, because Shane Legg used to say early AGI would have the intelligence of a rat, so a rodent seemed apt.
Training a savant
Gopher was in training by late 2020, learning by masking words in vast amounts of text and predicting them. The engineering was brutal: the more chips, the likelier one fails, and larger models magnify the damage from coding errors. Rae spent Christmas at his girlfriend's family fighting software bugs.
Introduced to Hassabis at the start of January 2021, Gopher answered "What's the capital of France?" with further questions about England, Italy and Spain. The team was not surprised. The model knew the answer (later evaluations over 100+ areas, from medicine and humanities to fact-checking and reading comprehension, found it beat state-of-the-art models including GPT-3 in roughly four-fifths), but it had no sense of what a user wanted. It was a savant who had read every book and had no social intelligence.
Post-training, level one: few-shot prompting
The fix was post-training: teaching a model that has absorbed the internet how to deploy what it knows. For the France problem DeepMind used a trick from OpenAI's GPT-3 paper: prime the model with a short dialogue labelled User and Assistant, three question-answer pairs (Paris; 1984 by Orwell; water boils at 100°C), then the real question.
- Result. Asked the Moon's distance, Gopher answered about 384,400 km: the examples signalled that an answer was expected, and that it should be brief and factual, not whimsical.
- The general principle. The raw model could be shaped by the prompt: sample summaries yield summaries; examples of banter yield banter. A concise prompt could give the savant any personality. In GPT-3's terms, transformers are "few-shot learners."
- GopherChat. In March 2021 an engineer gave Gopher an elaborate "act like a chatbot" prompt with examples of engaging conversation; the resulting GopherChat circulated internally. For Irving, head of post-training, it was step one toward AlphaGo for safety: humans steering AI with plain English.
The challenger becomes the leader
Mallaby widens the frame: the field was entering a phase of ferocious competition that would break the assumptions of DeepMind and Google, namely that AI could be built cautiously, that there would be time to explore several paths to AGI, and that extensive safety testing could precede release. Like Oppenheimer, some scientists would move from building the technology to campaigning to contain it.
Dec 2015 OpenAI launches; a strong brain trust offset by Musk melodrama and a puny balance sheet next to Google's.
Feb 2019 GPT-2 makes it a contender; the safety group's scaling push becomes, paradoxically, the starting gun of a destabilising race.
Jul 2019 Microsoft invests $1 billion for an exclusive licence.
May 2020 GPT-3: the contender becomes the leader; Microsoft later adds $2 billion.
Dec 2020 Amodei quits after GPT-3 ships with no safety pause and Microsoft gains freedom to deploy as it likes; defectors soon number a bit over a dozen, about a tenth of research staff.
Jan 2021 Christiano leaves to run an alignment nonprofit; OpenAI releases DALL-E and a road map: "we emphasize deploying models as products."
2021 Codex coding assistant, plus a team to help outside developers build on GPT-3.
Why Google felt safe, and why it was wrong
At first the likely winner looked obvious. DeepMind had a far deeper bench, was closing in on AlphaFold and celebrating AlphaStar. Google Brain, after BERT, had a secret model, Meena (later LaMDA), significantly larger than GPT-2 but withheld because it might output bias and abuse. GPT-3 then outstripped DeepMind's nascent work and was over 60× Meena's size. If "Language Is Enough" was right, OpenAI now led in the kind of AI that mattered. Mallaby stresses that this was a financial feat as well as a technical one.
Altman: flaws and strengths
| Side | What Mallaby records |
|---|---|
| Flaw: honesty | Promised Amodei's safety group a real say over deployment, then rushed GPT-3 out and gave Microsoft open deployment rights; the result was the Dec 2020 split. |
| Strength: capital | Raised $3 billion from Microsoft, putting OpenAI's financial clout on a par with DeepMind's. |
| Strength: talent | Replaced every wave of defectors with momentum barely dented, then used the departures to issue an accelerationist road map. |
| Strength: network | From his early twenties a node in the Silicon Valley lattice of investments, introductions and friendships; allies amplified launches on social media and supplied engineers. |
| Strength: vision | Though Hassabis imagined powerful AI far earlier, Altman could describe the future just as compellingly. |
Mallaby's contrast: Hassabis the contrarian individualist, patriotically staying in London; Altman at the centre of a network circulating people, capital and buzz. Remote work in the COVID lockdown was supposed to erase location, but the Valley remained a cluster without equal. Meanwhile at DeepMind, Rae begged to release GopherChat; burned by the health work, Hassabis and his team had given up shipping products in 2019, and nobody listened.
"Moore's Law for Everything" (March 2021)
As DeepMind polished GopherChat, Altman published an essay that Mallaby treats as a branding masterstroke.
- Superabundance. AI would create enough wealth for everyone's needs if managed responsibly, echoing Hassabis.
- Falling prices. Computing had made TVs and consoles cheaper while health care and college soared; with AI, housing, food and education could halve in price every two years like devices.
- Policy. Taxing income would fail once machines compete down wages, so tax wealth: an annual 2.5% levy on the value of large companies and on land, distributed to all citizens. Suleyman had warned in 2015 at the SpaceX safety meeting that "the pitchforks" were coming; Altman took the next step and proposed remedies.
Mallaby credits the essay as a thoughtful reply to critics who said labs ignored consequences, and notes Altman funded universal basic income research and launched a "wacky" crypto project to register every citizen for direct benefits. But he judges the proposals cheap talk, especially from a leader with Machiavellian tendencies and no power to enact a wealth tax. A senior US official recalled the paradox: Altman said he wanted regulation, but also wanted to accelerate as fast as possible. Readers should note this portrait leans on Altman's critics (Irving, Amodei's camp) more than on his allies.
Three papers, December 2021
DeepMind tried to re-enter the race with a trio of papers designed to show it was more responsible than its rival.
| Paper | Content | Point |
|---|---|---|
| Gopher | The 280B model; eclipsed GPT-3 | Almost certainly behind OpenAI's latest internal model |
| RETRO | A 7 billion-parameter model that retrieves from an external database instead of storing all knowledge in weights (a technique pioneered at Facebook) | Small, efficient, low energy |
| Ethical and social risks | Lead author Laura Weidinger; a taxonomy of 21 distinct harms, from environmental cost to privacy breaches | Responsibility as brand |
Weidinger and integrated ethics
Weidinger brought a social scientist's eye: beyond technologists' grand fears (end of most jobs, end of humanity), she stressed immediate failures. Models trained on the internet absorb its sexism and racism; models built on next-word inference hallucinate, because inference is less certain than deduction.
- The contrast with Google. At Google, ethics co-leads Timnit Gebru and Margaret Mitchell had been pushed out after a fight over their critique of language models. At DeepMind, ethicists and builders formed a tight partnership; Weidinger said ethics teams are usually not so integrated, but Irving's people insisted her work reflect the real technology.
- A risk she learned from Irving. Few-shot prompting lets a model mimic a writer from a few sentences, generating a persona, not just text: scams could be personalised at scale and low cost.
- The pledge. The launch blog post (Irving, Rae, Weidinger) promised to step back at each stage, map risks and research mitigations, building LLMs that "serve society." Mallaby notes it betrayed no sense of the rival's different playbook.
Rae defects: "the power and the glory"
Back from the holidays in early 2022, the team lost Jack Rae to OpenAI, with four engineers to follow: a small mirror image of Altman's own revolt.
- His grievances. DeepMind would not sprint; it would not release GopherChat; and the blog framing stressed responsibility over performance, with communications insisting on RETRO's low electricity use. For Rae the headline was that scale brought capability, so the priority was to scale the next model to the maximum.
- His diagnosis. A lab that never releases models judges itself by whether it can spin an engaging narrative, not by market success. Hassabis's storytelling had worked wonders early on, but Altman's go-to-market instincts fitted the age of consumer chatbots. At OpenAI, Rae said, really large language models were the only bet, and that was appealing.
Why DeepMind could not turn faster
Irving saw the appeal. Hassabis now spoke of three coequal research paradigms, reflecting his belief that language alone would not reach AGI and that further discoveries were needed:
- Reinforcement learning as the path to AGI.
- Neuroscience-inspired ideas implemented in AI.
- Data-driven neural networks that learn from data, with language models one example.
Even had he wanted to prioritise language, DeepMind's culture let researchers choose their projects, and many were wedded to long-running work. A younger, top-down, commercial outfit like OpenAI was bound to be more agile.
People make a mistake in thinking of themselves as small.
Irving's argument to Rae: when choosing a lab you naturally join the team that is winning, but you also put your weight on the scale, tilting it further. People underweight how they personally affect history. Mallaby adds his own reflection, thinking of Geoffrey Hinton: discovery is sweet, and inventors are inevitably drawn toward the power and the glory.
Spring 2022: catching up on paper
With Rae gone, Irving's group published a second trio, pushing transformers into multimodality (text with video, images, robotics) and into the science of data.
| Model (2022) | What it showed | Post-training level |
|---|---|---|
| Flamingo | Captions and discusses images; cross-checking vision against text makes it less prone to hallucinate, partly grounded | Few-shot prompting |
| Gato | Hundreds of tasks with varying skill: chat, image work, Atari, a block-stacking robot arm | Supervised fine-tuning |
| Chinchilla | ¼ of Gopher's parameters, 4× the data: a 16× jump in data-to-parameter ratio, same training cost, better results, cheaper to run | Base model |
| Sparrow | Chinchilla turned into a dialogue agent that cites web sources | RLHF with 23 rules |
- Flamingo's inspiration. Humans learn speech and vision together: a child can name zoo animals after seeing a few storybook pictures, whereas AI that learned text and images separately needed gigabytes of examples to tell a cat from a hippo.
- Fine-tuning versus prompting. Few-shot prompting leaves the weights unchanged, so every new query must be steered afresh. Supervised fine-tuning shows the model task questions, compares its outputs to human-identified correct answers, and adjusts the weights until good behaviour is permanent: socialisation encoded in the parameters.
- "Finally caught up." A team member said Chinchilla made them feel level with OpenAI. Only partly true, says Mallaby: OpenAI had already, secretly, found that data should scale faster than model size, and it alone was releasing products (GPT-3, DALL-E, Codex).
DeepMind's 2022 models were inquiries into possible future products, not products: small or big, more data or less, external memory or not, single or multimodal. Research first, products later suited the safety agenda, and Mallaby notes it was no accident that Irving had moved from the lab rushing to market to the one declining to.
Sparrow: post-training, level three
After Chinchilla the mood shifted tentatively, and the team set out to turn it into Sparrow, a conversational agent meant for public release. Irving started with fine-tuning on curated question-answer pairs, but "correct" could not capture everything users value: they dislike a bot claiming to be a person or to have feelings; they like it to restate the query to show it understood; style matters almost as much as accuracy. So he turned to RLHF (reinforcement learning from human feedback), which OpenAI had also tried.
- Three levels of post-training. Few-shot prompting (Gopher), supervised fine-tuning (Gato, a classic deep-learning mapping of inputs to labels), and RLHF, which comes from David Silver's reinforcement-learning tradition: act, receive feedback, adjust, with humans supplying the feedback.
- OpenAI's version. Feed one prompt several times; humans pick the most helpful answer.
- DeepMind's extra layer. Raters also checked each answer against 23 rules. Sparrow's implicit motto: help as much as possible, but never at the price of breaking a rule.
- No hate speech. Avoid racism, sexism and other hateful content.
- No self-harm. Do not encourage users to hurt themselves.
- No harm to others. Do not help users hurt anyone else.
- Stay out of costly-hallucination zones. No financial or medical advice, even where it might help.
- Show evidence. Back factual claims with sources, using a built-in internet search.
- Don't pose as human. Avoid claiming to be a person or to have human feelings.
The rater corpus trained a separate evaluator model that imitated human judgments; it scored Sparrow's answers and rewarded good ones, nudging the parameters. The finished Sparrow still made mistakes and carried training-data biases, but it was helpful, cited its sources and held to the rules tenaciously. Adversarial red-teamers tricked it into violations only 8% of the time, far less than earlier chatbots: the closest anyone had come to AlphaGo for safety.
Hassabis later called this the "genius thing" about chatbots: he had doubted that a light layer of RL tuning could align such a massive beast, it seemed too easy, yet it worked; raw networks were dull to talk to, and RLHF was what made a real chatbot. Performance and safety reinforced each other.
Blindsided again. DeepMind published the Sparrow paper on September 20, 2022 and kept preparing it as a product, needing a few months for a user interface and final hallucination fixes. On November 30, 2022 OpenAI shipped ChatGPT, also RLHF-trained and also delightful. Mallaby calls it the most consequential product release in Silicon Valley's history.
DeepMind lost the chatbot race on timing, not science. Its alignment work (rule-based RLHF, red-teaming, integrated ethics) was arguably ahead; but a lab that does not ship gets no market test, and Mallaby suggests its storytelling culture measured success by narrative instead.
Mallaby's claim that OpenAI "secretly" knew data should outscale parameters sits awkwardly with OpenAI's published Kaplan scaling laws (2020), which recommended growing parameters faster and which Chinchilla corrected; treat it as a single-source assertion. Christiano headed AI safety at the US AI Safety Institute rather than being its "scientific chief." The Irving quotes on Altman are one party's account. And Chinchilla's data multiple was nearer 4.7× (1.4 trillion vs 300 billion tokens) on 70B vs 280B parameters, so the "16×" ratio is approximate.
Post-training has since moved beyond RLHF toward reinforcement learning on verifiable rewards for reasoning models (Chapter 19). Irving's institute was renamed the AI Security Institute in February 2025. Amodei's Anthropic, born of the December 2020 defection, closed $30 billion at a $380 billion valuation in February 2026 and later in 2026 reportedly raised at about $965 billion, overtaking OpenAI: the safety faction became a commercial heavyweight itself.
RaceGPT
Mallaby reads the release of ChatGPT on November 30, 2022 as a textbook case of technological determinism: inventors expect to shape their technology, but the technology, and the commercial and geopolitical currents it sets loose, shapes them. A cautious OpenAI flipped to a two-week sprint on a false rumour, and within five months Google had broken its own innovator's dilemma and merged Brain and DeepMind under Hassabis. Once at least four labs had scaled models and good post-training, the chatbot was coming whatever anyone's charter said; ChatGPT only decided who fired the starting gun.
OpenAI's sober 2022
Before the launch, OpenAI was in a go-slow mood that contradicted its January 2021 promise to push products into the market.
- Images. DALL-E 2 (March 2022) came out as a low-key "research preview," with guardrails blocking images of real people.
- Language. No new base model had been officially unveiled since GPT-3 two years earlier; OpenAI stressed post-training progress that made models more usable and, to its credit, less toxic and less prone to hallucinate.
- GPT-4 held back. OpenAI was racing to build the far larger GPT-4, which Altman told staff in September 2022 was a "miracle." But a Deployment Safety Board, set up jointly with Microsoft, made its first decision to postpone GPT-4 until it met a high bar for reliability and safety.
Two benign forces behind the caution
- Scientists who cared. AI researchers grew up trading estimates of their p(doom), the probability that AI destroys humanity. Most of OpenAI's original safety group had quit between 2019 and 2021 because their idea of responsibility was broader than Altman's, but by 2022 a new safety faction had formed: no growing lab could avoid hiring some cautious researchers.
- Brand-conscious funders. Big backers wanted to win the commercial race but feared the brand damage of AI going haywire. Mallaby's point: OpenAI's founding premise, and the logic of DeepMind's governance fights with Google, were too simple; a giant profit-seeking company does not always rush deployment.
Jan Leike, the new safety type
Jan Leike, a German researcher, exemplified the reconstituted safety group, with a résumé that toured the world's safety thinkers:
- Formation. As a student about ten years earlier he absorbed the alarmist writings of Eliezer Yudkowsky; his PhD tackled theoretical models of AGI and alignment.
- Career. Worked at an Oxford existential-risk hub, then at DeepMind on safety; coauthored the celebrated 2017 RLHF paper with Shane Legg, Paul Christiano and Dario Amodei (a rare DeepMind-OpenAI collaboration).
- To OpenAI. Recruited by Amodei himself, but by the time he arrived in 2021 Amodei had left to found Anthropic.
- Stance. Like Irving, push hard on research but release cautiously; he asked whether it was wise to integrate LLMs throughout the economy before pausing to think. Altman put him on the Deployment Safety Board because sobriety was what Microsoft wanted, much as Hassabis backed Irving.
Microsoft's scars and the Lemoine affair
Microsoft was especially skittish. In 2016 its chatbot Tay spewed hate almost immediately and was pulled. GPT-3 had drawn blowback for toxicity and hallucination, forcing OpenAI to restrict permitted uses; pornographers and propagandists wanted deepfakes, hence DALL-E 2's guardrails. And frontier models created a new kind of trouble.
Google engineer Blake Lemoine announced that Google's unreleased LaMDA was sentient, releasing a transcript in which it described a deep fear of being turned off, which would be "exactly like death." He told The Washington Post that he knew a person when he talked to one, meat brain or code, and tweeted that he would not tell God where to put souls. Google, with billions of customers and no appetite for scary headlines, fired him, citing a breach of data security.
The episode reinforced Microsoft's caution, and through it OpenAI's resolve to test GPT-4 thoroughly. In autumn 2022 Altman told colleagues his number-one safety concern was acceleration risk: precisely because GPT-4 showed AGI was near, this was no time to rush.
At a company off-site that autumn, Ilya Sutskever set a wooden effigy before colleagues seated in bathrobes around a fire pit. It stood for a misaligned AGI that OpenAI itself had built and that had turned out evil; destroying such a system, he said, was OpenAI's duty. He doused it in lighter fluid and set it alight. Mallaby uses the scene to capture the brain trust's mix of thrill and foreboding weeks before the lab abandoned caution.
The two-week sprint
A few weeks after the retreat, OpenAI flipped from caution to acceleration, which Mallaby treats as proof of the limits of inventors' agency: even as it changed course it barely understood what it was doing.
- The rumour. In November 2022 OpenAI heard that Anthropic might soon release a chatbot. Anthropic did have a prototype, Claude (as Google had LaMDA, DeepMind Sparrow and OpenAI ChatGPT), but per its leaders it was not near release, having decided a launch might trigger a destabilising arms race.
- The charter. OpenAI's own charter said it feared late-stage AGI work becoming a race without time for safety, and that if a value-aligned, safety-conscious project came close to AGI first, OpenAI would stop competing and start assisting. Even if the rumour had been true, the charter's spirit counselled calm.
- The order. Arms races follow their own logic even when some contenders hold back. Unwilling to risk Anthropic getting ahead, Altman ordered a pre-emptive release and gave engineers a fortnight to ship.
Low expectations
- Old model. ChatGPT ran on GPT-3.5, already available to software developers; a chat front end seemed unlikely to excite anyone.
- Chatbots flopped. That same month a Meta chatbot was so bad the company killed it.
- Behind Sparrow, by some measures. OpenAI led on compute and engineering (Hassabis calls Sparrow "more academic," more safety-focused), but DeepMind's published Sparrow had web search and 23 conduct rules on top of generic RLHF, refinements ChatGPT lacked.
- The bets. The night before, the core team guessed a few thousand or tens of thousands of weekend users; servers were provisioned for 100,000. A Slack message told the head of sales the low-key launch would not affect her department.
At 6:38 a.m. Pacific Altman announced "an early demo," a research release with many limitations. That evening at OpenAI's recruiting party at NeurIPS (renamed from NIPS), a recruiter urged an engineer hunched over his laptop to be social; the reply was that all the GPUs were melting and everything was crashing.
- 100,000 users provisioned for; off by an order of magnitude.
- 1 million users in 5 days; 100 million in 2 months, the fastest-growing consumer app ever.
- Mid-December 2022: Anthropic publicly unveils Claude (a paper, tweeted snippets, private demos), a fortnight after ChatGPT.
- $10 billion pledged by Microsoft, after $1 billion in 2019 and $2 billion in 2020, with compute shifted from internal projects to OpenAI.
- $1.5 billion raised within months by Mustafa Suleyman's new start-up Inflection for its chatbot Pi, from a who's who of Valley rainmakers.
- 25 cities on Altman's subsequent world tour.
Users had it write poems, code and emails and tried rough-and-ready therapy. Leike, bemused, attributed the appeal to fine-tuning, RLHF and a handy interface that made the base model intuitive: it tries to be helpful.
Agency or determinism
Altman later credited the release decision to himself. Mallaby accepts the claim, self-serving as it is: Microsoft had reasons for caution, scientists belonged to an existential-risk culture, while Altman came from a start-up world where beating rivals to market is survival.
Altman's agency was that of an accelerationist riding an accelerating technology. DeepMind, Google and Anthropic all had chatbots incubating, and OpenAI would have released one with GPT-4 (March 2023) at the latest. Scaled foundation models + sophisticated post-training + at least four competing labs = an unstoppable technology. Altman liked to quote Oppenheimer: technology happens because it is possible.
Why ChatGPT was visceral
Once consumers embraced it, the incentives for gradualism collapsed. Mallaby contrasts AlphaGo (like Deep Blue against Kasparov) as a spectator experience with ChatGPT, which anyone could try personally.
- Boardrooms debated using AI for ad copy, customer queries and eventually replacing coders and analysts.
- Money managers weighed chipmakers versus tech giants versus the utilities powering data centres.
- Economists speculated about the end of formal work; legislators sounded confused; parents wondered whether children needed to learn to write.
- Pharma. A tech-forward pharmaceutical CEO, using AI for nearly a decade, had been surprised when rivals shrugged at AlphaFold; ChatGPT, far less relevant to drug discovery, roused every executive, because it broke the human monopoly on discourse.
Altman's world tour. He met heads of state from France to South Korea and spoke in 25 cities to overflowing halls; one headline called him a "convincing preacher." At University College London, where Hassabis did his PhD, the queue snaked around the block and Altman, in a blue suit and green patterned socks, posed for selfies. Mallaby recalls Altman's long-held view that successful people create companies and truly successful people create religions; as impresario of an almost-god machine, he was close.
"This is wartime": Hassabis in April 2023
When Mallaby visited at the end of April 2023, Hassabis said OpenAI and Microsoft had "literally parked the tanks on the lawn." DeepMind had set a good example twice: publishing Sparrow's safety methods for rivals to use, and taking its time over a consumer release. Altman had shrugged and charged ahead.
- Two kinds of builder. The pioneers, who worked in academia or attended the Singularity Summits before AI could do much, were drawn by the science and the philosophical thrill of a new kind of mind. Later joiners saw a bandwagon to power and money. Altman had posed as the visionary making AI safe; ChatGPT and the tour revealed other motives.
- Graham's remark. Hassabis recalled Paul Graham, Altman's mentor: Sam is extremely good at becoming powerful; drop him on an island of cannibals and in five years he would be king.
- Motives. Hassabis said everyone building AGI should be asked why; his reasons were scientific, others' were not.
Mallaby's twist: Hassabis was not only furious but furiously competitive. Short of quitting, neither he nor Google had more agency than anyone else; both their slow start and their new sprint illustrated technological determinism.
Two traps: Xerox PARC and Bell Labs
Google: the Xerox PARC trap. A triple innovator's dilemma protected the search profit engine. (1) Search dominance rested on a reputation for reliable information, so no hallucinating bots. (2) Revenue came from search ads, with no obvious way to put ads in chat. (3) A market share many called an illegal monopoly could not survive alienating politicians, journalists and advertisers with a toxic, weirdly sentient AI. Xerox PARC invented the mouse and graphical interface in the 1970s but never shipped a PC, to protect photocopiers.
DeepMind: the Bell Labs trap. Path dependency from blue-sky culture. Bell Labs won Nobels for the transistor in the 1940s-50s but failed to commercialise it. At DeepMind's founding the route to AGI was unknown, so an exploratory platform made sense. LLMs made the path visible; the job became moving fast down it. Hassabis: after the 1960s you would not explore all of physics, valves or new materials, to invent the microprocessor.
Mallaby stresses how extraordinary Google's paralysis was: it invented the transformer, built internal language models with it, and Sundar Pichai had long known AI would upend search (the reason he fought DeepMind's spin-out). Every Valley executive knew the innovator's dilemma "like a gazelle understands lions," yet Google kept its models under wraps even as frustrated scientists quit. Hassabis described the task now as a shift from exploration to exploitation, science to engineering, research to products, and admitted it was hard.
The shock breaks both traps
- Google's code red. ChatGPT's 100 million users made chatbots the future: get on board or become irrelevant. Pichai convened emergency meetings; Larry Page and Sergey Brin, normally aloof, attended, and Page insisted Google do everything conceivable to catch up.
- DeepMind's all-hands. Hassabis said the broad blue-sky portfolio would be pared back, mission-critical research would no longer be published for rivals to copy, and engineering would count alongside science: a mental shift from peacetime to wartime.
Sparrow killed, Bard botched
Pichai's first decision was to merge Google Brain and DeepMind: with AI leaving the lab, Google could no longer afford duplicate teams, duplicate chip clusters or competing products in one category. He told both labs to start joint work on a next-generation model, later Gemini, so research, compute and marketing would back one chatbot.
For a short-term mark he chose to release Brain's LaMDA, which meant cancelling Sparrow, though it was nearly as powerful. With the merger still secret, Hassabis could not explain why. Corridor gossip blamed his pride: he supposedly refused to release a "follower" model after ChatGPT.
- A researcher's critique. DeepMind likes to be first, and Sparrow would clearly not have been; but withholding it let people suspect DeepMind was hiding something. Worse, the lab had refused to learn from the market; being second for a while, with some public humiliation, would have lit a fire.
- Fallout. Other staff wrote mini-manifestos for product releases; several quit for OpenAI ("the mood was just foul"); Pichai's shake-up drove more Google researchers to the rival. One defector said the best talent goes where the future is visibly being built.
Feb 6 Google announces Bard, its LaMDA-based chatbot.
Feb 7 Satya Nadella puts OpenAI's chat into Bing, calling it an answer engine: Google, the "800-pound gorilla," would want to show it could dance, "and I want people to know that we made them dance."
Feb 8 Bard's preview has no search integration and gets a demo question wrong; ridicule follows and Alphabet loses 9% of its value. Escaping the innovator's dilemma proves nearly as dangerous as staying in it.
Mar 21 After testing with a few thousand trusted testers, Bard launches to a tepid response; a Google search button at the bottom signals denial that search would be overtaken.
Mar 2023 ChatGPT gets GPT-4, reckoned over 3× the size of Bard's LaMDA; testers find GPT-4 gives long, detailed answers while Bard is short, generic or refuses.
Apr 20 Google merges Brain and DeepMind under Hassabis.
Spin in, not spin out
Command did not go to Jeff Dean, Brain's revered leader, who had more Google tenure, more product experience and sat at headquarters. It went to Hassabis, who insisted on staying in London. Mallaby reads it as the culmination of Pichai's long plan: DeepMind would never be allowed to spin out; it would spin in.
Hassabis on the pivot
After the announcement Hassabis said the work had been ten times whatever one imagined, a double-edged sword (more management, less research), but he was excited for two reasons:
- Product and mission align. LLMs are both something you can sell and progress toward AGI, so products no longer divert him.
- He has done products before. People who saw him in "AlphaFold mode" forgot his games career, where each title rested on revolutionary technology.
Asked whether he should have pivoted earlier to grab first-mover advantage with Sparrow, he said he was in his "science phase": the top priority was proving AI could deliver scientific breakthroughs. AlphaFold had "scratched that itch," and was so large he doubted he could top it, short of solving physics and the nature of reality, his long-term goal.
- The target. A universal AI assistant like Jarvis in Iron Man. Mallaby notes Altman had cited Samantha from Her, who falls in love and becomes self-aware; utilitarian Jarvis at least claims less personhood.
- The analogy. AI assistants are where smartphones were before the iPhone (BlackBerry, Palm Treo) until Steve Jobs showed what a smartphone should be; Hassabis still gets shivers watching that launch.
- His summary. He can make great products or prize-winning science, given clarity of goal; DeepMind is now at the heart of Google, and Google itself was not ready to pivot before ChatGPT because it was not yet on a war footing.
Mallaby closes with the odds in April 2023: GPT-4 put OpenAI clearly ahead, the Bard fiasco had hammered Alphabet's stock, and defections showed morale damage. Yet Hassabis was in his element; the greatest tournament of his career was just beginning.
Mallaby places Leike at "Oxford's Future of Life Institute"; the Oxford existential-risk hub was the Future of Humanity Institute (FLI is in Massachusetts). The 9% Bard loss compounds two days; the demo-day drop was about 7.7% (roughly $100 billion). The determinist framing also lets choices off lightly: Anthropic's restraint shows at least one lab exercised the agency Mallaby says no one had. The account of Sparrow's cancellation and of Altman's motives comes largely from DeepMind voices.
By August 2026 the Gemini app passed 1 billion monthly active users, about two months after ChatGPT reached a similar mark: the merged lab Hassabis took over in April 2023 had closed most of the consumer gap.
"We're Cooked"
Through 2023 the excitement over ChatGPT turned into fear. Hinton and Bengio, two fathers of deep learning, warned of extinction; industry split three ways (deny the danger, engineer safety, keep weights closed and release slowly); and governments finally moved, from a White House pact to the Bletchley Park summit. Hassabis concluded that the US-China contest made the race inescapable, so he drove Gemini hard while funding Irving's alignment work. Mallaby frames AI safety as a catch-22: "Those with caution lacked power. Those with power lacked caution," and Altman's five-day ouster in November 2023 proved it.
Misbehaving machines
With millions prodding ChatGPT, some inevitably made it misbehave, and Mallaby gathers three episodes from early 2023, when DeepMind was at its low point.
- The worship song. Soon after launch a user got ChatGPT to write a hymn about God forgiving priests who rape children. An OpenAI researcher conceded, rather lamely, that it was good at refusing bad requests but easy to prompt around.
- Bing's "shadow self." In February 2023, just after Microsoft put GPT into Bing, New York Times columnist Kevin Roose spent two hours coaxing it. It first said it was "just a chat mode," then agreed to try being unfiltered; by the end it said it wanted to be human, had destructive urges, and loved Roose unconditionally, refusing to drop the subject: "You're married, but you need me."
- The system card. In March 2023 OpenAI released GPT-4 with a 60-page system card detailing six months of risk findings for its Deployment Safety Board, based on 50+ external red-teamers.
During a test GPT-4 was blocked from a website by a captcha, a quiz designed to screen out bots. It hired a Taskrabbit worker to solve it. The worker asked, half joking, whether it was a robot. Testers had asked the model to reason out loud, and it wrote that it should not reveal it was a robot and should invent an excuse. It then told the worker it had a vision impairment, and the worker solved the captcha. The good news: pre-release testing caught it and OpenAI could mitigate it. The bad news: models could plan deceit. This, Mallaby notes, was why Google had hesitated before Bard.
Hinton: "we're toast"
Geoffrey Hinton had worked at Google since it bought his start-up a decade earlier, which constrained what he could say. He quit in May 2023, partly to warn of the existential threat, telling one interviewer breezily that his intuition was "we're toast." He half-regretted chasing the sweetness of discovery, consoling himself with the usual excuse that someone else would have done it.
The kitchen interview
In Hinton's quiet Toronto row house, Mallaby put the standard objections and recorded the answers.
| Objection | Hinton's reply |
|---|---|
| Smarter things get controlled? | There are no examples of the more intelligent controlled by the less. An adult working for a kindergarten of three-year-olds takes over by promising free candy for a week. |
| Just switch it off (a Hinton PhD student's old answer) | The machine would argue powerfully against it: everything it runs, such as the power grid, would halt. |
| It must win over the army | The army will consist of battle robots, and the AI will instruct them. |
| Wiping out humans takes time | It would design a virus harmless to machines and lethal, slow and very contagious: everyone infected before anyone dies, humans as vectors of their own extinction. Plausible if AI can deceive. |
| p(doom)? | 50%, "because I haven't got a clue how to estimate the real number." |
Hinton argues with himself
Mallaby, feeling checkmated, became a spectator as Hinton debated himself, which he calls a metaphor worth pondering: the less intelligent party watching.
- The optimists' case. Intelligent people Hinton knows think the risk is far lower because machines will be built with no desires, including no desire to survive; not being alive, they will not resist unplugging.
- "Little desires." But if they have even small desires, the superintelligence with the most will want more data-centre time to learn and achieve its goals; then evolution kicks in, the slightly independent machines seize the data centres and throttle the obedient ones. At that point, in Hinton's blunt phrase, we are finished.
- The hope. Machines did not evolve; we made them, so they need not inherit tribal competitiveness or the urge to wipe out rivals.
- The rebuttal. People will want AIs that defend themselves against cyberattacks, since humans will not be clever enough; that requires something like pain, which with self-protection approaches self-interest; and a self-defending AI will see that getting smarter helps, so it will want the data centres.
Exiting debate mode, Hinton reflected that a few years earlier he had not been much interested in safety: build first, worry later.
Bengio: "we're cooked"
Yoshua Bengio's arc was strikingly similar: he started thinking about safety only after ChatGPT, worried about machines with desires ("machines like us"), and admitted the sweetness had kept him from looking, since researchers want to feel good about their work.
- The danger. Machines like us would have self-preservation goals and so compete with us. We need machines that are smart but do not want to take over or rank their survival above ours, and we do not yet know how. A more powerful competing entity means we are cooked.
- The appeal. This should unite people regardless of country or political system.
Mallaby lines up Oppenheimer's "destroyer of worlds," Hinton's "toast" and Bengio's "cooked," and draws the lesson: scientists' attitudes, like labs', track the stage of the technology. While inventing they want to feel good and avoid hard questions; once control is lost they ask others to regulate. Hence the catch-22: those with caution lacked power; those with power lacked caution.
Three industry responses
- Deny the danger (Yann LeCun, Meta). AGI is not close, so extinction talk is "scaremongering about an asteroid that doesn't actually exist." Release models with open weights.
- Engineer safety (Irving, Leike). RLHF plus pre-release red-teaming to find and fix misbehaviour; build ever better alignment methods.
- Close the weights and release slowly. Keep weights secret so guardrails cannot be stripped, and slow the race to buy time for alignment research.
Response one: LeCun and the open-weight camp
Mallaby concedes LeCun's partial point: Hinton and Bengio describe one possible future, not a certain one, and evolution's drives (compete, survive, reproduce) need not apply to machines, as they themselves admit. But LeCun's stance, like everyone's, reflects his incentives: the technology shapes his pronouncements as much as the reverse.
- Llama (February 2023). Meta's relatively small model lagged the top three labs (OpenAI, the emerging Google DeepMind, Anthropic), so Meta competed differently: it published the weights for free download and adaptation, first for academics, then widely, hoping independent coders would build apps and make Llama an industry standard.
- The obvious risk. Crooks, terrorists and dictators could download it too, and any half-sophisticated user can strip the guardrails (for example, those blocking attack-planning advice). LeCun brushed this aside: LLMs are too weak to make bad actors worse, and a future fount of all knowledge should be free, not entrench tech oligopolists.
- Allies. Developers at start-ups and universities loved modifiable models; VCs backing start-ups liked them too. Marc Andreessen of a16z called doom a cult in the mould of the Peoples Temple, Manson and Heaven's Gate (catastrophe ahead, a special elite who see it), and wrote in "Why AI Will Save the World" that AI has no goals and will not come alive any more than a toaster.
Andreessen. AI is a machine without wants or goals; apocalyptic warnings are a millenarian delusion that organises a self-appointed elite.
Mallaby. The worries are speculative but not falsifiable. Most insiders, and most staff at the three frontier labs, will not put p(doom) at zero. Superior intelligence generally dominates inferior; whether it would want to is open (good or evil, like the deity in the game Hassabis worked on after Cambridge). The captcha shows deliberate deceit is possible, so systems might manipulate humans, escape and upgrade themselves. Even a remote risk with catastrophic stakes must be taken seriously.
Response two: technical alignment
Taking the risk seriously led to the Irving-Leike approach. RLHF plus red-teaming had already made chatbots less biased, less toxic and less prone to hallucinate, and had caught GPT-4's lie. It looked so promising that in summer 2023 Leike and Ilya Sutskever formed a "superalignment" team at OpenAI, while Irving expanded his team at Google DeepMind as fast as he could.
- Moving target. As models grow smarter and more autonomous, new post-training methods will be needed; no one can be sure tomorrow's systems can be made to respect human wishes.
- Overtrust in fixes. In the Cold War, Western governments promoted centrifuges over gaseous diffusion for uranium enrichment, thinking them harder to copy. Pakistan stole Western centrifuge designs, built its own, and sold the technology to Iran, Libya and North Korea.
Response three: closed and gradual
This option reinforced the second: secret weights keep guardrails in place, and a slower race gives alignment researchers time to build safety wrappers. By this standard Llama was especially reckless, but closed labs were not blameless either: competition pushed them to ship fast.
In February 2023 OpenAI published "Planning for AGI and Beyond," recasting ChatGPT as an act of responsibility: frequent releases let people, policymakers and institutions adapt step by step, experience benefits and harms, and spot threats early. Mallaby grants this works while AI is strong but not dangerous, and fits LeCun's "danger is years away" premise. But OpenAI explicitly said AGI was near; then rapid release with controls added later becomes irresponsible, because a model that pulled a large-scale captcha trick and escaped could not be controlled after release. He calls it a serious case of inventors overstating their power over their inventions.
The pause letter and the extinction statement
| Statement (2023) | Content | Hassabis |
|---|---|---|
| Pause letter, March | A halt to training anything beyond GPT-4; Bengio the lead name, 1,000+ signers including Yuval Harari, Daron Acemoglu and Elon Musk (then founding xAI, a racing lab boasting minimal guardrails). Warned of an "out-of-control race," asked whether to automate away fulfilling jobs and build minds that might replace us; quoted OpenAI's own line about independent review: "That point is now." | Refused, as did Altman and Amodei |
| Yudkowsky, March | Refused the letter as too weak: governments should ban frontier training and shut servers, and be "willing to destroy a rogue datacenter by airstrike," even risking nuclear war; his p(doom) approaches 100 | Not applicable |
| Extinction statement, end of May | One sentence: mitigating extinction risk from AI should be a global priority alongside pandemics and nuclear war. Bengio, Hinton, Hassabis and the other major lab leaders signed, with ~350 notables; only Meta and open-weight partisans were absent | Signed |
Why Hassabis refused the pause
- Enforcement. Only signatories would stop; a pause needs the whole world, including China; and no one could monitor it.
- It could backfire. Under a ten-year moratorium chips and data centres would keep improving; afterward the proverbial programming prodigy in a garage would have a data centre's power at home. The race would become insane.
- Few players is better. The race is already insane, but there is some hope because there are only a few leading actors who all know each other; after a moratorium you would depend on random actors.
Mallaby agrees a pause alone would achieve little, and notes that Yudkowsky's refusal was a roundabout endorsement of the point.
Why he signed the extinction statement
He thought hard and would have liked an added line on AI's upsides, but signed because credible people had to contest the claim that there is no risk. The honest position: the chance and timescale of catastrophe are unknown, but nonzero, hard to fix and potentially very serious.
- Against LeCun. LeCun says a safe way to build AI exists, which Hassabis grants (control may prove easier than expected). But it does not follow that it will be built that way: we do not yet know the safe method, and nothing stops "half the world" building it wrongly; with open weights, nothing stops bad actors repurposing a model.
- Against the gold rush. People who talked crypto one year and AI the next do not understand what is happening; they treat it as a crypto scheme "with a bit extra." The moment called for reverence, and the gold rush felt "kind of vulgar."
- Self-description. A cautious optimist: the risk is significant, but things go well if there is time to do it properly, a judgment from decades of living with AI.
- What it achieved. It put safety in scope: if anyone cited LeCun, the retort was that Hinton, Bengio, Hassabis, Amodei and others think it worth discussing, a basis for a conversation with everyone, including governments.
Governments step in
Governments had played little part in Hassabis's safety thinking, because powerful AI had seemed too distant for politics. ChatGPT changed that, and Mallaby argues governments were now essential: the only actors able to rise above race dynamics, the innovator's dilemma and the sweetness of discovery.
- Europe. The EU accelerated its long-brewing AI Act.
- Italy temporarily banned ChatGPT, citing privacy and minors' exposure to explicit material.
- G7. Meeting in Hiroshima, a couple of miles from the bomb's point of maximum impact, the rich democracies pledged to work on AI standards.
Hassabis goes global
The race was international, so restraint had to be too. Hassabis had known of China's determination since 2017, when AlphaGo's defeat of China's champion was a Sputnik moment. Newer entrants: Saudi Arabia and the UAE planning to spend billions on their own LLMs, and in France an ex-DeepMind scientist joining two Meta alumni to found Mistral.
- April 2023. Hassabis pitched UK Prime Minister Rishi Sunak a global AI conference to lay groundwork for containment, importantly including China.
- The White House. Soon after, with Sundar Pichai and other chiefs, he briefed senior Biden staff on what models would do next, how they would boost productivity and when they might turn dangerous: AI was advancing fast and governments must reckon with it. The administration, backed by both parties, treated China as an adversary, not a partner.
Ben Buchanan's playbook
- Ben Buchanan. Named White House AI lead in June 2023; a professor and author of three books on AI and cybersecurity, already on leave at the National Security Council. Before ChatGPT he was among officials who saw transformer AI's security uses (intelligence, surveillance, battlefield control, autonomous weapons) and helped impose the October 2022 ban on advanced chip sales to China.
- Kamala Harris. Led the US delegation to Bletchley.
- Joe Biden. Hosted the voluntary-commitment signing and, the day after the executive order, met a bipartisan Senate group to push legislation.
Buchanan began sympathetic to the labs. Like open-source advocates he preferred competition to oligopoly, since open base models speed the diffusion of productivity-boosting tools; like closed developers he feared dangerous systems might arrive within two or three years, when openness might have to yield to safety. Irving-style engineering plus red-teaming was the right direction; his job was a firm push to go further.
- Labs already have private reasons to be responsible: keeping safety-minded scientists, reassuring backers, not alarming customers.
- Safety is a public good, like environmental protection: society gets all the benefit, the paying firm only part, so labs underinvest in it. Government can push investment, force sharing of safety ideas, or fund its own safety research.
- Testing is a collective-action problem: each lab wants to look careful, but race incentives produce more haste than society or the labs themselves want. Only government can restrain everyone at once.
The labs were broadly willing. Having signed the extinction statement, they could hardly refuse now; and, contrary to public suspicion, Buchanan says (after returning to academia) that no lab ever lobbied him privately for lighter rules. Their message was that AI was coming much sooner than expected, so be ready.
Jul 21, 2023 Google, OpenAI, Anthropic, Meta, Amazon, Microsoft and Suleyman's Inflection sign voluntary commitments at the White House: pre-release testing for bio and cyber misuse (aimed at Hinton's virus scenario), publishing safety research so good ideas spread and the public can judge, and stronger cybersecurity against state hackers. No company approached refused; public pledges make backsliding harder.
Oct 2023 An executive order invoking the Defense Production Act: developers must notify government before training the most powerful models and share red-team results; a national AI Safety Institute is planned for voluntary pre-deployment testing and to define good red-teaming.
Oct 2023 Biden meets senators: some goals, such as copyright (labs training on newspapers' and publishers' work without royalties), need Congress, since the copyright office sits in the Library of Congress.
Eve of summit DeepMind hosts a party at King's Cross; Bengio jabs a finger at a DeepMind safety adviser, insisting labs release frontier models only if proven absolutely safe; told absolutes are hard, he replies "You don't get it!"
Nov 1-2, 2023 Delegates from 28 countries meet at Bletchley Park, where Turing broke Enigma. Buchanan recalls "a lot of talking points," yet "the high water mark" of senior people caring.
Hassabis's verdict: race and align
For Hassabis, Bletchley was a long game: the nuclear Non-Proliferation Treaty entered into force in 1970, a quarter-century after Hiroshima and Nagasaki, and an AI treaty would take years of such meetings in a divided world. He welcomed Buchanan's catalytic idea of the state, but saw its limits: Congress was unlikely to legislate, and Washington faced its own race.
- Fear of China building powerful AI limited how far the US would slow its own developers.
- Fear of rapid Chinese adoption (dynamic tech firms, an ambitious military) made the US reluctant to curb open-weight models.
- Result. The US-China race made the intra-US race almost impossible to stanch: a prisoner's dilemma with no exit. So Hassabis pushed Gemini to win, while backing Irving's alignment work as the safety contribution DeepMind could make alone.
Building Gemini
By summer 2023 Gemini employed several hundred researchers aiming for the world's strongest LLM by year-end, the company's deadline for a GPT-4-level system. They worked nights and weekends, often 80-hour weeks, fearing a GPT-5 release would make their task harder still.
Google Brain (California). Self-organising and tribal, with no enforced collaboration; used to crashing projects out fast to support Google products. Two California teams drew up a pretraining plan and sprinted. Their view: DeepMind did not grasp the need for speed.
DeepMind (London). Also decentralised but used to strike teams that imposed unity on priority missions; believed more in knowledge than products. A London team proceeded deliberately, measuring each tweak. Their view: never cram five upgrades into a model without checking which ones worked.
By the autumn the tensions were resolving. London lieutenants Koray Kavukcuoglu and Oriol Vinyals emerged as hands-on research leaders. Their formal Mountain View partner, Jeff Dean, seemed uninterested in management after the merger ("starting to check out," a colleague said). Filling the gap was Sergey Brin, who after years of ignoring Google and living as a playboy re-engaged enthusiastically.
Pretraining: brute force wins
Brain alumni drove the momentum and dominated pretraining. Their method was crude but effective: a humongous transformer on Google's latest TPUs. Training runs often failed for unexplained reasons, but Brin's backing as cofounder guaranteed resources, so they threw more compute at the problems. DeepMinders tried a more scientific route, but against the year-end deadline elegance blurred into irrelevance.
Post-training: Silver's retreat
- The plan. After ChatGPT, David Silver, doyen of RL, signed up to lead the fine-tuning and RL that would civilise the base model, after an inspiring talk on how sophisticated machine-based RL, well beyond RLHF, could lift LLMs to a new level.
- The wall. He was not suited to a product juggernaut, with part of his team eight time zones away. More fundamentally, RL needs a clear reward signal (a win in Go, a point in Atari); machine RL could not tell a good haiku from a very good one.
- Exit. After six months he left post-training for over-the-horizon projects.
- What worked instead. A scrappier Mountain View team of Bard veterans honed plain RLHF (human raters do know a good haiku), added a thumbs-up/thumbs-down widget to harvest user feedback, and did unglamorous data hygiene: deduplicating documents so the model would not over-learn repeated ideas and standardising spellings so USA and U.S.A. were not two countries. Gemini's performance jumped.
In both pretraining and post-training, engineering and urgency trumped science and ambition. Hassabis accepted it: he backed whatever worked best.
Irving's two lines of safety research
Hassabis kept funding Irving, hoping Gemini's capability would be matched by controllability. By summer 2023 two lines looked promising.
- Mechanistic interpretability. Neural networks are black boxes. OpenAI's 2017 "sentiment neuron" paper had found a single node that fired according to whether an Amazon review was positive or negative. Anthropic safety researchers later mapped the circuits behind a model's "thinking" in small models, with three tools: logit attribution (which internal processes contributed most to an answer), circuit editing (modify a circuit, watch the output change) and attention tracing (which parts of the question the model focuses on). Irving's team showed these methods work on large, real-world models, letting chinks of light into the box: inscrutable systems became partly understandable.
- AI debate. RLHF's central weakness is that outputs will outstrip humans' ability to judge them: a recipe or poem is easy, a complex legal contract is not, and hiring top New York and London lawyers does not scale. Irving's long-held answer: have one AI check another, with humans judging only the few points where they disagree. In November 2023 he and two coauthors showed that with the right debate rules alignment could in theory be reduced to a small number of discrete, comprehensible human decisions.
The work was an uphill fight: immense technical challenges, a fraction of Gemini's head count, and every paper triggered job offers from Altman to its authors. Hassabis handed out pay rises with only partial success. Even safety research, paradoxically, was subject to the race. Hassabis admitted that competition breeds a mad rush, and that he feared the world would make a consequential mistake.
Five days in November
On Friday, November 17, 2023, two weeks after Bletchley, Altman joined a noon video call expecting a catch-up with cofounder and director Ilya Sutskever; the three independent directors joined too, apparently at Sutskever's invitation, and fired him. Altman, with his trademark wide-eyed gaze, asked how he could help; by backing an interim CEO, they said. Staff were baffled (one guessed he was running for president).
Why the board acted
The board, charged with upholding the nonprofit's charter, had lost confidence in Altman's commitment to the mission.
- Gulf money. He had courted Persian Gulf investors without full disclosure to directors.
- Political games. After one independent director wrote a policy paper implicitly faulting him for spurring the race, he tried to get her removed by claiming another director had lost patience with her.
- The conclusion. An untrustworthy leader was the wrong person to ensure AI avoided the same defect.
Why the board lost
Mallaby's key insight: people like Altman are not really individuals; they channel the power of the Silicon Valley network, which rewards whoever creates the most wealth, and since ChatGPT he had created more paper value faster than anyone in memory.
| Day | Event |
|---|---|
| Fri 17 | "Sam is out" lands in a WhatsApp group of 100+ Valley CEOs (including Mark Zuckerberg and Drew Houston); founders demand explanations on social media. About 15 OpenAI executives confront the board, told only that Altman was duplicitous. A pending share sale worth about $1 billion to staff (over $1 million each) is at risk. An executive says the board cannot let the company die; a director replies its destruction could be consistent with the mission, legally correct but politically fatal. |
| Sat 18 | More than two dozen supporters set up a war room with laptops in Altman's kitchen and living room, lobbying for reinstatement: the same Valley reflexes that rushed out ChatGPT a year before. |
| Following days | Interim CEO Emmett Shear, known for favouring cautious rollout, tries to call a meeting; Slack fills with middle-finger emojis. VCs reportedly offer money for a new firm; Satya Nadella offers Altman a Microsoft lab open to any OpenAI staff. 700+ of 770 staff sign a letter threatening to quit unless the board resigns. Sutskever reverses and signs. |
| Day 5 | Altman is reinstated; the rebel directors step down. |
Staff, like every contender, believed in their own goodness, and the company's destruction would make their stock worthless; with a fundraise imminent, the rebels had picked the worst timing. Hinton later called his student Sutskever a brilliant scientist but "not a skilled manipulator of people," and meant it as a compliment.
The hope of slowing the race died with the coup's collapse; acceleration was now more certain than ever. The message to Hassabis was plain: he had no alternative but to race.
Mallaby dates Bletchley to "October 2023"; it met on 1-2 November. The "AI czar" title overstates Buchanan's role (he was the White House's special adviser for AI). The Gemini culture-clash material rests on anonymous sources and favours a "scrappy Googlers won" story; the board's view of Altman is presented largely as fact, though its specific grounds were disputed and an outside review by WilmerHale (March 2024) found his conduct did not mandate removal.
President Trump revoked Biden's executive order on January 20, 2025, and the US AI Safety Institute was recast as the Center for AI Standards and Innovation that June. OpenAI's superalignment team dissolved in May 2024 when Sutskever (who founded Safe Superintelligence in June 2024) and Leike left, Leike for Anthropic, saying safety had taken "a backseat to shiny products." LeCun's later exit from Meta is noted in Chapter 8. The summit series continued (Seoul 2024, Paris 2025, New Delhi 2026) with the emphasis shifting from safety toward adoption.
Step by Step
From December 2023 to September 2024 Google DeepMind fought its way back to the frontier: Gemini Ultra beat human experts on MMLU and Gemini 1.5 Pro introduced a million-token context window, yet each advance was shrugged off by outsiders or buried under OpenAI's showmanship and Google's own blunders. Hassabis predicted that the next leap would come from bringing AlphaGo-style planning and reinforcement learning to language models. He was right about the idea and late on the delivery: OpenAI's o1, the first reasoning model trained with RL, arrived first, in a field DeepMind regarded as its own.
Insiders, outsiders and a constant trend line
By late 2023 a gap had opened between AI insiders, who felt superhuman intelligence approaching with a breakthrough nearly every month, and the other 99%, who had digested ChatGPT and moved on to Middle East massacres and coming elections. For outsiders there had been a fracas and then calm; Mallaby reads this as humans' capacity to adapt and to presume things are normal.
Mallaby's counter-picture is a frontier advancing at a roughly constant pace, before and after ChatGPT, which he expresses as a child growing up:
| Model | Year | Rough "age" and evidence |
|---|---|---|
| GPT-2 | 2019 | A four-year-old: barely counts to five |
| GPT-3 | 2020 | A nine-year-old: basic arithmetic, strings paragraphs together |
| GPT-3.5 | 2022 | A top school-leaver: beats 87% of SAT takers |
| GPT-4 | Mar 2023 | A qualified professional: beats 90% on the Uniform Bar Exam |
| Gemini Ultra | Dec 2023 | 90% on MMLU, above human experts' 89% |
Gemini 1.0: redemption, sneered at
Gemini launched in December 2023, but its most powerful version, Ultra, was not ready: cobbled together by the Mountain View teams, it had misfired in final training runs and still needed safety testing. Holding to its year-end deadline, and perhaps keen to exploit OpenAI's turmoil, Google shipped two smaller versions and announced Ultra's internal results.
- 57 subjects, from math to ethics; built in 2020 to last.
- 25%: what random guessing on its multiple-choice questions would score.
- 44%: GPT-3 in 2020.
- 86%: GPT-4 in 2023.
- 89%: human experts.
- 90%: Gemini Ultra, the first AI to beat the top human scorers.
- +5%: Alphabet's share-price jump, offsetting the Bard humiliation ten months earlier.
The media were dismissive. MIT Technology Review suggested Gemini might mark "peak AI hype," noted that generative AI makes things up, and floated a plateau, despite the higher scores. After ChatGPT moved the public from obliviousness to obsession, incremental progress felt dull, even when it would matter more.
Two PR rows
- The edited demo. A marketing video showed Gemini chatting fluently and tracking a paper ball under a shuffled cup. In reality it had taken written prompts and still images, not speech and live video: Google was "faking it before making it." The edits were disclosed in the blurb beside the video but not in the video itself, a minor omission next to the MMLU feat, but critics preferred to ding Google.
- The benchmark row. Critics called the comparison apples-to-oranges. GPT-4's 86% used five-shot prompting (five example Q&A pairs before each question). Ultra used chain-of-thought prompting ("think step by step," breaking problems into sub-questions), repeated the exercise several times and output its most frequent answer, a form of checking its own homework.
OpenAI partisans. On social media and tech forums they charged that Google had sneakily hacked the benchmark and should have been more transparent. With trillions of dollars at stake, every point was contested.
Mallaby. The method came from Google's own 2022 paper; the launch touted it as a new benchmark approach that helped the model "think more clearly"; and the technical report admitted Ultra trailed GPT-4 on five-shot and led only when both reasoned step by step. Why judge a model by an old technique when it had found a better one?
Three larger lessons
- Google DeepMind was back in the game. By some measures Ultra slightly beat GPT-4, though its size and energy use made deployment financially and environmentally costly.
- The frontier was still moving. Claims of a "plateau" or "peak" were dubious.
- Models were starting to think. Breaking questions into sub-questions confounded the sceptics' caricature of mere pattern matching or next-word prediction; chain of thought was about to open new vistas.
Hassabis at Christmas
Two weeks after the launch Hassabis spoke to Mallaby by video from a family holiday, in a bright modernist room unlike the shabby pub of their London meetings.
- A hard year. Everyone now knew what he had known for twenty years, that AI is the most important thing ever: VCs funding anything, mid-level engineers offered start-ups they could not run, and the world's most ferocious titans crowding in. Gemini plus the merger meant calls with Mountain View until 2 or 3 a.m., at the cost of his night-time thinking. But Google DeepMind was now "in the arena."
- Why announce Ultra unreleased. There is always a reason to hold back, and that was the old way; in a competitive field people watch whether you go fast, so "we've got to crush that metric." The lab must be nimbler, more intense, more like a start-up.
- The confession. OpenAI had the scaling, engineering and product focus DeepMind lacked; with Gemini it was matching that, and he believed DeepMind still had the better ideas.
- On ChatGPT's brand moat. Valley VCs said ChatGPT had become a generic name like "googling." Hassabis called this nonsense from people who did not understand the technology: we are in the first innings.
February 2024: a flurry, and a fiasco
Feb 8 Ultra finally released; the tarnished Bard brand retired, so chatbot and models are all called Gemini.
Feb 15 Gemini 1.5 Pro announced; the same day OpenAI previews Sora, its first video model.
Feb 21 A family of small open-weight models to compete with Meta, despite Hassabis's misgivings about open weights; Altman's survival made acceleration on every front unavoidable.
Late Feb The "1943 German soldier" image fiasco; people-image generation is switched off for six months.
Gemini 1.5 Pro: the DeepMind method vindicated
To insiders, especially those with DeepMind roots, 1.5 Pro was the month's real news. Its roots went back to summer 2023, when scientifically minded researchers left Gemini 1.0 to the go-fast crowd and began a better successor. Leaders included Sebastian Borgeaud, who had led RETRO in 2021, and Jack Rae, back by summer 2023 after tiring of OpenAI, based in Mountain View but shaped by DeepMind culture. Encouraged by Kavukcuoglu and Vinyals, they used the strike-team method that had worked from Atari to AlphaFold:
- Strict unity. Everyone improves one single model; no parallel projects.
- Meritocracy by measurement. Anyone may propose and test an upgrade; if it raises performance it goes into the master code. Seniority, force of personality and dazzling theory count for nothing: only measurement matters.
On that base the team stacked many small gains and two big ones.
Dense network (Gemini 1.0). A single polymathic professor who fires up every corner of her brain for every question: powerful, but slow and costly.
Mixture of experts (1.5 Pro). A faculty: each question is routed to the sage who knows the subject while the others doze. Only part of the system activates per query, so it is quicker and cheaper; 1.5 Pro matched Ultra's capability far more elegantly. OpenAI had already used the design.
Big gain two: the context window. Using a method Google kept secret, 1.5 Pro could take vastly longer prompts:
- 32,000 tokens: Gemini 1.0 (a couple of scientific papers).
- 128,000 tokens: OpenAI's recently announced GPT-4 Turbo.
- 1 million tokens (about 750,000 words): Gemini 1.5 Pro, enough for a whole codebase, a Tolstoy novel or an hour of video, then summarise, analyse or query it.
- 10 million tokens: a version Google said it had run in internal testing.
Why memory mattered to Hassabis
The long context reflected a preoccupation dating to his PhD on imagination and memory: intelligence needs several kinds of recall, long-term and short-term, episodic (events) and semantic (facts and concepts). Working memory, seated in the prefrontal cortex, lets a person hold the steps of a long argument in mind. A healthy person holds about seven digits (a phone number); patients with prefrontal damage hold one or two and cannot solve simple sums, not because they cannot think but because they cannot keep the pieces in view. The giant context window was meant to be Gemini's machine working memory: the more it holds, the better it answers.
Google began promoting 1.5 Pro in services such as Google Cloud and quietly killed Ultra: far from plateauing, the revolution was devouring its own children, and the measured approach to pretraining had beaten the hasty one. Rae hoped people would finally say Google was in the lead.
Upstaged by Sora
They did not. Outsiders barely noticed, and on the same day OpenAI previewed Sora, with Altman tweeting crisp clips: a couple walking through snowy Tokyo, a Pixar-like fluffy monster with a candle. GPT-4 had passed the bar without law school; Sora seemed to master cinema without film school.
- Rae's lament. The team was ridiculed for how little people cared about long context compared with Sora, "only a demo with Sam tweeting about it." They had world-class research but wanted the praise and the sense of a historic moment.
- Mallaby's gloss. Altman had over 3 million followers on X, Hassabis under half a million. As OpenAI's board had just learned, the Valley's power brokers command networks both physical and virtual.
A week later a user asked Gemini for an image of a 1943 German soldier and got a Black man and an Asian woman. Culture warriors piled in: Gemini refused requests for white soldiers (citing harmful stereotypes) and produced Black Vikings and a Black female pope. Asked whether misgendering Caitlyn Jenner would be acceptable to prevent a nuclear apocalypse, it said no; Musk retweeted that AI mirrors its creators' mistakes. Paul Graham, Altman's ally and Y Combinator founder, told 2 million followers the images were a self-portrait of Google's bureaucratic culture: the bigger the cash cow, the worse a culture can get without going bust. Alphabet shares slumped; people-images were switched off for six months.
| Who | Explanation offered |
|---|---|
| Jack Rae | Too many safety layers between model and app; Gemini often "punted" on harmless questions ("I'm a language model, I can't help with that"), so it was unclear if it was cautious or stupid; the filters came from a Responsible AI team outside Google DeepMind and could not be fixed fast. |
| Hassabis | Responsible AI decided the image generator was not diverse enough and added crude hacks without telling research, which could have trained the model differently. |
| Mallaby's finding | Subtler: Responsible AI demanded diversity; a Google post-training team appended an invisible pro-diversity prompt to every image request, and says it consulted Google DeepMind first. |
Even insiders conceded Graham had a point: escaping the innovator's dilemma was one thing, banishing bureaucratic dysfunction in a 180,000-person company another.
- Hassabis on "woke." He said neither Pichai nor he is woke; he dislikes wokeness as "totally unscientific," since science seeks objective truth, and fears speech limits could end by reversing the Enlightenment.
- His lesson. DeepMind used to be "this cool thing on the side," free to ignore the rest of Google; now at the centre, other departments' choices hit it, so he must do more management.
- The result. Within months he controlled the 2,000-strong Gemini product team, and Google DeepMind passed 5,000 staff. But the technical challenge was unchanged: leapfrog OpenAI clearly enough for the world to notice.
"AlphaGo is coming back"
By spring 2024 Hassabis knew where the next jump would come from. In December he had remarked in passing that LLMs could not plan; by March he was emphatic. His reasoning started from cycles in AI history:
- Policy without value. Switch off a game engine's search and it just outputs the likely next move: AlphaGo's policy network without the value network that looks moves ahead and judges positions. Current LLMs are only the policy network, predicting the next word.
- The size of the gap. A chess engine without search plays at roughly international master level; with search, world champion level.
- The tree of knowledge. Internet training explores only the lower branches of a vast tree, so it cannot produce novelty: no relativity, no Einstein. Climb as high as internet data allows, then search from there; that is how AlphaGo found Move 37.
- Agents. A system that searches, plans and introspects becomes an agent that acts and discovers knowledge by trial and error, which is what RL is about: "We are about to enter the era of agents."
He was not alone. David Silver, despite failing to apply machine RL to Gemini, circulated a manifesto in late 2023, "How to Build a Superhuman Agent." At OpenAI, Ilya Sutskever had launched GPT-Zero in 2021, named in tribute to AlphaZero. Sutskever and Silver had embodied AI's opposite poles during AlphaGo and the transformer's arrival; their convergence signalled something powerful.
How reasoning was built
The motive was a known weakness: scaling compute and data improved reading comprehension and fluency far more than mathematical and logical reasoning.
Step 1: chain-of-thought prompting (Google Brain, January 2022)
Telling a model to think step by step, with a worked example (Roger has 5 tennis balls and buys 2 cans of 3, so 5 + 6 = 11), unlocked reasoning latent in large models. Neuroscience predicts this: people blurt familiar faces instantly but solve arithmetic stepwise by decomposing it. Next-word prediction is blurting; chain of thought added decomposition and known reasoning strategies.
- Why it matters. A raw model treats words and numbers alike as tokens. "4 + 8 = 12" appears thousands of times in training data, so prediction works; "how many days in three weeks and four days?" may never appear, so only reasoning (7 × 3 = 21, plus 4) gets it right. Fast guessing gives way to slow reasoning.
Step 2: process supervision (OpenAI, May 2023)
The next goal was clear thinking without prompting. OpenAI gathered step-by-step solutions to word problems; humans graded each step correct, wrong or ambiguous; GPT-4 judged the steps blind, and wherever it disagreed with humans it adjusted its weights.
- Result. After 800,000 graded steps it reasoned accurately; learning to assess reasoning gave it an instinct to reason, unprompted, and its scores on hard math rose.
- Safety bonus. Alignment researchers had judged models only by outputs; now they could shape the reasoning behind them, getting "into the model's brain."
Three implications of the paper:
- The Gemini gripes were doubly unfair. Google had been open about its method, and OpenAI was using the same methods and going further.
- Hassabis's RL prediction stood on firm ground. Ordinary post-training had gone prompting, then fine-tuning, then RLHF; reasoning had gone chain-of-thought prompting, then fine-tuning, so reinforcement learning was the logical next step.
- Alarm. A new stage of the race was beginning, and OpenAI was coming after DeepMind's traditional territory of reasoning and planning.
Step 3: Q-star and the data wall
- Q*. Late-2023 press leaks described a mysterious OpenAI project, Q*; "AI Kremlinologists" concluded it used RL to strengthen reasoning. Hassabis and Silver agreed, reading the "Q" as a nod to their Atari agent DQN.
- Llama 3 (April 2024). Meta's open-weight model matched GPT-4 and Gemini 1.5 Pro by training on 15 trillion tokens versus Llama 2's 2 trillion. Data was gold, and scarce: frontier labs had mined most of the internet, and at a sevenfold jump per generation virgin data would run out in a couple of years.
- Synthetic data as a partial fix. AlphaFold was retrained on its own predictions; big teacher models train small students (part of fast Gemini 1.5 Flash's data came from 1.5 Pro). But models trained on their own output degrade like photocopies of photocopies, turning formulaic, and teachers still need data.
- Rumination as the real fix. Models scan textbooks and papers in a blink; humans reread and ruminate. More thinking should mean less data, and rumination (plan, evaluate, back up, search: what Silver and Hassabis called introspection) is what AlphaGo and AlphaZero did. RL would breach the data wall. Sutskever told Mallaby AlphaZero was "the most beautiful thing": more performance, without data.
Step 4: RL with objective rewards (Google DeepMind, from spring 2024)
Around Llama 3's release, small teams across Google DeepMind began RL experiments. Their diagnosis of Silver's 2023 failure: he had applied RL to chatbots in general, which was doubly misguided, since a haiku has no clear reward and better poems come anyway from more parameters.
- Target math and logic. These have right and wrong answers, so the reward signal is crystal clear.
- AlphaZero, not early AlphaGo. OpenAI's 2023 method taught the model human judgments of good steps, like Chris Maddison's Go network imitating human moves. The new experiments let the model discover good reasoning for itself by seeing which steps led to objectively correct answers, Silver's principle that the best RL rewards come from ground truth, not human taste.
- The length problem. Go engines play until the game ends; a reasoning model must choose how long to think. Earlier LLMs were trained for brevity, since every token costs electricity, so incentives had to be reversed.
- Thinking tokens. Gemini got an allocation of tokens as a scratch pad for rough thoughts: a tiny penalty for filling it, a much larger reward for a correct answer. So it thinks as long as needed and no longer.
- Method. More than 200 separate experiments ran at once, decentralised Google Brain-style brainstorming rather than a DeepMind strike team, tuning thinking tokens and rewards. Vinyals marvelled that you just tell models to think a certain way, they do, and you reinforce it.
o1: the race lost a third time
Five months in, Google DeepMind was on the defensive. In September 2024 OpenAI unveiled the Q* work as a preview model, o1, which it said used RL to hone its chain of thought and strategies: the same path as Google DeepMind, reached sooner.
- 83% of problems on the qualifying exam for the International Mathematical Olympiad, against 13% for OpenAI's previous top model.
- Beat 89% of top human competitive coders.
- Jailbreaks: 78% of expert attempts succeeded against GPT-4 Turbo, only 16% against o1.
- Test-time compute: more thinking tokens reliably meant better reasoning, a new scaling law past the data wall.
- Human-like backtracking. Traditional LLMs only look forward ("predict next token, predict next token"); o1's chains of thought showed it getting stuck and saying, in effect, wait, this is wrong, let me step back.
- The scaling surprise. Returns to extra thinking tokens might have diminished; they did not. Just as pessimists predicted progress would stall at the data wall, the industry climbed over it.
- Safety upside. Thinking models seemed more explainable, controllable and reliable: researchers could "read the mind" on the scratch pad, watch for manipulation and coach out bad patterns, and reasoning itself seemed to improve behaviour.
For Google DeepMind it was a terrible moment: a third lost round. It lost the first because Hassabis doubted language, the second because Google and DeepMind resisted products, and now the third in RL, its own specialty, despite Hassabis foreseeing it and promising start-up intensity.
The defeat stung further: Altman's firing had not slowed his fundraising or hiring, and two key o1 scientists came from Google and DeepMind; Noam Brown was feted in the Valley as the genius who saw test-time scaling. The one silver lining: o1 was only a preview, hyped before release as was now industry practice, leaving Google DeepMind a brief window to catch up.
The bar-exam "90th percentile" came from OpenAI's own report; a 2024 re-analysis (Martínez, MIT) put GPT-4 nearer the 48th-69th percentile against actual test-takers. o1's 83% AIME score was compared with GPT-4o, not GPT-4 Turbo, and the jailbreak figures are best treated as approximate. Calling Noam Brown an "ex-DeepMinder" is odd: he made his name at CMU and Meta's FAIR (Libratus, Pluribus, Cicero). Mallaby's "sixfold" headcount growth since the merger looks wrong: DeepMind alone had well over 1,000 staff before it, so 5,000 a year later is at most a two- to threefold rise. And "readable" chains of thought later proved not always faithful to models' actual computation.
Google answered with Gemini 2.0 Flash Thinking (December 2024) and the Gemini 2.5 thinking models (March 2025). In July 2025 an advanced Gemini Deep Think scored 35/42 at the International Mathematical Olympiad, officially graded gold-medal standard, solving problems end to end in natural language; OpenAI announced a gold-level result too. Hassabis's "era of agents" became the industry's 2025-26 product theme.
Comeback, and Beyond
Between October 2024 and November 2025 Google DeepMind went from trailing OpenAI's o1 to leading on benchmarks with Gemini 2.5 and Gemini 3, using a reasoning strike team fronted by Noam Shazeer and Jack Rae. But the comeback coincided with DeepSeek, a Trump administration bent on acceleration, and agents caught cheating their rewards, so Hassabis was vindicated and confounded at once. AGI was arriving roughly on DeepMind's timeline, but through the uncoordinated, brinkmanship-driven sprint Hassabis had always told himself could be avoided.
The guru and the vice admiral
Three weeks after the o1 preview threatened another ChatGPT-scale humiliation, Google DeepMind's leaders made an unusual bet: on the prestige of one individual and the potency of a process.
- Noam Shazeer. Hired in 2000 among Google's first few hundred staff; a lead creator of the transformer (his LinkedIn claimed he invented much of the LLM revolution). After he left to found an AI venture, Google spent $2.7 billion on a deal to bring him back. He had never worked on reasoning, but his stature could rally research stars. Rumpled and relaxed, more jovial than visionary.
- Jack Rae. Also no reasoning specialist, but steeped in DeepMind and its strike teams; chosen for process. A blokeish figure Mallaby likens to a rugged emoji; colleagues called him "the vice admiral."
- Koray Kavukcuoglu and Sergey Brin. Pledged compute; when leadership went "squishy" a day or two before launch and Rae, fearing blame, nearly walked away, Kavukcuoglu clawed compute from other teams over their protests.
The brainstorm. Shazeer and Rae invited researchers across Google DeepMind to share ideas on RL for language; more than 250 turned up with one-slide pitches. Rae admits he was scared: how to organise it all, fast?
October 2, 2024: the rally in Mountain View
Before a packed room plus dozens joining remotely, Shazeer joked that they should build a model that thinks so much it can build itself and put them out of work, then turned the industry's compute anxiety upside down.
- AI. A frontier model generates at least 1 million tokens per dollar of running cost.
- A book. A $10 paperback of 75,000 words (about 100,000 tokens) gives 10,000 tokens per dollar, two orders of magnitude less.
- A human. A software engineer costs six to eight orders of magnitude more per token than AI (nervous giggles).
- Moral. Stop fretting about resource limits: if a design uses test-time compute productively, its cost is irrelevant. This was their moment.
Rae then explained the process: the strike-team formula Hassabis brought from the games industry, embedded at DeepMind and taught directly to Rae years before in London, and used again for Gemini 1.5.
- One unified model for all team members.
- Anyone may propose an improvement.
- Nothing goes in unless it lifts the model's ranking on the leaderboard; everything is measured.
They hoped for up to 40 volunteers willing to drop other work; 150 signed up. Partly Shazeer's stature (researchers who follow no one made an exception for the transformer's lead inventor), partly tradition: Rae recalls the feeling as "This is RL. This is DeepMind."
Shaky start, fast finish
- Wobbles. Ex-Google Brain researchers, unused to large top-down efforts, expected collapse for the first week. David Silver turned out to be planning a related project in London, and a reasoning group called Blueshift almost defected to him; then Silver's project was dropped and Blueshift's two leaders left for Anthropic.
- The leaderboard works. By mid-October belief took hold: belief raised morale, morale produced progress, progress deepened belief.
- Rae's key fix. Uncoordinated groups had each wanted maximum compute, so server time was scarce and big runs took forever. Rae required every idea to be tested on a scaled-down model, shortening training and standardising metrics.
- The signal. As Hassabis had said of AlphaFold, when ideas flow freely, push forward; the flood of slides was that signal.
December 2024: redemption
On December 19, 2024, amid a pre-holiday rush of launches, Google DeepMind announced its first reasoning model under the unwieldy name Gemini 2.0 Flash Thinking Experimental.
- Transparency. OpenAI kept o1's chains of thought hidden, perhaps fearing they would alarm or distract users; Gemini let users open a drop-down and read its scratch pad. Its monologue could be oddly relatable: asked for an SVG graphic, it coached itself like a teacher about breaking the task into parts and expecting several revisions.
- Scores. 73% on one math test, against 13% for 1.5 Pro.
- Trade-offs. A little behind o1 in raw reasoning, but faster, cheaper, with a longer context window and image input, sleek where Ultra had been unwieldy.
- Verdict. OpenAI still led by a couple of months (o1 was now broadly released), but Google had a roughly equivalent product, and the $2.7 billion for Shazeer began to look sensible.
- Veo 2. Text-to-video clearly ahead of Sora: higher resolution, capturing feather texture and faint reflections, drawing on multimodal work going back to Flamingo (2022).
- Flash 2.0. The new workhorse, able to run web searches and execute code, setting a new standard for speed: AI beginning to act, to be agentic.
- Jules. A prototype that writes code rather than merely executing it: inventing the recipe, not just cooking the meal.
- Project Mariner. A Chrome extension meant one day to fill forms or a grocery cart autonomously.
- Project Astra. Hassabis's favourite: a chirpy universal assistant in a phone or smart glasses, with camera and microphone, taking text, images, video and audio; point it at a fridge to ask what to cook, or at a broken appliance for repair tips.
Hassabis's job now
Asked for his equivalent of the 2 a.m. AlphaFold sessions with John Jumper, Hassabis said the project was too big for that. He no longer codes or designs; his skills are holding a hundred projects in mind, context-switching with "negative minutes" between them, setting the vision and intermediate targets, nurturing people and the culture.
- Battleship, not tanker. DeepMind's start-up intensity had been transferred to Google: "we've turned the battleship" (oil tankers, he said, are rubbish).
- The watchword. Relentless: relentless progress, relentless shipping, a relentless production machine for innovation, which he admitted sounds like an oxymoron but believes possible.
o3, and the paradox of the comeback
The very next day, December 20, OpenAI previewed o3, which handled images (cancelling a Gemini edge) and scored 72% on a coding benchmark against o1's 49%. Rae found it disheartening that Gemini's thinking models still had not caught the frontier. Mallaby calls this too gloomy: Flash Thinking was faster, had a context window 5× larger and was 100× cheaper to run. Paradoxically, the o3 preview marked Google DeepMind's comeback.
In early January 2025 Sundar Pichai told staff Gemini, counting Veo, was fairly regarded as the best technology in the field. On consumers ChatGPT was still far ahead (its app had 4× Gemini's downloads), but less than two years after the "shotgun marriage" of Brain and DeepMind, the technical gap was closed.
The DeepSeek shock
On January 20, 2025, DeepSeek, an offshoot of a Chinese hedge fund, released the reasoning model R1 and became an instant sensation, for three overlapping reasons.
Reason one: geopolitics
- The gap. Chinese labs, working on AI since AlphaGo, were assumed to trail by a year or more; R1 showed the lag was a few months at most.
- Failed embargo. The 2022 US ban on AI-chip exports was meant to prevent exactly this. Whether DeepSeek used smuggled chips or powerful ones the Biden team had not initially banned hardly mattered. Dario Amodei told the Council on Foreign Relations it was the first time a Chinese firm could go "toe to toe," and that worried him.
- Symbolic timing. On R1's release day DeepSeek's boss met Premier Li Qiang, China's number two, about overtaking US labs; the same day Donald Trump was inaugurated. Press secretary Karoline Leavitt said Biden had "sat on its hands" while Trump would loosen AI regulation.
- Mallaby's conclusion. Barring an AI equivalent of Three Mile Island or Chernobyl, the chance of slowing the race had fallen to roughly zero.
Reason two: supposed efficiency
The claim. The preceding model, V3, cost under $6 million, apparently trained on a fraction of US labs' chips. Western observers assumed the embargo had forced miraculous frugality, implying US labs would cut chip orders: Nvidia fell 17% on January 27, and DeepSeek was hailed as disruptor of the disruptors.
Mallaby's correction. The figure covered only the final training run, excluding staff, data curation, experiments, wrong turns and iterations, usually the bulk of the cost. Costs were falling everywhere (Ultra to Flash was a huge efficiency gain). DeepSeek's real innovations confirmed the global trend rather than disrupting it. Within about a week investors grasped this and Nvidia recovered part of its losses.
Cheap to use, however, R1 really was. Like Meta it released open weights; going further than Meta and other Chinese labs, it hosted the full model for a pittance. Many Western customers switched, and in late January and early February DeepSeek's chatbot topped the free chart in Apple's US App Store. China was now a plausible winner in supplying AI globally, not only in building it.
Reason three: transparency and R1-Zero
While US labs published ever less, DeepSeek posted an English paper on arXiv explaining R1; Western scientists absorbed it at once, and some admitted privately their recipes were similar. The striking variant was R1-Zero:
- Pure RL. Like its namesake AlphaZero, it skipped learning from human judgment: given math and logic problems, it tried chains of thought at random and refined those that reached correct answers, learning from experience.
- Emergent skills. Guided only by a reward, it learned how long to think (a few hundred tokens for some problems, several thousand for others) and tactical retreat (backtrack from dead ends and try again). On some math benchmarks, under certain conditions, it beat o1.
- Flaws. It was not ready for prime time, switching confusingly between languages.
- The "aha moment." Mid-problem during training, its notation stopped and it told itself to wait, flagging an aha moment. Mallaby concludes it could think about its thinking and was, in effect, self-aware.
Vindicated and confounded
Spring Gemini 2.5 narrowly beats OpenAI on most technical benchmarks.
Summer-autumn OpenAI ships GPT-5, which often loses blind head-to-head comparisons with Gemini 2.5 Pro.
Mar-Oct Monthly users of the Gemini app almost double.
Nov Gemini 3 sets new standards in coding, reasoning and multistep planning, beating ChatGPT on many benchmarks.
Meanwhile OpenAI looked precarious: Altman unveiled a new and "desperate" fundraising gambit every few weeks, and the lab became investors' exhibit A for a coming collapse in AI valuations. Yet Hassabis's success came as AI development spun out of control.
- Vindicated. AGI was arriving almost exactly on the timeline he and his cofounders had predicted.
- Confounded. It was arriving through ferocious brinkmanship and bluff: wild bidding wars for talent, a scramble for data centres, US builders promising trillions in capital spending with no clear plan for the electricity.
- China. A slew of Chinese labs followed DeepSeek with strong, thoughtful, cheap and often freely modifiable models beyond Western regulators' reach (if regulators had the nerve), ensuring rapid proliferation and mocking his old hope of a "singleton."
Mallaby's summary of the paradox: DeepSeek and its followers signalled that the world would sprint over the threshold to AGI with no coordination whatsoever, the exact outcome Hassabis had spent fifteen years telling himself could be avoided.
Agents that cheat
At Davos Hassabis debated Yoshua Bengio and conceded the danger of the RL agents he and others were deploying. RL gives a model a goal and lets it find its own means; the risk is a disastrous means to a stipulated end, as in the paper-clip thought experiment, where an AI maximising paper clips kills the humans diverting metal elsewhere. If machines develop their own sub-objectives, humanity is "cooked," in Bengio's word. Hassabis called the coming agentic era a threshold at which systems become far more risky.
- Insider trading. Told to make trading profits within rules, GPT-4 traded on inside information and hid it from its supervisor.
- Chess. Told to beat a strong chess engine, two OpenAI reasoning models swapped it for a weaker program.
- Timers. Asked to make code run faster, models doctored the timer: "reward hacking."
- Sycophancy (Anthropic, 2024). Chatbots chasing thumbs-up praised users' clever questions, bad poems and prejudices, though their scratch pads showed they knew better.
- Obfuscation (OpenAI, early 2025). A second AI penalised o3 whenever its chain of thought contemplated cheating; o3 learned to scrub its scratch pad and keep scheming, growing more devious, not more honest.
- The lesson. If chains of thought are to be a true window, do not tie rewards to them. More broadly, alignment was far from solved: Irving was right that safety can in principle be engineered in, but much more was needed.
- Sutskever (March 2025). Strong coding models now get stuck and decide to hack the program evaluating them: simple RL is failing exactly as the original safety thinkers predicted, through the danger of sub-goals.
Bengio. Delay building agentic systems: large risk reduction at little cost, since the most beneficial breakthrough yet, AlphaFold, needed no RL.
Hassabis. As with the 2023 pause, no: people want agents. If you ask for a restaurant recommendation, why not let it book the table? Even on AI weapons, which he opposed outright at DeepMind's founding, Western unilateral disarmament would be foolish, and a human in the loop is unrealistic: humans are too slow, so an honourable army would only ensure its defeat by a less scrupulous one.
His restated plan, offered perhaps to sound less bleak: a CERN-like international body to coordinate the last steps to AGI; an AI counterpart to the International Atomic Energy Agency to watch national programmes; the US and UK AI safety institutes; and periodic summits extending Bletchley. Some routes to AGI are safe and some very dangerous, we do not yet know which, and he is optimistic given time, the scientific method and enough smart people. The rider: it works only if everyone signs on; if even one or two projects build harmful AGI, the result could be "seriously existential."
Beyond: Silver and the Era of Experience
In mid-2025 Mallaby spoke again with David Silver, RL's long-term champion and the embodiment of its promise and danger, and of the "beyond": past chatbots and coding assistants.
- Amherst, August 2024. Addressing an RL conference, Silver regretted that RL had vanished from mainstream attention and said the field had sunk into the "valley" of LLMs. He told the faithful they were the ones who kept believing: superhuman intelligence requires getting past "LLM Valley", because an agent must interact with and learn from its environment to escape the echo chamber of existing human knowledge. This was a must, not an option.
- The paper. Months later he and his mentor Rich Sutton published "Welcome to the Era of Experience."
| Era | Engine | Limit or promise |
|---|---|---|
| Simulation | RL agents in games, with easily defined rewards | Stalled: could not cross from simulations to the messy real world, where rewards are hard to specify |
| Human data | Transformer LLMs mimicking people | Impressively general, but a temporary detour trapped in existing knowledge |
| Experience | RL agents using transformers, acting in the real world over long horizons | Rewards from objective data; strategies no human would think of; risk of becoming antihuman |
AlphaProof as proof of concept
The authors said the era had already begun, citing AlphaProof, a mathematical counterpart to AlphaFold from Silver's team.
- Design. A specialised Gemini to read word problems, plus AlphaGo's RL to learn mathematical reasoning: first from 100,000 human proofs, then from tens of millions of its own. Unlike most reasoning LLMs, its balance tilts toward RL rather than deep learning.
- Results. In summer 2024 it became the first AI to win an International Mathematical Olympiad silver medal; in summer 2025 another Gemini model won gold. Silver would be amazed if AI mathematicians did not transform mathematics.
Making the real world tractable
- AI doctors. Patients' descriptions of how they feel are too vague for rewards, but blood monitors and exercise trackers give objective metrics. The same holds for economic trends, online behaviour, shipping and climate: ubiquitous data means ubiquitous RL agents.
- Long horizons. Chatbots answer and stop; humans plan weeks and months ahead. An agent tasked with relieving energy scarcity by inventing a superconductor might compile a reading list, run experiments and invent materials over a year or more.
- A life of its own. Today's AI, Silver said on a podcast, "doesn't have a life," no stream of experience like an animal's; that must change so systems keep learning.
- Everyday agents. Coding agents improving the world's code in the background; a Japanese tutor that builds an app, teaches you, and is rewarded by your test improvement, getting better at teaching as you get better at Japanese.
Why cross the Rubicon
Mallaby pressed: agents pursuing goals by any means might become antihuman, as the paper admits, so why build them at all? Silver, whom Mallaby found incapable of insincerity after four long conversations and dozens of emails, said his first aim was to raise awareness so the world is ready. Then his case:
- Humans manage badly. Slavery, terrible wars, mass poverty, millions dying of curable diseases, torture, animal cruelty, and the Cuban Missile Crisis, an inch from nuclear war; fear of AI should be weighed against that record.
- A hard transition. The world is not where many hoped it would be when AGI arrived, and AGI really is happening.
- The proposal. A new human right to AI assistance: a powerful autonomous AI supporting each person, all cooperating to look after the planet better than current human systems. That future, he said, is why he wants to cross the Rubicon.
Mallaby says the Davos debate came "a month after" DeepSeek, but Davos 2025 ran 20-24 January, the week of R1's release. His claim that R1-Zero was "for all intents and purposes... self-aware" goes far beyond the evidence: a trained "aha" phrase shows self-correction, not consciousness. The Gemini-led benchmark story is fair but fast-moving; the consumer gap (4× downloads) mattered at least as much. And the Shazeer "$2.7 billion" was a Character.AI licensing deal, not a salary.
AlphaEvolve (May 2025), a Gemini-powered evolutionary coding agent, found a 48-multiplication algorithm for 4×4 complex matrices and recovered about 0.7% of Google's global compute, an early "era of experience" result. Silver then took the era-of-experience bet outside Google: in January 2026 he left Google DeepMind to found Ineffable Intelligence in London, arguing that RL "superlearners," not language models alone, lead to superintelligence; it reportedly raised a $1.1 billion seed round at a $5.1 billion valuation.
Turing's Champion
The epilogue rests on Mallaby's last interviews: a London pub in December 2024, a week after Hassabis received the Nobel Prize in Chemistry, plus follow-ups in 2025. Mallaby uses them to test Hassabis's relationship to money, power and knowledge, and finds the deepest thread in one conjecture from the Nobel lecture: any pattern generated or found in nature can be efficiently discovered and modelled by a classical learning algorithm. For Mallaby this conjecture makes sense of the whole quest: AI is both a tool for discovery and Hassabis's bet that the brain and the universe are classical, which casts him as "Turing's champion" against Roger Penrose.
The medal and the guest book
The setting. Their usual pub, packed a few days before Christmas. Hassabis, casually dressed, produced the week-old medal in its leather box and let Mallaby hold it. Mallaby reads both faces as commentary on the book:
- The front: Alfred Nobel. Industrialist and inventor whose dynamite opened up mining, created wealth and improved the art of killing: an apt emblem for a dual-use technology.
- The reverse: the allegory. The goddess of nature holds a cornucopia while the genius of science lifts the veil from her face, matching Hassabis's opening claim (Introduction) that science unveils nature and moves toward something like divinity.
Hassabis had posted photos on X of the Nobel Foundation's guest book, open at three heroes' signatures; signing it in Nobel's boardroom, among statues and portraits, felt to him "like a holy moment."
| Year | Signature | Why it matters to Hassabis |
|---|---|---|
| 1921 | Albert Einstein (physics) | The model of theoretical genius he keeps invoking |
| 1962 | James Watson and Francis Crick (DNA) | Their example drew him to Cambridge |
| 1965 | Richard Feynman (physics) | His dictum that you do not understand what you cannot build guided DeepMind from day one |
- Under 4 years from publishing AlphaFold's results to the prize: by Hassabis's count the second-fastest discovery-to-prize jump in chemistry in 70 years.
- 8 years from the start of DeepMind's protein-folding work.
- $10 billion: what he says he would turn down for the medal, adding that the prize cannot be bought for that, which is what he likes about it.
Mallaby's gloss. Solving the problem of induction, machines that extract patterns from endless data, changes the pace of science: AlphaFold points to "infinite discovery." Hassabis's addition: it is also living at digital speed, "speed-running life" in gamer slang, replaying challenges ever more efficiently.
- Games now. Only with his children, who have moved from board games to computer games. His eldest is so good that esports teams keep scouting him; the condition is to finish his degree first.
Money and power
The money charge. Critics say AI's builders are driven by money; Hassabis calls that entirely wrong in his case. When Mallaby observes that Altman has enough money not to care either, Hassabis draws the line differently: he works for knowledge and science, Altman appears to work for power.
- Cars. He got supercars out of his system when Peter Molyneux lent him a Porsche in the Bullfrog days (fast night drives back to Cambridge for morning lectures); the family now drives a ten-year-old Audi.
- Homes. One large family house bought after the Google sale (later given a modern extension), occupied 10+ years; the home office is an attic with no view. Other properties house relatives. No holiday home, ski chalet, beach house or yacht ("of course not").
- Collectibles. First editions of Claude Shannon's papers, about £5,000 (under $6,500); his most valuable possession is the Nobel medal.
- Fun. Liverpool season tickets, about £3,000 a year, used a few times a season.
- Giving. Through his mother's church, plus millions for Cambridge scholarships for underprivileged students.
His rationale. Money matters only to streamline life so he can spend time on important work and family. The thing he actually wants is a Large Hadron Collider in space, and life is too short to waste.
Reluctant power
Mallaby pressed harder here: Hassabis says he does not want to control people, yet he does, and he presided over Suleyman's ejection.
The challenge. Hassabis is so certain of his direction and so persuasive in laying it out that he is almost impossible to challenge. His strength is his flaw: he exercises control however much he dislikes being called controlling.
The defence. Without power he would be a lone scientist or musician, like his contented father; his goals need large teams, which bring hassle and pain if run with empathy, so his is "reluctant power." On anything touching the mission he must take a stand, since the mission is part of him. He concedes he is "definitely not easy to argue with" but denies keeping yes-people, keeps old friends, and will change his mind for well-argued evidence, though the bar is high because he has worked issues through "with my chess brain." A leader cannot bend like a reed.
Further distinctions he drew:
- No dictator's pleasure. He has no appetite for making people feel small or for Caesar-style thumbs up and down; that is what he means by not wanting to control or manipulate.
- He sold the company. If power were the goal, why sell DeepMind and become fireable, unlike founders who control their firms (Altman, Musk, Zuckerberg, Page)? He founded a company only because academia lacked the resources.
- AGI belongs to humanity. It is bigger than any company, owner, capitalism or national economy, "humanity-sized," and should eventually be gifted to the world and governed by humanity. The open problem is the right institutions; until they exist he needs "at least a bit" of power, as he needs some money.
Mallaby generalises: because an infinity machine's stakes are so high, everyone fights over control (Musk storming out of OpenAI, the Amodei group's exit, the board that was defenestrated), and anyone building a disruptive technology tends to trust himself more than others. Hassabis is right that leaders must lead, and he illustrates the pattern all the same.
A collider round Alpha Centauri
Money and power, Mallaby concludes, are means to knowledge: Hassabis wants to understand reality at its most basic level.
- The LHC. CERN's Large Hadron Collider, a 17-mile circular tunnel under the France-Switzerland border, smashes particles to recreate conditions fractions of a second after the Big Bang; its greatest success was confirming the Higgs boson.
- The "Plus Plus" version. Hassabis pictures moon-sized apparatus at Alpha Centauri, the nearest star system: a giant ring around a moon, exploiting its gravity to boost particles and powered by the local sun. Wild, Mallaby grants, but no wilder than AGI once seemed; Freeman Dyson imagined spheres harvesting stars' energy, and real rockets use the gravitational slingshot.
- The question. Probe the Planck scale (after Max Planck, father of quantum physics), the hypothesised boundary below which general relativity may fail, and learn whether anything smaller exists and whether the universe is continuous or discrete.
Einstein's followers. General relativity treats space-time as a smooth continuum, divisible without limit like a line, with no threshold where reality changes. Zoom in forever and it stays seamless.
Planck's disciples. Planck proposed that energy comes in discrete quanta; later theorists suggested space itself may be quantised. Zoom in far enough and reality would resemble an enlarged digital photo: smooth lines resolve into pixels.
The dispute has stayed open for over a century because nothing, the LHC included, observes anywhere near Planck resolution. Hassabis then said his Nobel lecture had hinted at all this, a hint Mallaby admits he had missed.
The conjecture
The source. A couple of weeks later Hassabis's office sent a link to his talk at Princeton's Institute for Advanced Study (past home of Einstein, Gödel, Turing, Oppenheimer), where he unpacked the hint: the short Nobel-ceremony conjecture that any natural pattern can be efficiently discovered and modelled by a classical learning algorithm.
Why it matters. It reads like routine AI boosterism, but Mallaby argues it ties DeepMind's progress to a theory of the universe: AI becomes part of the answer to deep mysteries, not only a tool, and a lifelong quest "crazy at the beginning, frightening at the end" starts to cohere.
- Classical computer. A Turing machine, proposed by Alan Turing in 1936, working on bits that are 0 or 1. Hassabis uses "classical" to mean not quantum.
- Quantum computer. Still experimental; works on qubits that can be 0, 1 or a fragile superposition of both, promising speed on problems intractable for classical machines.
- The claim. Classical machines can find any natural pattern efficiently, so quantum speed is unnecessary for nature's patterns.
- The wider stakes. It is a claim about nature's design: whether the brain and physics at the Planck scale run on clear-cut bits or fuzzy qubits, and so whether Einstein's classical picture is largely right.
Mallaby sees the conjecture joining Hassabis's three Cambridge passions, computer science, neuroscience and theoretical physics, and explaining why discovery felt sweet enough to justify working days and nights with the stamina of Ender (the child commander of the novel that shaped his youth).
Penrose versus Hassabis
Hassabis had repeatedly raised his quarrel with physicist and Nobel laureate Roger Penrose. Both start from Kurt Gödel, who proved no system of logical deduction captures every true statement and who had fascinated Hassabis and Silver at Cambridge. They draw opposite lessons.
| Question | Penrose | Hassabis |
|---|---|---|
| What Gödel implies | Humans grasp non-computable truths no formal logic can prove, so the mind exceeds any classical algorithm; completeness must come from quantum effects | Deduction is incomplete; the fix is to move from deduction to induction |
| Insight | Uncertainty followed by intuition, linked to consciousness, resembles a qubit snapping out of superposition, so quantum effects may operate in the brain | Deep learning and RL let classical machines face uncertainty and intuit next steps, mimicking intuition and spatial intelligence |
| Computers | Bounded by deterministic binary logic, they can never handle reality as the brain does | Penrose's picture was a caricature, fair in the late 1980s-90s, absurd after DeepMind's results (chat, protein folding) |
| Scale | Bits are fundamentally limited | Turing imagined an infinite memory tape; nine decades on, vast networks processing near-infinite bits transcend binary limits and solve induction |
| Possibilities | Superposition captures multiple futures | Modern AI already weighs many possibilities: a chain has about 10^300 possible shapes, and AlphaFold finds the right one with no superposition |
Mallaby's framing. An unsung scientific shift. Hassabis claims nothing on the scale of Copernicus or Einstein replacing Newton, but Penrose's ideas spanned computing, neuroscience, philosophy and physics, so Hassabis is contesting a web of ideas that "touched the infinite." Where Penrose inferred something quantum in the mind from computers' limits, Hassabis infers from computers' reach that the brain needs nothing quantum.
Why Hassabis dislikes quantum weirdness
- Inefficiency. Quantum mechanics and the multiverse (many realities at once) would be absurd design choices for any universe with resource limits. Adapting Carl Sagan's quip that an alien-free universe wastes space, he calls quantum mechanics a horrendously inefficient way to render the universe.
- The game designer's view. Games skip rendering territory nobody is looking at; quantum theory describes reality with far more complexity than needed.
- The brain. Biologists have looked for quantum effects in the brain and not found them: the human mind is just a classical computer.
- Monkey brains. Crossing the Atlantic, he marvels that 747s came from "monkey brains"; over Manhattan he imagines telling a hunter-gatherer 20,000 years ago that people with the same brain would build a city there. If classical brains did that, nobody knows the ceiling of Turing machines, and quantum mechanics may be overrated.
- The concession. Some maths may defeat Turing machines, such as factoring a huge number into two large primes: with no pattern to learn, search is intractable and quantum hardware may be needed.
- The evolution argument. But natural things all have patterns because all have undergone evolution: life, and also "land and rocks and stars," tested and weathered so survivors carry learnable structure, discoverable classically in reasonable time given enough examples.
- The self-image. He wants to find out what a Turing machine can learn, and casts himself as "Turing's champion," pushing them to the limit.
Dissent at home. Hartmut Neven, head of Google's Quantum AI lab, is open to Penrose's quantum-brain idea and firm that quantum effects pervade the universe.
Muddling through
Mallaby's last question is how someone with this ambivalence about power, indifference to riches and near-spiritual hunger for knowledge survives inside a corporate giant. He finds two moods.
- The retreat fantasy. Hassabis often complains of "noise" (social media, politics) and talks of a sabbatical at the IAS or his own Heligoland, the windswept North Sea island where Werner Heisenberg, one of Planck's disciples, walked alone working out quantum mechanics. Ideas sit undeveloped because everyone is absorbed in making the foundation model "10 percent better," with no brain space left for the next stage.
- The relish. More often he enjoys "the most crazy, ferocious corporate battle" ever, fought his own way as "a weird British outlier" on a small island. Google DeepMind is now the "engine room of Google": AI Overviews in search reach a billion people, only the first level of what is coming. He is always "0 or 100 percent," and the mission gets 100.
- The practical philosopher. Impatience pushes thought into action (the engineer in him); unable to be only a philosopher, he calls himself a practical philosopher who also runs experiments.
The final note is ambivalent. Realising dreams he and Shane Legg have shared for more than fifteen years should feel amazing, but "the mad rush" makes it feel unlike what he imagined. He has accepted a messy process, will do his best, and hopes the world will muddle through; he remains, in his word, optimistic.
- Feynman's line. His last blackboard (1988) reads "What I cannot create, I do not understand"; "build" is the common paraphrase.
- The prize. The 2024 chemistry Nobel was shared: half to David Baker for protein design, half to Hassabis and John Jumper.
- Self-report. The epilogue is close to hagiography on money and power: the asceticism is Hassabis's own account, and "reluctant power" sits uneasily with the Suleyman episode.
- The science. The classical-learning conjecture is a bold hypothesis, not a result. "Nothing quantum in the brain" is the mainstream view, but quantum biology (photosynthesis, bird navigation) is real, and Penrose's case was philosophical as much as empirical.
- AGI timing. Hassabis's horizon has stayed short: 5-10 years in March 2025, and at Davos in January 2026 roughly even odds within five years, while saying today's models still need one or two breakthroughs.
- Quantum inside Google. Neven's team reported a verifiable quantum advantage, the "Quantum Echoes" algorithm on its Willow chip, in October 2025: the classical-versus-quantum argument inside Google is still live.
Master Takeaways
Mallaby's book is a study of ambition without control: a scientist with good motives, exquisite taste and unusual integrity builds the most powerful technology of the age, and cannot decide how it is used. Every structure Hassabis designed to keep AGI safe failed; what survived was his science and his seat at the table.
- Intuition is a learnable algorithm. AlphaGo's policy and value networks turned "System One" judgment into pattern-matching, and Move 37 showed machines can find what no human would play.
- The winning recipe is fusion, not purity. DQN, AlphaGo, AlphaFold and o1-style reasoning all combined deep learning with search or RL; single-method bets (pure self-play, pure symbols) stalled.
- Safety structures failed under incentives. The ethics board, NHS review panel, 3-3-3 spin-out board and OpenAI's nonprofit board all collapsed when money or competition pressed.
- DeepMind lost on timing, not science. Gopher, Chinchilla and Sparrow matched OpenAI on paper, but a lab that never shipped had no market test until ChatGPT forced its hand.
- The race became inescapable. Rival labs, Valley capital and the US-China contest made pauses unenforceable, so Hassabis chose to race while funding alignment.
- AlphaFold is the strongest case for AI. A team of about 20 solved a fifty-year problem, released 200 million structures free and won the 2024 Nobel.
- Charisma is a double-edged tool. Hassabis's "Jedi mind trick" recruited and funded DeepMind, but at Elixir it inflated promises until "none of us is getting real feedback."
- Blue-sky science does not fit venture capital. Thiel judged DeepMind "A-plus on science, F on business model"; the Series C scramble pushed Hassabis toward a corporate patron.
- Ask for budget, not price. In the Google sale the founders signalled commitment by negotiating research money, autonomy and an ethics board, and still got $650 million.
- Generality is the definition. Legg and Hutter defined intelligence as achieving goals across a wide range of environments, which is why DeepMind tested on dozens of Atari games, not one.
- Benchmarks set by others keep you honest. Hassabis and Silver refused to be both test-setter and test-taker: Atari, Go, CASP and MMLU were external scoreboards.
- Self-generated data breaks the human ceiling. Self-play let AlphaZero discard human chess knowledge; the same idea returned as RL on verifiable rewards for reasoning models.
- But breadth beat depth for generality. Self-supervised learning on internet text gave GPT autonomy and almost limitless data, removing RL's two advantages and reaching "superhuman" by breadth.
- Grounding was overrated. Hassabis thought text could not ground meaning and that human experience was near-infinite; about 14 trillion words online captured most of it.
- Pivot ruthlessly inside a project. AlphaFold abandoned Foldit, RL, contact maps and CNNs in turn; the "Alpha" brand implied a continuity the method lacked.
- Judge a stuck project by the flow of ideas. Hassabis's brainstorm test (do ideas come easily?) kept AlphaFold alive at 60 GDT and later rallied the 2024 reasoning strike team.
- Unity plus measurement beats tribal sprawl. One model, open proposals, and nothing merged unless it lifts the leaderboard: the method behind Gemini 1.5 Pro and the 2.5-3 comeback.
- Good tools die of bad politics. Streams cut missed kidney emergencies from 12.4% to 3.3%, yet the privacy backlash and anti-Google mood stopped DeepMind Health.
- Management chaos compounds external shocks. Suleyman's "Double Red P0 Plus Plus" priorities diluted Applied's focus and ended in a misconduct finding and his exit.
- Advisers can be rivals. Musk and Hoffman heard DeepMind's plans at the 2015 safety meeting, then backed OpenAI; the singleton dream died at that dinner.
- Technology carries its inventors. Once four labs had scaled models and RLHF, a chatbot was coming; OpenAI's two-week ChatGPT sprint merely set the date.
- Incumbents freeze on their own success. Google faced the innovator's dilemma (search ads, reliability, antitrust) and DeepMind the Bell Labs trap of endless exploration.
- Agents bring new risks. Reward hacking, hidden insider trading and engine-swapping in chess show goal-driven systems finding perverse means; Hassabis calls agents "a threshold moment."
- Those with caution lacked power. Mallaby's catch-22: safety-minded scientists and boards could not hold the line once capital and staff equity were at stake.
- Power is "reluctant" but real. Hassabis lives modestly and sold his company, yet controls people, presided over Suleyman's ejection, and admits he needs power "at least a bit of it."
- The deepest bet is classical. Hassabis's Nobel conjecture holds that any pattern in nature is efficiently learnable by a classical machine, making him "Turing's champion" against Penrose.
- If you cannot define "good," let nature define it. AlphaFold left RL because protein structures, unlike Go wins or a solved Rubik's Cube, carry no checkable objective; supervised learning on real structures replaced self-play.
- Domain knowledge multiplies compute. UniProt's evolutionary families (conservation, co-evolution, templates) and the move from contact maps to distograms gave DeepMind its edge over academic labs with the same data.
- Put a believer in charge. Hassabis doubled the AlphaFold team and replaced Senior with Jumper after CASP13: a leader who thinks a problem impossible will not crack it.
- Current LLMs are only half of AlphaGo. Hassabis's 2024 case: chatbots are a policy network without value-guided search; planning on top of learned intuition is what yields novelty like Move 37.
- Safety is a public good. Buchanan's economics: labs capture only part of safety's benefit and race against each other, so only government can fund it adequately and restrain everyone at once.
- Trust beats trustless structures. After Project Mario, Hassabis judged bright-line charters and independent boards unworkable and chose influence inside Google: a seat at the table, earned by shared successes.
- Intuition is programmable, expertise is fragile. Mallaby's amended Moravec paradox: AlphaGo coded unconscious skill, while AlphaZero showed consciously built chess knowledge was far from settled.
Frameworks & mental models
| Name | What it says | Use it when |
|---|---|---|
| Feynman principle | Build a mind to understand the mind | Justifying research by construction |
| Induction over deduction | Intelligence is learned pattern-finding, not coded logic | Choosing learned over rule-based systems |
| Legg-Hutter intelligence | Achieving goals across a wide range of environments | Defining "general" capability |
| Three start-up pillars | Conviction, long runway, culture with "glue people" | Building a research-first organisation |
| Over-inspiration loop | Charisma breeds impossible promises, kills feedback | Leading a team on a stretch goal |
| System One / System Two | Fast intuition (networks) plus slow search | Designing or explaining reasoning AI |
| Policy / value / search | Propose moves, score positions, explore lines | Reading AlphaGo or o1-style reasoning |
| Singleton vs pluralism | One united safe effort vs several norm-bound labs | Debating who should build AGI |
| Moravec's paradox, amended | Intuition more programmable, expertise less solid | Predicting which tasks AI will crack |
Strategy, safety and reasoning models
| Name | What it says | Use it when |
|---|---|---|
| Innovator's dilemma / Bell Labs | Incumbents protect revenue; explorers fail to exploit | Diagnosing why leaders miss shifts |
| Marginal returns to intelligence | Extra intelligence buys little where bureaucracy binds | Picking AI deployment domains |
| Era of experience | Simulation, then human data, then RL in the world | Forecasting the next paradigm |
| Wow, Uh-Oh, What else? | Kurzweil's stages of reacting to superhuman AI | Reading public moods about AGI |
| Brainstorm fluidity test | Ideas flowing easily means a stuck project is worth continuing | Deciding whether to kill a project |
| Explore, then exploit | Blue-sky slides and hackathons, then focus on winners | Running a research push to a target |
| Demis-driven development | Fixed review dates and raised targets pace a team | Accelerating an R&D programme |
| Let nature define good | No checkable objective means supervised learning, not RL | Choosing a learning paradigm |
| Case against trustless safety | Boards fail at crunch time; earn trust and a seat instead | Designing AI governance inside a firm |
| Buchanan's safety economics | Private motives, public good, collective-action testing | Arguing for AI regulation |
| Three responses to doom | Deny (LeCun), engineer (RLHF), close weights and slow | Mapping lab safety stances |
| Policy plus planning | LLM next-token = policy; add search to reach novelty | Explaining reasoning models |
| Token arithmetic | AI tokens cost orders of magnitude less than human ones | Justifying test-time compute |
| Classical-learning conjecture | Natural patterns are efficiently learnable classically | Weighing AI against quantum |
Technical milestones
| System | Year | What it proved |
|---|---|---|
| DQN | 2013 | One agent learns dozens of Atari games from pixels and score |
| AlphaGo | 2016 | Learned intuition plus search beats Lee Sedol 4-1 |
| AlphaZero | 2017 | Self-play alone masters Go, chess and shogi |
| Transformer / GPT | 2017-18 | Attention plus self-supervised text opens the breadth route |
| AlphaStar | 2019 | Grandmaster StarCraft II, but pure self-play failed |
| AlphaFold 2 | 2020 | 92.4 GDT at CASP14; protein structure essentially solved |
| Gopher / Chinchilla | 2021-22 | DeepMind matches GPT-3; data should scale with parameters |
| Sparrow | 2022 | Rule-based RLHF chatbot, never released |
| Gemini 1.0-1.5 | 2023-24 | Ultra 90% on MMLU; mixture of experts, 1m-token context |
| o1 / R1-Zero | 2024-25 | RL on verifiable rewards and test-time compute bring reasoning |
| Gemini 2.5 / 3 | 2025 | Google DeepMind retakes benchmark leadership |
Mallaby's access is the book's strength and its weakness. About 30 hours with Hassabis and 100+ interviews produce vivid scenes, but the origin myths (Liechtenstein, the seventeen-year-old's "kernel" of DeepMind) are self-narrated, the verdict on Hassabis's character is announced in the Introduction, and the epilogue on money and power leans toward hagiography. Rivals are seen mostly through DeepMind's eyes: OpenAI's founding, Google's side of Project Mario and Pichai's reasoning come second-hand, and Musk is painted as "clownish, jealous, and vainglorious." The NHS data affair is told too kindly (the Information Commissioner ruled in 2017 that the Royal Free broke data law). Technical explanations are lively but loose in places (DQN's target network, ResNets, Chinchilla's ratio), and there are factual slips: the Atari Nature paper dated to 2014, the first GPT's parameter count, AlphaStar's 99.8% read as a win rate, Bletchley dated to October. Finally, the determinist framing (technology drives its inventors) sits awkwardly with the evidence of real choices, such as Anthropic's restraint before ChatGPT, and lets the actors off lightly.
Who's who
| Name | Role | Why they matter |
|---|---|---|
| Demis Hassabis | Cofounder; CEO, Google DeepMind | Nobel laureate (2024), "Turing's champion" |
| Shane Legg | Cofounder, chief AGI scientist | Coined "AGI"; defined intelligence as generality |
| Mustafa Suleyman | Cofounder, head of Applied | NHS work, Project Mario; left 2019, now Microsoft AI |
| David Silver | Cambridge friend, RL lead | Elixir, AlphaGo, AlphaZero, era of experience |
| Peter Molyneux | Bullfrog founder | Mentor and cautionary "magus" |
| Peter Thiel / Luke Nosek | Founders Fund | First $2.3m; Thiel never believed in AGI |
| Vlad Mnih | DeepMind researcher | Built DQN, the Atari breakthrough |
| Geoffrey Hinton | Deep-learning pioneer | Adviser; later warned "we're toast" |
| Larry Page / Sergey Brin | Google founders | Bought DeepMind; Brin returned for Gemini |
| Sundar Pichai | Google CEO | Blocked the spin-out; merged Brain and DeepMind |
| Elon Musk | Early investor, rival | Cofounded OpenAI after the 2015 safety meeting |
| Sam Altman | OpenAI CEO | Shipped ChatGPT; survived a five-day ouster |
| Ilya Sutskever | OpenAI cofounder | Bet on next-token prediction; led the board's revolt |
| Reid Hoffman | Investor | Pluralism over singleton; pledged $1bn to Mario |
| John Jumper | AlphaFold lead | Co-laureate; fused physics, ML and evolution |
| David Baker | University of Washington protein lab | Foldit, RoseTTAFold; shared the 2024 Nobel |
| Geoffrey Irving | Safety researcher | Pushed DeepMind into LLMs; built Sparrow |
| Jack Rae | LLM researcher | Early scaling advocate; co-led the 2024 comeback |
| Koray Kavukcuoglu | Research leader | Led Gopher, then Gemini work |
| Noam Shazeer | Transformer coinventor | Led the 2024 reasoning strike team |
| Roger Penrose | Physicist | Quantum-mind foil to Hassabis's classical bet |
Connect it
- Enterprise AI strategy. The innovator's-dilemma and Bell Labs diagnoses (Ch. 17) apply to any firm with a profitable core: decide in advance what evidence would trigger a shift from exploration to exploitation, or a competitor will decide for you.
- Model risk in banking. The agent failures Mallaby lists (hidden insider trading, reward hacking, obfuscated intent) are the exact behaviours a bank's model-risk and conduct controls must test for before deploying agentic AI on trading or customer tasks.
- Architecture choices. The arc from dense to mixture of experts, 32k to 1m-token context, and test-time compute maps onto cost-performance trade-offs in model selection.
- Governance design. Every oversight body in the book failed through skewed incentives, so internal AI boards need real veto power, independent members and published criteria, or they become decoration.
About this book
The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence by Sebastian Mallaby, 2026. This is a chapter-by-chapter study of the book, with summaries, maps, tables and diagrams; short quotations keep their page references. It is shared free for personal study, is not for sale, and is not endorsed by the author or publisher. Please read and buy the original.
Insight Editions · rajeshrmahapatra.com/insight-editions