Insight Editions · No. 277
The Thinking Machine
Jensen Huang, Nvidia, and the World's Most Coveted Microchip
Stephen Witt
Stephen Witt traces how Jensen Huang built Nvidia from a near-bankrupt gaming-chip start-up into the company that powers modern AI, through a decade-long bet on parallel computing.
A study guide to The Thinking Machine by Stephen Witt. Shared free for personal reading; not for sale and not endorsed by the authors or publishers. Please read and buy the original.
The Book on One Page
Witt tells how Jensen Huang turned a 1993 gaming-chip start-up into the company whose hardware runs almost all of modern AI, worth over $3 trillion by 2024. Nvidia won by marrying two technologies the industry had written off, parallel computing and neural networks, and by holding a decade-long, money-losing bet on CUDA until AlexNet proved it right. Around that bet Witt sets a portrait of a brilliant, volatile, fearful CEO who refuses to think about AI's dangers, and admits that he himself has "The Fear."
The argument in brief
The man (Intro, Ch. 1-2). A Taiwanese boy sent alone at ten to a Kentucky reform school, then shaped by ping-pong, Denny's and a decade designing chips at AMD and LSI Logic, Huang reasons from first principles, works without limit and thrives under adversity. Witt's organising theme is amplification: the same few precepts, diligence, courage and mastery of fundamentals, applied at ever larger scale.
Near-death and the parallel trick (Ch. 3-8). The exotic NV1 bet against triangles and was stranded by Microsoft's DirectX; with thirty days of cash, Nvidia skipped prototypes, shipped the copycat Riva 128 from an emulator, and Huang took full control from cofounder Curtis Priem. Armed with Christensen's disruption theory and TSMC's foundry, Nvidia split shading into simultaneous pipelines to please Quake's John Carmack: parallel computing on one chip. It coined (or popularised) the GPU, crushed 3dfx, survived a 90% stock collapse and was carried by hardcore gamers caught in a deliberately addictive compulsion loop.
The second outcast (Ch. 6). Neural nets, exiled since 1969, taught themselves backgammon in the 1990s (TD-Gammon, Jellyfish) but stalled for lack of data and compute; training was costly, inference cheap.
The bet (Ch. 9-10). Persuaded by John Nickolls that Dennard scaling would end and serial CPUs stall around 2005, Huang turned the GPU into a cheap supercomputer with CUDA (2006), free on every card, for a "zero-billion-dollar market." Downloads fell, Bumpgate brought a loss, the stock went nowhere for a decade and Starboard Value circled; he held on, guided by "resonance" with neglected scientists. Witt calls it the bet that made Jensen Jensen.
The convergence (Ch. 11-14). In 2012 AlexNet, trained on two $500 GeForce cards, won ImageNet; GPUs trained nets hundreds of times faster than CPUs. Catanzaro's cuDNN wired nets to CUDA, Huang declared an "O.I.A.L.O." and remade Nvidia as an AI company, Google ordered 40,000 GPUs, Huang hand-delivered the first DGX-1 to OpenAI while Musk and Bostrom warned of extinction, and in 2017 Nvidia hit $100 billion largely unnoticed.
Scale and the siphon (Ch. 15-17). Google's transformer (2017) suited GPUs and grew smarter with size; Google failed to ship it and OpenAI scaled it into GPT and ChatGPT. Nvidia's moat was software: of a 1,000-fold speedup in a decade, only 2.5× came from transistors. After ChatGPT all AI money, talent and innovation flowed through one corporate siphon.
Limits and fears (Ch. 18-23). Power, not compute, became the bottleneck; disappointment, not competition, the main risk; Taiwan an unhedged exposure; the $3 trillion valuation a bet on one man with 55 direct reports and no successor. Bengio, Hinton and Sutskever feared extinction and were sidelined by capital, not refuted. Shown Arthur C. Clarke, Huang exploded ("We are serious people, doing serious work"), which Witt reads as the key to both his success and his blind spot.
Chapter map
| Ch | Title | One-line payload |
|---|---|---|
| Intro | Introduction | Two bets on failed technologies built a $3T firm |
| 1 | The Bridge | Reform school to Oregon State: adversity sharpens him |
| 2 | Large-Scale Integration | AMD and LSI; ambition hidden behind modesty |
| 3 | New Venture | 1993 founding; NV1 bets against triangles and fails |
| 4 | Thirty Days | Emulator, no prototypes, Riva 128; Huang takes control |
| 5 | Going Parallel | Christensen, TSMC, twin pipelines for Quake |
| 6 | Jellyfish | Self-taught backgammon nets hit a data wall |
| 7 | Deathmatch | IPO, the "GPU," 3dfx destroyed; not gambling is riskiest |
| 8 | The Compulsion Loop | 90% crash; addicted gamers rescue the firm |
| 9 | CUDA | Nickolls's forecast; free supercomputing; Bumpgate |
| 10 | Resonance | Doubling down on CUDA against Wall Street |
| 11 | AlexNet | Two GeForce cards win ImageNet; $44m Google auction |
| 12 | O.I.A.L.O. | cuDNN; Nvidia reorganises as an AI company |
| 13 | Superintelligence | Cloud demand, Musk's fears, DGX-1 to OpenAI |
| 14 | The Good Year | 2017: Switch, Nobels, crypto, Endeavor, $100bn |
| 15 | The Transformer | Google invents it; OpenAI scales it into GPT |
| 16 | Hyperscale | Mellanox, ARM bid, the software moat |
| 17 | Money | ChatGPT boom; all AI money flows through Nvidia |
| 18 | Spaceships | Inside Voyager; neural graphics; risk dismissed |
| 19 | Power | Electricity limits; disappointment is the risk |
| 20 | Most Important Stock | $277bn day; Israel, China, Taiwan; luck or intellect |
| 21 | Jensen | How to read him; fear at the summit; maths on demand |
| 22 | The Fear | Bengio, Hinton, Sutskever versus LeCun and Huang |
| 23 | The Thinking Machine | Huang's outburst; inside the Eos supercomputer |
Introduction
Witt frames the book as four stories in one: how a niche vendor of video game hardware became the most valuable company in the world (above $3 trillion), how one stubborn founder got rich pushing a radical idea of computing for thirty years, how a small band of engineers defied Wall Street to build it, and how their silicon gave birth to a powerful and frightening kind of artificial intelligence. Nvidia won because Jensen Huang bet the company twice on technologies that had always failed in the market: parallel computing, then neural networks.
The man at the center
Witt's portrait of Jensen Huang is of an engineer first. He has run Nvidia for thirty-two years, the longest tenure of any technology CEO in the S&P 500, and knows circuitry with something close to intimacy. His method is to reason from first principles about what chips can do today, then gamble with conviction on what they will do tomorrow. He loses some bets; the all-in wager on AI is, in Witt's judgment, one of the best investments in Silicon Valley history.
- Temperament. Charming, funny, self-deprecating and often self-contradictory, with a constant deadpan patter.
- Background. Born in Taiwan, arrived in the United States at ten (in 1973, without his parents); Denny's was where he assimilated, working there as a teenager and eating through the whole menu.
- Identity. He still sees himself as an outsider: "You're always an immigrant... I'm always Chinese."
- Founding. Cofounded Nvidia (pronounced IN-vidia) in 1993, aged thirty, aiming at the young market for high-end game graphics; its customers were PC builders who sometimes bought transparent cases to display the hardware.
Witt interviewed Huang over breakfast at a Denny's, the chain where Nvidia's business plan had been drafted thirty years earlier. Huang ordered seven items (among them a Super Bird and a chicken-fried steak), joked to the waitress that he had worked his way up from dishwasher to busboy, and at the end left a $1,000 tip beside a table of half-eaten plates. The scene sets the book's register: a billionaire who still performs the striving immigrant.
The parlay in outline
The introduction compresses the whole causal chain the book will unpack:
- Parallel computing (late 1990s). To render the Quake games better, Nvidia altered its circuit design so processors could work on many problems at once. Huang's claim: before Nvidia the commercial success rate of parallel computing was "literally zero"; every start-up that tried had failed.
- The wilderness decade. For more than ten years he pursued the idea against Wall Street's wishes, hunting for non-gaming customers with huge computing needs (weather forecasters, radiologists, deep-water oil prospectors). The stock languished, the bet lost money, and he had to fight off corporate raiders to keep his job.
- AlexNet (2012). Dissident academics in Toronto bought two consumer gaming cards to train a neural network, a brain-inspired design most researchers then dismissed as an obsolete toy. Seeing how fast these networks trained on his hardware, Huang staked the company on the pairing. He now needed two historic market failures to succeed together.
- The payoff. Nvidia's value rose several hundred times. In a decade it went from $200 gaming accessories to multimillion-dollar supercomputers that fill a building floor, and (with partners such as OpenAI) sped up deep learning more than 1,000×. Midjourney, ChatGPT, Copilot: every major AI application was developed on Nvidia machines.
- $3 trillion+ market value, rivalling Apple and Microsoft.
- $30,000 AI-training chips containing about 100 billion transistors.
- More than a year of wait time for the newest hardware; double the list price on China's black market.
- Several hundred times growth in company value since the AI bet; >1,000× deep-learning speed-up in ten years.
- ~200 people interviewed by Witt: employees, cofounders, rivals, old friends.
Shovel seller of the gold rush
With a near-monopoly on AI hardware, Huang is arguably the most powerful person in AI and has made more money from it than anyone. Witt's analogy is Samuel Brannan, the San Francisco merchant of prospecting supplies who became California's first millionaire in 1849: Huang sells the shovels, except his shovels are training chips.
Huang insists he thinks like an engineer, not a businessman: break hard ideas into simple principles and lever them. His stated aim is survival ("I do everything I can not to fail"). His larger claim is that AI is reinventing the architecture of computing, essentially unchanged since IBM set it in the early 1960s: "Deep learning is not an algorithm... It's a new way of developing software." That software talks, writes essays, solves maths problems, diagnoses illness and scales with compute without plateauing.
Huang on risk
Witt arrived rattled by a video of a robot gazing at its own hands before sorting coloured blocks. Huang waved it off:
- The microwave line. "I know how it works, so there's nothing there"; AI is no different from a microwave oven, "all it's doing is processing data," and he has never once worried about it.
- The May 2023 statement. Hundreds of industry leaders signed a declaration equating runaway-AI risk with nuclear war; Huang did not sign.
- Horses. Asked whether AI could do to humans what industrialisation did to the horse population: "horses can't type."
- Writers. He granted Witt a few years before his "John Henry moment": AI "will come for the fiction writers first."
Huang. Almost alone among AI leaders, he holds that AI can only lead to good, calls it a new industrial revolution, and tolerates little dissent. One executive likened dealing with him to putting a finger in a power socket; Witt thinks staff would follow him out of a skyscraper window if he saw a market there.
The doomers. Many technologists, including the Toronto scientists who first ran AI on Nvidia chips, fear AI threatens human survival. Even OpenAI's founding mission is framed as averting catastrophe.
An elusive subject
Witt calls Huang the hardest subject he has reported on. He hates discussing himself, once physically ran away from a question, never read Witt's New Yorker profile, and on hearing of the biography said: "I hope I die before it comes out." Yet he opened doors to almost two hundred sources.
The private man who emerged is a goofy, beloved family man, quite unlike the "carnivorous" executive, and Witt argues the two are linked. Huang spoke openly of insecurity, fear of letting employees down and of shaming the family name. Money is not a scorecard for him but temporary insurance against calamity, an oddly touching attitude in a man worth about $100 billion. Anxiety combines with fascination: he never set out to pioneer AI, but once it arrived he pushed a maximalist agenda as hard and fast as possible, working 12 to 14 hours a day, seven days a week after three decades as CEO.
Witt's organising idea is amplification: Huang applies the same few precepts (diligence, courage, mastery of fundamentals) over and over, at ever larger scale. Much of the later man was already visible in the ten-year-old who arrived alone in 1973 at a rural school so hostile that his survival looks miraculous, which is where Chapter 1 begins.
Huang's "literally zero" success rate for parallel computing is his own framing: Cray, Thinking Machines and others built real (if niche) parallel businesses before collapsing, so "no lasting mass-market success" is fairer. The "100 billion transistors" is a round number: the H100 has about 80 billion, the Blackwell B200 about 208 billion.
Nvidia passed $4 trillion in July 2025 and $5 trillion on 29 October 2025, so the introduction's "$3 trillion" already looks modest. The China "black market" Witt mentions grew into a policy fight: Washington restricted even the cut-down H20 in April 2025 (Nvidia booked a multibillion-dollar charge) before allowing sales under a revenue-sharing arrangement later that year.
The Bridge
Witt traces Jensen Huang from Taipei through Bangkok to an Appalachian reform school, a Portland ping-pong club, a Denny's kitchen and an Oregon State engineering lab where he met his wife. The pattern that later built Nvidia is visible at ten: Huang thrives under adversity and improves through relentless, patient work on fundamentals.
Kentucky, 1973 to 1974
The bridge and the bullies
In late 1973 the ten-year-old Huang, newly arrived from Thailand with his older brother Jeff, walked each morning from his dormitory down to a rope footbridge with missing planks over a cold river, on the way to Oneida Elementary in Clay County, Kentucky, one of the poorest counties in the United States. He was too young for the boarding school where he lived, so he went to the public school instead.
- Profile. Bright, conscientious, had skipped a year into sixth grade, the smallest boy in class, imperfect English, the only Asian pupil.
- Classmates. Children of tobacco farmers and coal miners, almost all white, many very poor; some homes lacked running water.
- The welcome. On day one the principal introduced him as extremely intelligent; the bullying began immediately ("a perfect target," says classmate Ben Bays).
- The racial edge. Many local families had men who had fought in Vietnam; Huang says he was called a racial slur "every day," recounting it fifty years later without visible emotion.
- The bridge game. Bullies hid at both ends and swung the ropes while he was mid-span to throw him into the river. Bays recalls Huang looking as if he were having fun.
Ben Bays and the fight-back
Bays had been the bullies' previous target (small, studious, sometimes locked in lockers for hours). He came from a desperately poor preacher's family of eight living in a "holler" house with pit toilets, and had never met anyone like Huang. The two became fast friends.
- Academics. Despite the language gap Huang displaced Bays as top student; he drew well and had perfect penmanship, though only in capital letters.
- The Bruce Lee bluff. Local boys knew China only from Bruce Lee films, so Huang claimed to be a martial-arts expert. The schoolyard exposed the bluff, but he always fought back, sometimes wrestling bigger boys down. Bays says he was never pinned; Huang laughs that he remembers otherwise.
- Outcome. Huang taught Bays to fight back too, and the bullying subsided. By year's end Huang had won a silver dollar as best student and had faced down racists, once including a teacher; after school he led the "rowdy boys" running into the woods.
How a ten-year-old got there
Taiwan and Bangkok
Huang was born in Taipei in February 1963, the middle of three brothers. His parents came from Tainan, spoke Taiwanese Hokkien, and had lived mostly under outside rule: Taiwan was a Japanese colony until 1945, and after Chiang Kai-shek fled the mainland in 1949 the island lived under martial law.
- Father, Shing Tai. A chemical engineer; when Jensen was five he took a petroleum refinery job in Thailand and moved the family to Bangkok. In the late 1960s, passing through Manhattan to train with air-conditioning giant Carrier, he was so struck by New York that he resolved to move to America.
- Mother, Chai Shiu. A former primary teacher who spoke no English, yet taught it anyway: each night the boys memorised ten random dictionary words and were tested the next day. After about a year she enrolled them in an international school.
- Early memories. Hazy: setting lighter fluid alight on the family pool, a friend's pet monkey.
Sent away
In October 1973 about half a million protesters filled Bangkok's streets against the military dictatorship; Huang remembers tanks. His father sent Jensen and Jeff to an uncle in Tacoma, Washington, keeping the youngest son at home. The uncle wanted a boarding school willing to take two unaccompanied boys (ten and twelve) and picked the Oneida Baptist Institute (OBI), perhaps believing it a prestigious prep school.
OBI was a juvenile-reform academy in a town of 300, founded in 1899 by Baptist preacher James Anderson Burns to end a deadly family feud (he conceived it after being clubbed with a rifle and left for dead). By the 1970s it was a last-chance institution. Cigarette butts littered the grounds; Huang thought he was the only boy without a pocketknife. His seventeen-year-old roommate showed him fresh stab wounds on the first night. The roommate was illiterate: Huang taught him to read, and in exchange learned to bench press, building a habit of 100 push-ups a night that he kept for life.
- New names. Jen-Hsun became Jensen, Jen-Chieh Jeff, and the youngest, Jen-Che, later Jim.
- Contact with home. Audiocassettes by international post: listen to the parents' message, then record over the same tape. Jensen felt only occasional homesickness; it seemed a grand adventure.
- Summer labour. Jeff went to a tobacco farm; Jensen cleaned the dormitory toilets ("It wasn't a punishment... It was just my job") and cut brush with a scythe.
Leaving Kentucky
Huang spent summer 1974 in the dorm (Sunday night movies, apples from the tree outside his window) and began seventh grade at OBI, protected by his roommate. A year later his father found work in the US and the brothers joined the family in Oregon. Bays, later a nursing-home administrator, did not see him again for 44 years, until Huang returned in 2019 to donate a building to OBI. Bays says he always expected greatness.
Sent 8,000 miles from his parents, bullied, isolated, rooming with a knife fighter and scrubbing latrines, Huang thrived. His own explanation is unreflective: there were no counsellors then; "you just had to toughen up and move on." Witt treats this as the first instance of the book's theme: adversity sharpening rather than breaking him.
Witt notes that time has softened Huang's account. At the 2019 donation he spoke fondly of the (now demolished) bridge without mentioning the boys who tried to shake him off it; he says the chores taught him hard work, while admitting he would have said otherwise at the time; in a 2020 remote commencement speech he called OBI one of the best things that ever happened to him. Much of the chapter rests on recollections fifty years later, and Huang and Bays disagree even on whether he was ever pinned.
Oregon: clubs, ping-pong and Denny's
From 1976 Huang attended Aloha High School outside Portland: denim and velour, a helmet-shaped haircut, fast-improving English and a small clique of nerds in the math, science and computer clubs ("the popular kids! I didn't have a girlfriend"). In 1977 the school bought an Apple II, one of the first mass-produced PCs; he played Super Star Trek on it and coded his own Snake in Basic.
Table tennis: the defining trait
- Mentor. Lou Bochenski, owner of the Paddle Palace, a club in a converted Elks Lodge ballroom; his daughter Judy had been among the 1971 "ping-pong diplomacy" visitors to Beijing. Huang used a Western grip and had never played in Asia.
- The summer. He did almost nothing but practise, scrubbed floors at night to pay tournament fees, had a key, and sometimes slept in the ballroom rather than go home.
- The rise. After just three months of competition, Bochenski wrote to Sports Illustrated calling him the most promising junior ever in the Northwest. Within a year he was nationally ranked and in the under-16 doubles final in Las Vegas. Signature shot: an arcing forehand loop; style: relentless attack (friend Joe Romanosky).
Witt's judgment: Huang was athletic, but his distinctive quality was exceptional focus. When he set out to improve, everything else vanished; he outworked everyone, did not get frustrated or stuck, and never hit a plateau, watching skill accumulate from patient work on fundamentals.
Denny's
Near graduation he took a job at Denny's, then known for burnt coffee, powdered eggs and 24-hour service, and rose from dishwasher to server. He ate methodically through the menu (first bacon cheeseburger, first chicken-fried steak; favourite the Super Bird). He credits restaurant rush hours for his calm in crises: under adversity, he says, his heart rate goes down.
Motivation and college choice
He made the National Honor Society and insists his parents were not "tiger parents"; both brothers were bright but "terrible students," and he declines to analyse why he alone drove himself. With a near-perfect GPA, a skipped grade and national-level sport, he skipped the elite admissions race and in 1980 enrolled at Oregon State University (acceptance rate over 70%, ninety minutes from home).
Huang's version. No real thought went into it: his best friend Dean Verheiden was a legacy OSU student, "I just followed my best friend."
A longtime friend's version. He could have gone to the Ivy League or Stanford; after three countries and at least five schools by seventeen, he wanted to stay near his family.
Oregon State, circuits and Lori
OSU had no computer science degree, so Huang took electrical engineering. Witt stresses that the introductory sequence set his life course: he learned circuit design, which he has done ever since, and met Lori Mills.
- Lori. An earnest eighteen-year-old who lived by a timeline (career by 22, marriage by 25, children by 30); randomly assigned as his lab partner in week one, one of about three women among 250 EE students.
- The courtship. Looking "about twelve" and youngest in class, Huang competed not on looks but on homework, his self-described "superpower": he phoned every weekend to study together. After six months of breadboarding he asked her out; afterwards they were rarely apart.
- Circuit. Electricity flows from a source through components and back to the source; closing the loop is always the final step.
- Breadboard. A plastic prototyping grid on which students wire amplifiers and adding machines by hand.
- Transistor. A component that can act as a digital switch.
- Logic gate. Several transistors combined to perform a logical operation.
- Adder. Gates combined to compute 1 + 0 or 1 + 1; chained adders do serious mathematics.
- Semiconductor chip. Logic printed onto treated silicon discs with concentrated ultraviolet light, then diced into microchips; because every component is fixed in place, also called integrated circuits.
He graduated early with highest honours just as the 1980s silicon boom took off. PCs, CD players, cars, toys and microwave ovens all needed chips (later chargers, fridges, credit cards and toothbrushes would too), so skilled circuit designers were scarce, as they still are. He headed for Silicon Valley.
1963 Born in Taipei, February.
c.1968 Family moves to Bangkok for his father's refinery job.
1973 October unrest in Bangkok; sent with Jeff to Oneida, Kentucky.
1974 Summer in the OBI dorm; seventh grade at OBI.
1975 Rejoins parents in Oregon.
1976 Enters Aloha High School.
1977 School buys an Apple II.
1980 Enters Oregon State, electrical engineering; meets Lori Mills.
c.1984 Graduates early with highest honours; heads to Silicon Valley.
2019 Returns to OBI to donate a building; reunites with Bays.
Large-Scale Integration
Witt follows Huang through his first decade in chip design, at AMD and then LSI Logic, where he served the two Sun Microsystems engineers, Curtis Priem and Chris Malachowsky, who would ask him to run their start-up. Huang's colleagues saw what his later modesty denies: an outrageously hard-working, ambitious engineer who could make "1 + 1 = 3" and who wanted to run something by thirty. The chapter ends with him treating the start-up offer as an engineering problem to be researched before it is decided.
AMD, a proposal and a crash (1983 to 1985)
First job
Huang joined Advanced Micro Devices at twenty, too young to drink, on a starting salary of $28,700, a figure he could still recite forty years later. He lived frugally and within a year had saved enough for a Toyota Supra sports car and an engagement ring.
Witt uses the setting to mark an era. In 1984 the Bay Area was still a frontier outpost: the most valuable US companies were industrial giants such as DuPont and General Electric. Huang's generation of founders would eventually hollow those conglomerates out and leave the stock market dominated by tech.
The proposal and the black ice
AMD's Christmas party, one of the Valley's most lavish, filled San Francisco's Moscone Center with free drinks and a live set from the band Chicago. There Huang, 21, proposed to Lori Mills, 22 and still a student; she accepted. He then proposed driving through the night, nine hours over snowy mountain roads, to tell her parents in person. After more than five hours, near the California-Oregon border in old gold-rush country, the Supra hit black ice, slid off the shoulder and rolled. The car was totalled; responders had to cut the couple out. Lori was mostly unhurt; Jensen needed stitches in several places and wore a neck brace for months. Years later his main regret was the car ("Incredible car").
Witt's point is that the dependable, mature Huang could still do things only a 21-year-old would think sensible. The engagement survived, perhaps strengthened. Huang was close to the Mills family, especially Lori's father, an all-American type friends compared to Jimmy Stewart; friends joked Jensen was closer to her parents than to his own.
Drawing chips by hand
- The method. At AMD Huang sketched each layer of a chip on paper (transistors at the bottom, interconnects above). Each sheet went to the back office to be transferred to coloured cellophane, from which stencils called photomasks were made and sent to a fab.
- Learning Mandarin. The photomask workers were all Chinese women with little English. Huang, whose home language was Taiwanese Hokkien (as distant from Mandarin as German from English), learned Mandarin phonetically by chatting with them; they reminded him of his mother.
- The AMD shares. He bought AMD stock through the employee-purchase plan and held it, Witt notes, "with escalating irony" for his whole career, since AMD became Nvidia's rival.
He remembers his two years at AMD fondly.
LSI Logic and the end of hand-drawn chips
In 1985 a co-worker lured Huang to LSI Logic, which made the first software-design tools for chip architects. Chips now held hundreds of thousands of transistors, beyond paper draftsmanship: Witt's image is a tennis court covered in a maze of human hair.
- Hand layout. Each layer drawn on paper, transferred to cellophane photomasks and sent to a fab; Huang's AMD years.
- Large-scale integration (LSI). Software automates low-level blocks of circuit design so engineers can concentrate on architecture.
- Very large-scale integration (VLSI). Abstraction so high the individual transistor disappears from view; still the entry point for most engineers. Only "graybeards" like Huang remember the artisanal chip.
The Valley of the 1980s
Lori graduated in 1985 and joined Silicon Graphics (SGI), maker of expensive 3D graphics workstations and the Valley's other prestige employer; at first she out-earned Jensen.
- US 101. A 25-mile strip from downtown San Jose to Stanford, with exits for Cupertino (Apple), Santa Clara (Intel), Milpitas (Cisco) and Mountain View (SGI). No place on Earth had produced so much wealth per square foot.
- Tethered. Huang would spend his entire career within a five-mile radius.
- The look. Not Manhattan glamour but bland mid-rise boxes, parking lots, strip malls and freeways; inside, CRT monitors, drop ceilings, fluorescent light and reconfigurable "action office" cubicles. LSI's low cubicle grid was called "the pit." Huang, in big glasses and button-downs, found it heaven.
Huang and Horstmann in the pit
His pit-mate Jens Horstmann, a German engineer who came for a six-month mentorship and stayed, became his closest friend (same age, same initials, both immigrants). Horstmann was the extrovert in private life; at work Huang was the risk-taker. There were no weekends: in at 7 a.m., girlfriends phoning at 9 p.m. to ask when they were coming home.
- SPICE mastery. Huang mastered the Simulation Program with Integrated Circuit Emphasis: type an ordered list of components at a command line, get back a text-only table of voltages. Others treated it as a teaching tool; he used it to push circuits further than anyone thought possible, with no graphical interface or even colour monitor. Where colleagues said "There's no way," he said "Let me see what I can do."
- No ratholes. Horstmann's distinction: many engineers get absorbed in problems, but Huang could sense when a problem had stopped yielding and switch direction instead of getting lost.
- Risky promises. The hungriest customers were computer-graphics designers. With Huang's encouragement Horstmann signed contracts for products they did not know LSI could build; older engineers warned this could end their careers. Almost everything was delivered, and the reward was harder problems, a difficulty curve Huang relished.
- 1 + 1 = 3. Horstmann's phrase for turning customer orders into tools, and tools into methodologies; most engineers cannot reach 2, "you're lucky to get one and a half."
Three traits recur in every account of Huang at LSI: work ethic without limit, a willingness to promise the not-yet-possible and then deliver, and the judgment to abandon dead ends early. Witt presents these as the operating system Nvidia would later run on.
The square couple
The Huangs were the responsible ones of their circle: first married, first homeowners (a two-storey, four-bedroom tract house in east San Jose, 1988, with a barbecue patio), stable jobs, maxed-out tax-deferred retirement accounts, and a much-loved dog, Sushi.
- Lori's depth. Horstmann recalls asking her about a client's satellite chip failing from cosmic-ray interference; she had solved a similar problem requiring both electrical engineering and particle physics.
- "Kinda square." They worked constantly, rarely travelled and socialised almost only within semiconductors. When Horstmann introduced Huang to a friend who ran a craft microbrewery, Huang was baffled that such a person could exist; he seemed to have no friend outside tech.
Huang was promoted repeatedly and took night classes at Stanford for a master's in electrical engineering. Work was so heavy that it took eight years; by the time he graduated in 1992, much of the early material was obsolete. For years he shuttled along the 101 between school in the west, home in the east and work in the middle.
The future cofounders
Through LSI Huang came to serve two Sun chip designers. Sun Microsystems, like SGI, made high-end workstations, and these two were exacting customers whose requests ordinary salespeople could not follow. Malachowsky says LSI dug deep to find its most outward-looking technical person: Jensen.
| Founder | Role in the trio | Background and manner |
|---|---|---|
| Curtis Priem | The architect: traced electricity through circuits in his head | Cleveland suburbs; mother wanted a professional cellist until a North Carolina music camp seated him last chair, second orchestra; switched to computers, Rensselaer Polytechnic; spoke in endless technical monologues whether or not anyone followed; built his own flight simulator |
| Chris Malachowsky | The mechanic: could build anything Priem imagined | New Jersey ex-"long-hair"; burly, gruff, quick to laugh; saw computers as physical machines; flew his own plane |
| Jensen Huang | The logistician: ran LSI's fab tooling to mass-produce their designs | Younger, technically an outside vendor, but acted as their manager; they trusted him more than their own bosses amid Sun's politics |
The three say they never stepped on each other's toes. Priem: "There was politics at Sun like you wouldn't believe."
How Huang managed
- By example. He drove himself hardest, avoided gossip and apportioned credit carefully.
- Instant accountability. If a product would slip or a function could not be delivered, he immediately explained what went wrong, who was responsible and the fix. Malachowsky could think of few Valley product managers who reliably did what they said.
- The flaw. Candor bordering on insult; little patience for disagreement; real surprise that some engineers did not want fourteen hours a day in the simulator. To quarrelsome workaholics like Priem and Malachowsky, this only proved his fitness to lead.
The Sun GX and a rising star
The collaboration produced the Sun GX in 1989, a line of 3D graphics processors for workstations used by scientists, animators and CAD modellers. It took a wire-frame skeleton of points and "painted" on textures pixel by pixel, yielding rotatable blocky polyhedra in sixteen colours on a CRT: crude now, but a glimpse of the future.
LSI founder Wilf Corrigan then promoted Huang to run a system-on-a-chip platform letting customers combine 3D graphics, video and game controllers on one die. It sold well; Malachowsky believed Corrigan was grooming Huang as his successor.
The decision to leave
Priem and Malachowsky proposed a cheaper GX for PC video games; Sun refused (an executive sniffed that Sun served scientists, not gamers). Wanting to build it themselves but unwilling to run a business, in 1992 they asked Huang to be their CEO. Witt lays out what weighed against it:
- No plan, no name. Only a rough sketch of a product Sun had judged unprofitable, versus Huang's own division on a secure management track.
- A combative pair. Constant yelling matches and slammed doors; staff asked if the team was disbanding. Priem mocked Malachowsky's "impedance function" (resistance to current); Malachowsky retorted that Priem's "user interface" was poorly designed.
- Family strain. Son Spencer (1990) and daughter Madison (1991) had arrived (Sushi tried to steal their pacifiers). Without reliable childcare, Lori had left SGI to raise them, abandoning the two-income mortgage plan. The family had six months of savings.
Horstmann, godfather to both Huang children, notes that his own wife and Tina Malachowsky also left engineering to raise children, and that all three women were superior engineers. He admits some guilt ("maybe we should have tried a little harder" with nannies). Huang showed visible discomfort recalling that he first asked his brilliant wife to pause her career, then asked permission to quit for a start-up. Lori told him to go for it: "She always believed in me."
What tipped him
- Lost turf. In his twenties Huang ran an LSI division with $250 million in annual revenue, with older engineers reporting to him. To ease the friction Corrigan hired a senior director from Intel to co-manage it. Huang was incensed and called it political, in Witt's phrase the most profane word in an engineer's dictionary. Horstmann: he had built the division from nothing and it was taken away. Witt suggests this may have been the final push.
- Being wanted. Obnoxious or not, Priem and Malachowsky were brilliant, and Huang was their first and only choice; they trusted no one else.
In his 2020 OBI address Huang described himself as an unambitious staffer with a state-school education whom nobody would have picked to run a company. Witt finds that everyone who knew him then called this a fib. Malachowsky remembers Huang telling them over dinner at his house that he wanted to run something by thirty. Analyst Hans Mosesmann (later involved in Nvidia's IPO) heard of an LSI manager who left Huang's review grades blank and wrote, in effect, that he looked forward to working for him one day. A smaller wrinkle: Witt's dates (two years at AMD, an LSI move in 1985) imply Huang began at AMD around 1983, before his 1984 graduation, so the start date is approximate.
Business as an engineering problem
Huang did not commit at once. His reasoning was a ladder of difficulty: start-ups are hard, hardware start-ups harder, consumer-hardware start-ups hardest; most never get past a prototype. Decisions, for him, are clinical, with no room for hope.
- Principles first. Break the venture into simple governing principles, as an engineer would, then lever them.
- Research the fundamentals. Market, supply chain, competition, technology and product fit.
- Talk to people. Exploratory calls with customers, game developers and graphics experts.
- Read the boring data. Years of industry reports, bar charts, sales figures and surveys, hunting for an upward trend.
In short, he had to do his homework, and Witt closes by noting there was only one place he would do it (the Denny's of the next chapter).
c.1983 Joins AMD at twenty on $28,700.
1984 Proposes to Lori at AMD's Christmas party; Supra crash next morning.
1985 Moves to LSI Logic; Lori graduates and joins SGI.
1988 Buys a four-bedroom house in east San Jose.
1989 Sun GX ships; Huang promoted to run system-on-a-chip platform.
1990-91 Spencer and Madison born; Lori leaves SGI.
1992 Stanford master's after eight years; Priem and Malachowsky ask him to be CEO.
New Venture
Nvidia was incorporated in April 1993 to build a graphics accelerator for PC games, a market with at least 35 rival start-ups. Its first chip, the NV1, sold more than 100,000 units in its first Christmas and then collapsed. Priem's exotic design bet against the industry's choice of triangles, and when Microsoft's DirectX standardized on triangles, Nvidia's whole product roadmap was stranded. Huang's own verdict on those first years: "Every single decision we made was wrong."
The Denny's spreadsheet
The setting
The founding meetings took place at a 24-hour Denny's beside a San Jose freeway, near Huang's house (he took his children there), amid gas stations and money-transfer shops. Some tables still had ashtrays; a Grand Slam breakfast cost $3.99. The quiet back section was full of police officers writing reports: in 1993 California was in the worst crime wave in its history, with more than 4,000 murders in a year, and San Jose was not spared. Priem and Malachowsky mostly drank free coffee refills ("I was there for the pie," says Priem); Huang compensated by ordering plate after plate, chatting with staff and tipping lavishly.
The opportunity
- The vacuum. Sun had declined PC gaming, and so had Silicon Graphics, the 3D leader, whose staff were busy animating the dinosaurs of Jurassic Park. A swarm of start-ups rushed into the gap.
- The concept. Take the hardware that painted wire-frame skeletons of planes and dinosaurs and repurpose it to animate controllable figures in 3D games.
- The product. A graphics accelerator: an add-in circuit board built around a graphics chip, clicked into a motherboard slot after unscrewing the PC case, plus a thin software layer running "on top of the metal" so game programmers could reach the hardware.
- The crowd. At least 35 competitors; Huang worried there was no room for a thirty-sixth.
Advice not taken
Huang befriended Jon Peddie, the leading graphics expert and textbook author, phoning him late into the night. Peddie told him the field was too crowded and the best engineers already taken: "the best advice he never took."
Tuning the spreadsheet until it said yes
Huang's magic number was $50 million a year in revenue, the minimum that would make the venture worth the effort. His laptop spreadsheet projected revenue for the next few years; nudging one cell pushed the projection below $50 million, nudging another pushed it above. After a few sessions Huang had talked himself into it. Witt's framing is gently ironic: the founding analysis was as much persuasion as forecast.
Naming, founding, funding
The three took their unnamed firm to Palo Alto lawyer Jim Gaither, one of the Valley's most sought-after advisers (he once attended 129 board meetings in a year). Struck by Huang as a natural leader, Gaither resolved they would not leave without hiring him.
- "NV." Lacking a name, Gaither wrote "NV" for new venture as a placeholder. By coincidence the founders already called the chip NV1, joking that it would make rivals "green with envy."
- The name. Priem compiled NV-themed words from several languages' dictionaries. "Nvision" was taken by a maker of recycled toilet paper, so they chose Nvidia, from Latin invidia, envy.
- Roles and equity. Huang CEO, Priem CTO, Malachowsky VP of engineering, each with an equal third.
- Capital. Incorporation needs only a token sum: Gaither took the roughly $200 in Huang's wallet in exchange for his third of the shares (Huang collected his cofounders' portions later), which Witt notes proved "a decent investment."
- The deadline. Huang turned 30 in February 1993; the certificate of incorporation was filed in April, about six weeks later. "He just missed his deadline," says Malachowsky.
Back at Denny's to settle terms, Malachowsky wandered to the window (view: a Taco Bell and a decaying gas station under sodium lamps) and noticed the top of the glass was pitted with bullet holes, possibly from people shooting at police from the overpass. The three paid, tipped big and fled to Priem's townhouse, where the papers were actually signed. None returned for years. The founding-at-Denny's story is therefore myth, but a plaque now marks the booth at 2484 Berryessa Road, San Jose.
Two bedrooms and a kitchen table
Nvidia inverted the garage cliché: Priem's furniture went into the garage and the company worked from two upstairs bedrooms of his condo (he kept the third). The hardware group designed circuits on high-end Sun workstations in one room; the software group wrote the protocols game developers would use in the other.
- Unpaid believers. The GX's reputation drew several Sun engineers to quit and work without pay, expecting salaries to be backfilled once money was raised.
- The doubter. Over beers, Horstmann watched Huang take a heated call in Taiwanese; afterwards Huang reported his mother had told him to quit and rejoin a big company.
- The center of traffic. Huang's "office" was a small round table by the kitchen, so anyone fetching a snack passed him. However powerful he became, he kept himself at the center of foot traffic.
The Sequoia pitch
Huang stayed up all night on a business plan and ended with nothing ("I still don't"). What he had was the backing of LSI founder Wilf Corrigan. The pitch to Don Valentine, the blunt founder of Sequoia Capital whose favourite question was "Who cares?", went badly: Huang fumbled and Priem interrupted with irrelevant technical asides. Valentine took Huang aside: it wasn't very good, but Corrigan said he had to fund them, so they were in business. Sequoia and Sutter Hill Ventures each wrote checks for several million dollars.
The board seats went to Mark Stevens (Sequoia, 33) and Tench Coxe (Sutter Hill, 36), both affable Harvard MBAs in Patagonia, hard to tell apart. Both say they backed Huang himself, not the proposal.
In computing, Coxe argues, the ordinary CEO's habit of listening to the customer is a mistake, because customers don't know what's possible. Backing a world-class engineer who can see what can be built beats conventional management, which he says later hobbled Intel and Microsoft.
Building the NV1
Jurassic Park and the scale of the problem
Jurassic Park opened two months after Nvidia's founding, the first film to blend CGI convincingly with live action. One three-second shot of a T. rex bursting through a log took ten months to render. The NV1's output would be crude by comparison, but Peddie's point is that the difference is of degree, not type: games and effects are still "blocks sliding around hitting each other."
Priem's differentiation bets
The overriding goal was to stand out from dozens of rival products. Priem, the architect, loaded chip and software with esoteric features; asked about the NV1's architecture, he once talked for 27 minutes without pause, explaining he wanted an architecture to last one hundred years.
- Quadratic texture mapping. Painted wire frames using curved surfaces rather than triangles, for depth and realism; among consoles only Sega used the approach.
- Object-oriented software. Reusable blocks of code for programmers.
- Bundled extras. A sound chip and joystick port on the board so Sega could port games such as Virtua Fighter (martial artists drawn in blocky quadrilaterals) to the PC.
- A console deal. Huang signed Sega for the NV2, the graphics chip for the coming Dreamcast.
The Sunnyvale strip mall
By 1994 Nvidia had moved to a dingy Sunnyvale strip-mall office: frayed carpet, stained drop ceiling, smells from a Chinese takeout next door, a bathroom shared with another firm, and a Wells Fargo opposite that was robbed twice while they rented (visible from the front window).
- Culture. Lunch round a ping-pong table where Huang beat his staff; a lab of dismantled equipment and a Sega console where Priem held most high scores and played mid-workday.
- Huang's command center. The conference room whiteboards, then the walls, covered in strategy, deadlines and colour-coded chip diagrams in his immaculate all-caps hand. Board members felt they were standing inside his brain.
- The adult in the room. During a board meeting near tape-out (sending the first blueprint to the fab), a cheer went up; Huang rushed out hoping for a finished schematic and found Priem had set a record in Road Rash. He confiscated the console and gave it to his kids. An early employee: even when youngest, "he was the adult."
The merchant-chip model
Nvidia did only design and quality control. A European fab made the chips; the US board vendor Diamond handled assembly, distribution and retail.
- Jeff Fisher. Filmed the arrival of the fifteen prototype chips on a VHS camcorder: a crowd of excited men in loud nineties shirts as Priem pried open a fingernail-sized die with pliers. Recalls Huang's heavily starched shirts ("very, very, very well-pressed"); Huang himself missed the unboxing.
- Dwight Diercks. Son of a Minnesota pig-farming family, large and unhurried, whom colleagues warned Witt not to underestimate. Ran validation: played an "art demo" through the prototype and checked every frame for errors, like editing a film frame by frame, before the corrected blueprint went to mass production.
Boom, then bust
The market explodes
Two hits made 3D demand outrun even Huang's projections: Myst (September 1993), a puzzle game that became the best-selling PC title to date, and Doom (three months later), a demon-shooting horror game. Opposite poles of gaming, each sold millions and each sent buyers to stores for accelerators. Developers embraced the PC, escaping Sega and Nintendo's closed ecosystems; titles such as Civilization II and Command & Conquer offered endless, addictive replay.
Launch, Christmas 1995
The NV1 launched in fall 1995 into a shelf crowded with Matrox, S3, 3dfx, Cirrus Logic and ATI. Diamond muddied things further by selling Nvidia's chip as "Diamond Edge" and 3dfx's as "Diamond Monster." The Edge cost $249, more than a Super Nintendo, for a flimsy green board that would be obsolete within a couple of years. The key variable in Huang's spreadsheet was how many buyers would climb this upgrade ladder; more did than anyone expected.
- 100,000+ NV1 chips sold by end of 1995, pulled along by bundled Virtua Fighter.
- 100+ employees after Huang's confident hiring spree; an over-the-top Christmas party.
- Q1 1996 problems surface once Virtua Fighter's charm faded: other games rendered badly.
- No depth buffer and curved surfaces only, so objects could not be sorted in a scene; characters sank into pavements or passed through walls ("clipping").
- Blue Screen of Death in the worst cases, as the card stopped talking to Windows ("catastrophic," says developer Tim Little).
- 1995 Microsoft announces DirectX, a standard supporting triangles only.
The collapse
Sales tapered, and customers used big-box return policies to send cards back. Months after launch Diercks found dozens of opened, marked-down Diamond Edge boxes at Fry's Electronics. Just as payroll peaked, demand for Nvidia's only product vanished.
Microsoft, tired of graphics peripherals crashing Windows, had set the rules: DirectX meant triangles. Worse, Nvidia's whole supply chain and roadmap, the Sega NV2 and the NV3, rested on the same quadratic architecture. Huang's beautiful wall diagrams had to be wiped.
Decades later Priem still defends the design he built to last a century. Huang and Malachowsky speak of it with the bitterness of a failed relationship: Huang calls it a "disaster," Malachowsky uses a cruder word, and Huang sums up: "We missed everything."
Clever engineering cannot beat a platform standard. Developers preferred triangles and Microsoft codified that preference, so Nvidia's distinctive features became liabilities overnight, and because the next two chips shared the architecture, one product mistake became a company-threatening one.
Witt deflates the Denny's founding myth himself, but other details are thin. He does not name the European fab (it was SGS-Thomson, which also sold the chip under its own brand), and "DirectX announced in 1995" compresses events: the triangle-based Direct3D arrived with DirectX 2.0 in 1996. The chapter's numbers (several million dollars from each VC, the $50 million threshold) come from founders' recollections rather than documents.
Thirty Days
In 1996 Nvidia cut staff from more than 100 to 35, abandoned Sega for Microsoft's standard, and spent its last money on a hardware emulator so it could skip physical prototypes altogether. The copycat Riva 128 saved the company, and Huang drew the lesson that desperation, not inspiration, wins: "Our company is thirty days from going out of business." The same crisis settled who ran Nvidia and first exposed the Wrath of Huang.
A company that looked doomed
David Kirk, a graphics consultant who had watched many start-ups fail (his own included), arrived in 1996 and judged Nvidia doomed on sight: haggard staff, a product that did not fit the market, and founders locked in a "technical discussion" that was plainly about more than technology. His condition of employment was a hand-delivered paper check every Friday, because he doubted the firm would last longer than a week.
Huang's pivot
Before Kirk arrived Huang had told the staff the truth:
- The architecture is dead. The NV1's quadratic design would be abandoned.
- Switch camps. Nvidia would leave Sega and align with Microsoft (DirectX triangles).
- Race a knockoff. The goal was an affordable, generic accelerator shipped faster than anyone, cutting every corner possible.
- Mass layoffs. Head count fell from 100+ to a skeleton crew of 35 engineers, working overtime.
Kirk walked into a half-empty office of fluorescent-lit cubicles. Where the ping-pong table had stood in the break room sat the hardware emulator: a cumbersome, half-finished-looking machine with exposed circuitry and cables across the floor, bought with the last of the company's money. It was, in effect, a fake microchip built in code rather than silicon, on which a design could be tested before it existed.
Huang versus Priem
The proxy fight
Kirk discovered a second, unspoken job: restraining Curtis Priem. Kirk had actually invented quadratic mapping, yet advised dropping it ("I have lots of ideas"). That only made Priem defend it harder. Priem was a purist who saw technical compromise as surrender to the money men; Malachowsky's verdict was that Priem thinks about the end point but lacks the makeup "to, like, stay in business."
Kirk saw that quadratics were a proxy for control. Priem, as CTO, wanted total autonomy over the technical roadmap; Huang wanted customers to stop returning the product; Malachowsky, the mediator, had lost control, and closed-door argument had become public shouting.
- Priem's oddity. Many found him hard to work with ("brilliant... but not a people person," says Kirk). Witt's example: to beat spam he created thousands of email addresses, one per correspondent.
- The broken pact. The founders' implicit deal was Huang on business, Priem on tech. Priem held an equal founding stake, had hosted the signing, and had chosen Huang; overruled, he felt betrayed.
- No leverage. Priem held about a tenth of the shares but, like Malachowsky, had no board seat. Huang had persuaded both that only the CEO should sit on the board, and the two, who hated business so much they had hired a boss, went along. With the board backing the Microsoft strategy, Huang overrode Priem unilaterally.
- The moment Priem knew. In a hallway argument Priem told Huang what to do; Huang shouted back about everything he had to carry. Priem concluded Huang "was all alone."
Witt's point is structural: equal founders are not equal owners of decisions. By keeping Priem and Malachowsky off the board, Huang ensured that when tech and business clashed, the CEO with the board's support would win, and he did, at the moment the company's survival depended on it.
The Wrath of Huang
When Huang first argues a point he is calm, moving from premises to arguments to conclusions. At that point, Witt writes, the fuse is lit: agree, or risk detonation, a furious public berating in front of colleagues.
- A CEO-era trait. Jens Horstmann says Huang had no reputation for outbursts at LSI; it began with the CEO role. One employee recalls the charming, self-deprecating boss suddenly screaming in front of a hundred people.
- Audience by design. Huang dresses people down in public so others learn: "Failure must be shared." Whoever is responsible for a late project must stand and explain everything that went wrong, followed by Huang's withering analysis.
- The survival test. Dwight Diercks says you can tell at once who will last: anyone who gets defensive will not be at Nvidia long. He insists Huang never "just yells"; he waits for a meeting to make it educational.
Witt does not accept Diercks's defence wholesale. Some episodes were simply verbal abuse: after a minor mistake, one employee was asked, before three dozen executives, how long he had worked there and his salary; Huang computed his career compensation and demanded it all be refunded. It did not feel like a joke, and the man barely slept for three weeks. Watching was nearly as bad as receiving. Kirk was targeted only once, when he intervened for a colleague, and compares it to standing up on a battlefield to shout "stop shooting": the gun turns on you.
Spending the last dollar
The Sega check
Sega had agreed to pay $1 million on delivery of working NV2 prototypes. In mid-1996 Huang delivered them (the only NV2 chips ever made), then told Sega, with great deference, that Nvidia would not build the Dreamcast because it was going over to Microsoft, but that the delivery technically met the contract and without payment Nvidia would go bankrupt. Sega paid ("They took it pretty well, considering"). The money bought the emulator.
Triage began. Bills were ranked by how long payment could be delayed: vendors first, then utilities, then employees. Huang was determined to make payroll until the lights went off.
Why last place helped
The gamble was stark: if any transistor on the coming NV3 was wrongly placed, a failed production run would end the company. Huang, who had always fought for first, discovered that last place was liberating, even better than the middle.
- Freedom. A last-place firm can take the shortcut nobody else dares: no company in chip history, Witt says, had ever skipped prototyping; Nvidia would go straight from emulation to mass production with a "digital napkin sketch."
- Second-mover intelligence. Rivals sent preview cards to magazines and websites for publicity. Kirk worked his media contacts to learn what those cards could do, and because Nvidia was skipping both prototypes and previews, there was just time to clone those features into the NV3.
The emulator grind
- Inverted frame rate. Games ran at about 30 frames per second; the emulator produced roughly one frame every 30 seconds, so engineers audited the demo reel in agonising slow motion.
- Double shifts. Diercks ran the machine by day, Kirk by night (venting late-night tension by beating Priem's high scores at Virtua Cop with a plastic gun; "annoying Curtis was just a bonus").
- Result. A normal twelve-month development cycle shrank to about three months.
Tape-out and the Riva 128
The NV3 taped out in early 1997 and the blueprints went to Europe. Nvidia's three dozen staff celebrated with beers at a cheesesteak shop in a strip mall; Huang toasted, privately rating the odds at fifty-fifty, "but we were going out of business anyway." While the chips were in the fab he offered Kirk a permanent job with "borderline-stupid" equity and the title chief scientist; Kirk took it despite having to work under Priem.
The chips arrived in late spring. Everything rode on whether all 3.5 million transistors matched the emulation. Diercks mounted them in a tester and played the demo reel: it ran flawlessly at 30 fps.
NV1 (1995). A revolutionary design developed by industry best practice with physical prototypes; quadratic surfaces; Sega-centric. Result: returns and near-bankruptcy.
Riva 128 (1997). Mostly a copycat on Microsoft's triangle standard, built in an improvised rush with no prototype, plus two innovations: a 128-bit memory path (double the norm) and multi-purpose acceleration of games, spreadsheets and DVD video. Result: 1 million cards in four months.
The breadth was reflected in the name: NV3 was rebranded Riva 128 (Real-time Interactive Video and Animation). Boards reached Best Buy in August 1997, with Nvidia running on "vapors... We had nothing left." Having sent no previews, Nvidia had to beg for coverage, but reviewers loved it: rendering up to five million triangles per second, "the best accelerator money can buy."
- 100+ → 35 staff in the 1996 purge.
- $1 million from Sega, the last cash, spent on the emulator.
- 12 → 3 months of development time.
- 3.5 million transistors, all matching the emulation.
- 5 million triangles per second; 128-bit memory path.
- 1 million cards sold in the first four months.
The lessons Huang kept
- Emulate, don't prototype. Nvidia abandoned physical prototypes; Huang says it is still the largest user of emulators in the world. The industry's preference for hard prototypes had seemed as sensible as crash-testing a car, the choice an "adult in the room" would make.
- The adults take too little risk. Huang, himself the confiscator of game consoles, learned that best practice had produced a flop while a frantic improvisation had produced a hit. Sometimes you have to gamble.
- Permanent crisis. Witt's summary of the lesson: desperation, not inspiration, is the mother of victory. Huang told staff to behave as if bankruptcy were imminent even amid huge profits, and for years opened presentations with "Our company is thirty days from going out of business," still Nvidia's corporate mantra.
"No firm had ever skipped prototyping" is Witt relaying company lore; emulation and simulation were already established in chip verification, and what was unusual was going to volume production without first silicon. The account of the Huang-Priem fight is also largely Priem's and Kirk's, with Kirk openly admitting he and Priem did not get along.
Going Parallel
After sidelining cofounder Curtis Priem in 1998, Huang armed Nvidia with three things: Clayton Christensen's theory of disruption, a foundry partnership with TSMC, and a chip built to please John Carmack. To render Quake better, Nvidia split pixel shading into two pipelines running the same instructions at once on one chip: parallel computing, the secret that would later remake all of computing. Nobody, Huang included, suspected what that change would one day mean.
Priem's exit from power
The last fight
In 1998 Huang and Priem had a final shouting match that lasted most of a day: hours of yelling, calming, and erupting again until both were hoarse. Staff sensed a divorce. Priem broke first, slammed his office door, and afterwards refused to speak to Huang.
- Wounded pride. Given a blank sheet to design the NV1, Priem never recovered from seeing it remaindered on shelves.
- The provocation. Earlier in 1998 Huang promoted David Kirk to co-develop the technical architecture, making Priem's former subordinate his peer.
- The retaliation. Priem locked several employees out of the production database so they could not submit work, which Witt calls childish.
- Mediation. Malachowsky gave up adjudicating; on the board's advice Nvidia hired a mediator who had worked with John Sculley and Steve Jobs at Apple (an episode that ended with Jobs fired). She judged Huang and Priem "significantly worse."
- The house rule. "Never let Curtis talk to investors, and never let Curtis talk to customers." Priem later conceded both were sound.
Two more demotions
Priem briefly became Kirk's technical adviser but would not work for former reports, then was moved to managing the patent portfolio, out of daily decisions. Kirk would oversee close to a thousand people; Priem never again managed more than four. He kept his shares and his friendships: Malachowsky was best man at his 1999 wedding, a year after his second demotion in three years. Witt's balance sheet: Priem saw the game market, named the company and housed it in his bedrooms, but from 1998 he had little to do with its success.
Kirk, poaching and the rule of three
David Kirk became Huang's consigliere. He disliked the Valley's pressure culture (he first spoke to Witt from Hawaii, wine in hand, wearing a "Procrastination University" shirt) and came across as a popular professor. Underneath was a pitiless competitor, perhaps shaped by an alcoholic, absent father and a sickly childhood of chronic strep throat: people who say winning doesn't matter "have never won anything."
- The poaching list. As rival start-ups failed, Huang wrote a master list of competitors on his whiteboard and, with Kirk, picked the two or three best engineers at each to hire away.
- "I removed its brain." At a trade show Kirk demoed the Riva 128 to a rival engineer, who gave up on the spot; Kirk hired him within days and that company died. Each acquisition of talent made survival harder for the remaining start-ups.
At Denny's in 1993 Huang had pictured sharing a big market; by 1998 he wanted it to himself. Forty companies today; in five years there will be three:
- A big one in charge (Huang's goal for Nvidia).
- A laggard playing catch-up.
- A small disruptor attacking the other two.
Huang's business education
Tench Coxe says that at first there was "a great deal he didn't understand." Huang fixed that by reading every business book he could find; his rarely used office is still stacked with them.
- Not literary. He had read none of the Valley canon (Rand's The Fountainhead, Asimov's Foundation, Adams's Hitchhiker's Guide), says he has never read science fiction, and the only novelist he recalls enjoying is Paulo Coelho.
- Encyclopedic on business. In a pricing dispute with an MBA, Huang (who had read ten or fifteen pricing books) asked him to name his three favourite books on pricing. The man could name none; Huang listed his own three and said they would resume once the man had read them (story via Dwight Diercks).
- The book. The Innovator's Dilemma (1997) by Harvard's Clayton Christensen coined "disruptive innovation": small firms erode incumbents by serving niche, marginally profitable customers the leaders dismiss. Disruptors need not be high-tech (scrap-metal recyclers, hydraulic-shovel makers).
- Canonical case. In the early 1960s Honda sold the off-road Super Cub to American teenagers (the Beach Boys sang about it). GM preferred selling Cadillacs to businessmen, so ignored it; Honda later built compact cars and raided the US car industry from below.
- The escalator. It is easier to move up the profitability escalator than down: going down means deliberately making inferior, lower-margin goods, which upsets investors.
- The counterintuitive rule. At times it is right not to listen to customers, to invest in lower-performing, lower-margin products and to chase small markets: the point popular "disruption" talk usually misses.
- Nvidia's reading. Sun and SGI skipped PC gaming not out of contempt but because margins were poor and gamers would not move their profits. Like GM, they erred: gaming could be Nvidia's base to overtake workstations, and even Intel.
Huang became obsessed with Christensen (a towering, kindly Mormon bishop and former college basketball player): he read all his books, assigned The Innovator's Dilemma to executives and later hired him as a consultant. He talked to staff of one day disrupting Intel, then among the world's most valuable firms; Kirk recalls Huang saying even then that Nvidia could be bigger than Intel.
Meanwhile the method toward Intel was continuous tactical retreat, a Tom-and-Jerry relationship: "To this day, we don't compete with Intel... Whenever they come near us, I pick up my chips and run" (Huang, 2023). Christensen's gospel meant selling offbeat products to customers Intel would never want.
TSMC and Morris Chang
Finding a foundry
Riva demand outran the European fab, so Huang turned to TSMC, by consensus the best independent chipmaker, with a huge complex in Tainan. As a foundry, it manufactures but does not design; Witt credits the rise of foundries with a surge of innovation, since upstarts could try radical designs without owning factories. TSMC's precision came from a "militarized" hierarchy and a "996" schedule (9 a.m. to 9 p.m., six days a week).
After unanswered voicemails, Huang mailed a personal letter to CEO Morris Chang, expecting nothing. Chang phoned back at day's end while staff were gaming noisily; Huang's polite "hold it down" was, per a veteran engineer, an expletive-laden order to be quiet. Chang, running a far bigger company, liked the near-bankrupt young CEO at once for being "open and forthcoming and candid," and a contract soon followed.
- Chang's path. Born in China in 1931, came to the US as a teenager, long a senior Texas Instruments executive, passed over for the top job in the 1970s (some blamed anti-Asian bias), then moved to Taiwan and built TSMC into Asia's largest tech company.
- Shared outsider status. Both were Chinese immigrants in an industry run almost entirely by white men. Asian employees were overrepresented in Valley firms but scarce at the top: in 2010 Asian Americans held only 0.3% of corporate management posts versus 5%+ of the skilled workforce, what consultant Jane Hyun calls the "bamboo ceiling." Huang brushes it aside: he was "the only Chinese CEO of the time, but it never occurred to me."
A near-fatal start
In early 1998 TSMC misapplied a chemical step at the end of production, shorting many chips. Nvidia had sunk most of its working capital into the run; more than half the chips were scrapped, and it survived only by selling equity to circuit-board partners. Diercks: the near-bankruptcies were "not just a saying." Over time the partnership became hugely beneficial to both, especially as Nvidia's chips grew more complex.
Back to Taiwan
The relationship took Huang back to Taiwan for the first time since childhood.
- The fab. In the late 1990s he toured a TSMC plant: booties, gloves, an air shower to strip lint and skin, an airlock, then constant vertical airflow over light-printing machines so sensitive a footstep could spoil them. After weeks of layering, wafers were cut by a diamond wire saw into chips; a good year meant tens of millions of chips.
- The night markets. He returned to the street stalls of his childhood, where at about four a vendor cleaning a knife had cut his cheek; he still bears the scar. He never misses a bowl of beef noodle soup on a plastic stool, and was already imagining a transpacific marriage between his American team and Taiwan's manufacturing.
Carmack and the twin pipelines
The customer who mattered
John Carmack, lead programmer of Doom and Quake, felt let down by hardware makers. A "code surgeon," he worked at the level of individual bits: rather than use stock lighting algorithms, his team hacked a variable's address structure to compute lighting, code one commenter called the most beautiful ever read. Thin, blond, nasal and socially awkward, he coded in darkened rooms 14 hours a day for weeks.
- Quake. The first best-selling 3D shooter, with polygonal monsters and sound effects by Trent Reznor; Quake II followed (Witt admits it cost him years of college).
- Deathmatch. Multiplayer combat; rendering dozens of players at once still strained hardware. Carmack wanted a faster Quake so players could "frag somebody."
- Inside Nvidia. The Sega consoles sat idle; staff were so hooked on Quake that Huang banned daytime deathmatch. Dell and other PC makers now bundled Nvidia cards in new machines, bypassing retail.
- Kirk's insight. Whoever rendered this one game best would win the graphics wars, so Huang ordered a chip built for Carmack: a custom Stratocaster for programming's Jimi Hendrix.
Twin shaders, two answers
Carmack wanted multiple pixel shaders (the algorithms colouring each pixel), so a wall could be lit first and then splattered with blood. That doubled the computation.
3dfx (the market leader). Put two graphics chips on one card. Works for two light sources, but each extra light means another chip, and the board runs out of space.
Nvidia (Kirk's team). Split shading into two datasets and run the same instruction on each pipeline simultaneously, on one chip. Graphics repeats the same calculations endlessly, so it suits this parallel computing, and pipelines can be multiplied on silicon.
Huang's first-principles reasoning
Huang was wary: the Valley was "littered with corpses" of parallel-computing firms, and he insists no parallel company except Nvidia ever succeeded. But he reasoned forward: Carmack would never stop at two. Imagine an arena of lights, guns firing, a spaceship crashing, twin suns; a chip per light would exhaust 3dfx's board. Only multiplying pipelines on one die could meet future demand. Then he did his homework on why parallelism kept failing.
- The graveyard. Seymour Cray's parallel supercomputers were floundering on cost and complexity; Larry Ellison (Oracle) had poured millions into nCube, failing by the late 1990s.
- The difficulty. Parallel programming meant juggling multiple memory banks and a steep learning curve.
- Moore's Law. Intel's serial CPUs did one calculation at a time but doubled in power roughly every 18 months, after Gordon Moore's 1965 prediction. Outside niches like weather and high-energy physics, one CPU sufficed. Bill Dally's summary: rewrite a million lines of code for a parallel machine, or wait two years for a CPU twice as fast.
Why graphics escaped Intel
In 1993 the graphics card was one slot-in among many (sound, networking, printer cards). By the late 1990s Intel had folded sound, networking and printing into the motherboard. Graphics cards alone survived because their appetite was unbounded: audio stops needing power once it reaches CD quality, but 3D is never finished "until you were living inside the Matrix." Analyst Hans Mosesmann: Intel missed how far graphics had to run; demand was basically infinite, so "twice as fast" would never be enough.
Moore's Law kills any task with a ceiling; it cannot kill a task without one. Because realistic 3D has no ceiling, graphics was the one market where parallel hardware could beat waiting for the next CPU, and that gave parallel computing its first durable commercial home.
Riva TNT
The Riva TNT launched in June 1998; TNT meant "twin texels," two pixel pipelines under a sophisticated switching mechanism. Carmack called it "the perfect card," tuned Quake III: Arena for it and told fans Quake played best on Nvidia. He also saw Huang's Christensen prophecy coming true: the TNT was "better, in many cases, than a $10,000 computer," a gaming base for attacking workstations.
- Brand building. Kirk poached from SGI too, including Dan Vivoli, an engineer turned marketer, who used the TNT to make Nvidia a brand that gamers fetishised. Huang's reading pile filled with marketing textbooks.
- Selling immersion, not parallelism. Nvidia never advertised the parallel design; it sold the dream of living inside the computer. The "Matrix" idea came from William Gibson's Neuromancer (1984), not the 1999 film; in a 2011 Paris Review interview Gibson said arcade kids seemed to want to be inside the games.
- An unofficial mission. Huang does not believe in mission statements, but Witt suggests Gibson's line would serve: total immersion, reality falling away. Huang reached by deduction what Gibson read in body language.
The TNT's twin pipelines were more than a new arrangement of transistors: Witt closes the chapter by calling it a change to all computers, buried so deep that Huang, Kirk, Carmack and Vivoli all missed it in 1998.
The 1998 toolkit at a glance
| Lever | Source | What it gave Nvidia |
|---|---|---|
| Control | Priem demoted, Kirk promoted | One technical line, set by the CEO |
| Talent | Poaching from failing rivals and SGI | The "geniuses" of the dying start-ups |
| Strategy | Christensen's disruption | Low-margin gamers as a base to climb from |
| Manufacturing | TSMC and Morris Chang | Scale and precision without owning a fab |
| Customer | Carmack and Quake | A design target that forced parallelism |
| Brand | Vivoli's marketing | Immersion as the promise, not the specs |
- TSMC's name. Witt calls it "Taiwan Semiconductor Manufacturing Corporation"; it is "Company." Chang did not merely "take control": he founded TSMC in 1987 at the Taiwanese government's invitation.
- Moore's Law. Moore's 1965 paper predicted doubling every year (revised to two years in 1975); "18 months" is a later popular gloss.
- The "beautiful code." The famous Quake lighting trick (the fast inverse square root in Quake III) is widely traced to others, including Gary Tarolli of 3dfx, not to Carmack himself.
- "Not one, zero." Huang's claim that no parallel firm ever succeeded is his recurring talking point, not a neutral history.
Jellyfish
Witt leaves Nvidia to introduce the book's second underdog technology, the neural network, through a 1997 backgammon match in Dallas that almost nobody noticed. The Jellyfish program, a neural net that taught itself by playing millions of games against itself, was the first to surpass humans at a game, yet the approach stalled because researchers lacked enough data and computing power. The chapter ends with Jellyfish's creator giving up, unable to imagine what could make neural nets work.
The Dallas match, 1997
A few months after IBM's Deep Blue beat Garry Kasparov in 1997 (front-page news; the New York Times ran Kasparov with his head in his hands), a backgammon match in a suite at the Bristol Suites hotel in Dallas drew no press at all. Witt's contrarian claim: this obscure contest, not Kasparov's defeat, marked the dawn of the new machine age.
- The setting. A room of proposition gamblers, obsessives who played Go, poker, Scrabble, bridge and chess at elite level, trading cash and side bets on nearly every roll.
- For humanity. Nack Ballard, several times world number one, who studied six hours a day and once played 84 hours straight, with a Zen-like calm Kasparov lacked; and Mike Senkiewicz.
- For the machine. Malcolm Davis, himself a master, convinced software had overtaken humans. He played the computer's moves as its agent for $200 a point, risking $100,000 or more if he was wrong. Controversial recommendations popped up in a clunky dialogue box and set off fresh side bets.
- The program. Jellyfish, a neural net: instead of executing human-written rules, it passed the position through a grid of artificial neurons whose synapses were a vast matrix of numbers, the weights, and returned an evaluation.
Why neural nets were outcasts
In 1997 mainstream computer scientists saw neural nets as toys, and AI as a career graveyard.
1940s "Nervous nets" built as power-hungry electromechanical hardware copying neurons and synapses.
1969 Marvin Minsky (MIT), in Perceptrons, shows a single layer of neurons cannot perform even a simple logic operation; funding evaporates and the machines are dismantled.
1970 Seppo Linnainmaa discovers backpropagation (not applied to neural nets).
1974 First AI winter as symbolic AI stalls. Paul Werbos rediscovers backprop and pitches it to Minsky, who (per Werbos) nearly throws him out.
1980s Expert systems boom and stock bubble, burst after the 1987 crash; Japan, the UK and the US spend billions on national AI programmes judged largely wasted.
1986 David Rumelhart, Geoffrey Hinton and Ronald Williams publish backpropagation for multilayer nets.
c.1990 Gerald Tesauro makes reinforcement learning work (TD-Gammon).
1994 Jellyfish, the first neural net sold to the public.
1997 Deep Blue beats Kasparov; the Dallas match.
Through the 1970s and 1980s a band of renegades kept faith that neural nets could be rebuilt in software and that multiple layers would overcome the single layer's limits. Colleagues thought them misguided or mad.
- Measure the error. Run the network and compare its output to the right answer.
- Compute the gradient. Use multivariable calculus and linear algebra to get partial derivatives showing which neurons are most to blame.
- Adjust. Nudge each weight in the opposite direction of its contribution to the error.
- Repeat. Run the loop any number of times, across all layers.
Witt's emphasis is on what backprop meant: like a brain forming synapses when it learns a task, it let networks fine-tune themselves on new information. Software no longer had to be explicitly programmed; it could make its own rules and, in his word, evolve.
TD-Gammon: the machine that innovated
Why backgammon
In the late 1980s Gerald Tesauro at IBM passed over the prestigious chess group to tackle backgammon, a low-status race game on a 24-point board driven by two dice. Gamblers liked its unpredictability; Tesauro liked that simulated dice could generate hundreds of thousands of games, i.e. training data. He worked almost alone, since few took either neural nets or backgammon seriously.
Self-play
Training the net to imitate expert humans produced little. Around 1990 he stripped out all strategic knowledge, leaving only the rules and randomly weighted neurons, and had the program play itself. This was reinforcement learning, and Tesauro was the first to make it work.
| Games played | What TD-Gammon learned |
|---|---|
| 0 | Moves checkers aimlessly |
| A few thousand | Lone checkers are bad, stacked pairs good: a competent beginner |
| Tens of thousands | Builds walls from multiple stacks |
| 200,000 | Strong intermediate play |
| Millions (by 1995) | Strategies never seen by humans |
What it discovered
- Openings. Humans risked too much chasing an early edge; conservative openings were better.
- Endgames. It often passed up a certain win to try to double its score, which humans thought reckless.
- Middlegame. Subtle moves experts understood only after long reflection. Witt: it was no longer just learning, it was innovating.
Chess computers. Brilliant where variations can be calculated out; weak in vague positional play. A dumb machine that calculates faster than humans.
TD-Gammon. Strongest in vague positions where judgment, not calculation, decides. A smart machine that learns from experience much as humans do.
IBM never commercialised it: a business-server vendor saw no point in selling backgammon software to a few hundred customers.
Fredrik Dahl and Jellyfish
The niche was filled in 1994 by Fredrik Dahl, a Norwegian with eclectic passions (backgammon, chess, simulated tank battles, jiu-jitsu, mushroom foraging). He worked for Norway's defence establishment modelling a hypothetical Soviet invasion, inspired by the 1983 film WarGames, and joked that losing his funding when the USSR collapsed was "a terrible time." His doctoral net had predicted combat outcomes by fighting millions of simulated battles against itself; ported to backgammon, it surpassed Tesauro.
- First on sale. Jellyfish was the first neural net sold to the public, via Dahl's primitive website.
- The name. An homage to the jellyfish's nerve net: the program had about a hundred "brain cells," roughly jellyfish level. A hundred cells sufficed to master backgammon, as they sufficed for half a billion years of jellyfish survival.
Jellyfish trained on many millions of games, yet the finished net fit on a 3.5-inch floppy disk. This set an early, lasting distinction: training (how the machine learns) is computationally expensive; inference (using what it learned) is cheap. Witt's analogy: the three-pound brain does inference; hundreds of millions of years of evolution did the training.
The result in Dallas
Each human played 300 games for statistical fairness. Ballard beat the machine by 58 games, winning $11,600; Senkiewicz lost about as much; Davis broke even, and the match was called a tie. Later analysis showed Ballard had been lucky with the dice, and he knew his era was over. No human ever again challenged a backgammon program for money.
Why Jellyfish changed the game and Deep Blue did not
- Deep Blue. An expensive supercomputer using brute force humans could not copy; it left expert chess largely unchanged and was dismantled after 1997.
- Jellyfish. Cheap software on any Windows PC. Woolsey's New Ideas in Backgammon collected positions where it contradicted human intuition; the computer proved always right. Players came to be rated by how far their moves deviated from the computer's.
The wall
Witt calls Jellyfish the first neural net to surpass humans at any game. Dahl then evolved a poker net by reinforcement learning that beat everyone at two-player ("heads-up") limit Texas Hold'em, licensed it to a slot-machine maker, and it played all comers for real money on the Las Vegas Strip, unbeaten.
No-limit Hold'em broke the method. With unlimited bet sizes the synthetic dataset was far larger than the closed worlds of backgammon or limit poker, and the net made "reasonable plays" but never reached expert level.
- Opacity. Dahl had almost no idea how the bot worked: reading strategy from individual weights was like studying consciousness by looking at brain cells under a microscope.
- The plateau problem. Once a net hit a training plateau, as most did, there was rarely an obvious fix. This was the core academic objection to neural nets.
- Biology, not logic. Classical programming is orderly; tuning nets is like a biology experiment, where small changes have unpredictable effects.
- Everything tried. Dahl changed the evaluation function, the memory setup and the neurons' activation trigger, and even built a simpler data universe. None worked.
Dahl eventually gave up, shelved the bot, and took a job analysing medical data with conventional statistics. Witt notes that many neural-net critics were similar disillusioned apostates: early success, then years of mediocre results. Dahl never quite stopped believing, but he dismissed the approach "because I just didn't have the data," and could not imagine what would make it succeed.
Neural nets already had the algorithm (backprop) and a proof of power (TD-Gammon, Jellyfish). What they lacked was data and compute on a scale no one could then supply, which is exactly where Nvidia's parallel chips would later come in.
Witt credits Minsky alone for the 1969 critique; Perceptrons was co-written with Seymour Papert. Calling Jellyfish the first net to surpass humans is generous, since TD-Gammon reached near-world-class play first; Jellyfish was the first sold commercially. In 1986 Hinton was at Carnegie Mellon, not UC San Diego, though the backprop work began in Rumelhart's San Diego group.
Deathmatch
Between 1999 and 2001 Nvidia went public, coined the term GPU, destroyed 3dfx in a war of products and lawsuits, absorbed its best engineers, grew from 35 to 600+ staff and joined the S&P 500 in place of Enron. Huang's rule for a merchant chip firm was that standing still means being commoditized: "Not to gamble was the biggest risk of all." Witt also shows the cost: a reputation for ruthlessness, a messy code base, and a peak in wealth Huang would not see again for fourteen years.
The pro gamer's edge
Witt opens with Johnathan Wendel, "fatal1ty," one of the first professional gamers, to show what Nvidia's speed meant in practice.
- Profile. An athletic outlier among skinny pros (hockey, tennis, golf). At 15 he beat a hundred college players at a Quake tournament in Wichita (1996), then dropped out of college to game full-time, modelling himself on Tiger Woods ("if you win just one tournament, you'll be forgotten").
- Training. Deathmatch sparring 8 to 12 hours a day; dumped his girlfriend to practise more; ran miles for his reflexes; reaction drills (click when the screen turns green) cut his response time to 140 milliseconds.
- Results. Dozens of tournament wins in Quake III: Arena; the top earner in competitive gaming for seven straight years. His strength was reading opponents and setting traps.
One kill, frame by frame
Hearing an ammo-loving opponent approach a pile of ammunition, Wendel began firing a frame or two before the man appeared. His Nvidia TNT2 rendered Quake III at 60 to 70 frames per second, against 20 to 30 on commodity hardware, doubling the frames inside his 140 ms reaction window from five to ten. Each frame took about 200 million calculations. Frame after frame of damage, and the kill was his.
Wendel likened his Nvidia deal (free hardware for promotion) to Michael Jordan and Nike. Asked in 2024 about the parallel technology inside, he shrugged; he only knew "the frame rate on that thing was insane." He upgraded every cycle, and soon every pro did too, which is precisely the upgrade ladder Huang's business depended on.
IPO and the six-month cycle
Wall Street loved Nvidia's cadence: new cards every six months, twice as fast as any rival (a new line each fall for back-to-school, a refresh each spring). Flat-screen monitors boosted demand, and within a few years accelerators were standard in most PCs.
- Early 1999: IPO at a $600 million valuation, under six years after founding.
- $6 million: Sequoia's entry valuation, a hundred-bagger that covered many failed bets.
- $12: listing price on NASDAQ as NVDA; it doubled at once and reached $60 by year end.
- 3 million+ shares each held by Priem, Malachowsky and Huang; Huang was now worth hundreds of millions.
There was no celebration: Diercks recalls no champagne, no relief, not even congratulations. He kept an email Huang sent the day after the IPO:
- The fight. The TNT2 team must win "every single minute" inside Dell and Compaq for motherboard business, where S3's Savage 4 was doing well.
- The goal. Design wins to take share from ATI and keep S3 down.
- The target. 250,000 TNT2 units by April to make the quarter.
- The priority list. "There are three priorities: one, two, and three." Then: "Get it done."
Killing 3dfx
The GeForce and the invention of the "GPU"
Nvidia's nemesis was 3dfx, whose Voodoo cards had led the market for two years. In late 1999 Huang launched the Voodoo killer, GeForce ("Geometry Force"), powered by the NV10.
- Specs. 10 million triangles per second, with lighting that recoloured pixels to match movable light sources: "transformation and lighting" unified on one chip, a long-sought goal. Jon Peddie: it did "for $2 what a workstation could do for $2,000."
- The marketing coup. Dan Vivoli, treating a small budget as an opportunity, noticed buyers relied on about six independent reviewers. He told them GeForce was the world's first "graphics-processing unit," a term his team made up. Reviewers adopted it as a category and soon every accelerator was a GPU. Vivoli: "We invented the category so we could be the leader in it."
Nvidia. Four pipelines on one chip; a frantic six-month cadence; aggressive marketing (Peddie concedes "a few tricks in benchmarking") but also, he says, better engineering and better management.
3dfx. Maker of the dominant Voodoo line; its pricier Voodoo 4 prototype used four chips and still could not match GeForce. Perfectionists left behind by the pace; when a 3dfx founder publicly floated a truce to agree standards, Kirk "knew we had him... one of us had to die."
The lawsuits
Kirk's "brain extraction" brought legal trouble, since defecting engineers often carried proprietary ideas. Between 1996 and 1999 four rivals went to court:
| Plaintiff | Claim | Outcome |
|---|---|---|
| S3 | Patent infringement | Settled out of court |
| Silicon Graphics | Patent infringement | Settled out of court |
| Matrox | Nvidia encouraged staff to breach confidentiality agreements | Settled out of court |
| 3dfx | Patent infringement | Refused arbitration; bet survival on winning at trial |
Aug 2000 On a disastrous earnings call, 3dfx CEO Alex Leupp forecasts a $100M+ loss in one quarter. An hour later Nvidia announces a countersuit with questionable patent claims; many suspect a nuisance suit meant to drain 3dfx's legal budget.
Sep 2000 Preliminary ruling for 3dfx; Nvidia's countersuit rejected entirely. Nvidia's lawyers stall the payout.
Late 2000 Leupp offers 3dfx to Intel, the favourable ruling its only asset; Intel and everyone else decline. 3dfx offers itself to Nvidia, which buys selected assets for $70 million; the suit is dropped.
Dec 2000 3dfx's Austin staff are locked out, escorted from the building, herded through a drainage ditch and across a road to a warehouse of unsellable cards, and laid off.
Winning the enemy's engineers
Inside 3dfx Huang was called "Darth Vader" (and worse, says Peddie). The rumours were near-mythic: he poached people, lifted ideas, spun reviewers, kicked the fallen. Mostly, Witt says, they hated him for beating them. Peddie, a friend: Nvidia made enemies among partners and suppliers too, and Huang "was ruthless."
- The valuation. Internal documents show Huang valued 3dfx's best engineers at $1 million a head, reflecting both their worth to Nvidia and the value of denying them to rivals.
- The list. From about 500 staff, Huang and Kirk chose 120. Over the weekend rumours spread that Nvidia was a sweatshop run by a screaming tyrant; some vowed never to join.
- The offer. Programmer Ben Garlick, needing a job before Christmas, was among the first called back. On Monday Huang personally offered him a 20% raise, benefits and stock options. Garlick stayed seventeen years.
- The yield. 106 of the 120 accepted.
Garlick's verdict: Huang is "a good person at heart who had to be ruthless," unlike CEOs who are ruthless at heart and pretend to be good.
Garlick was appalled by Nvidia's code: "cancer," growing and mutating without efficiency, with midnight-crunch hacks under critical systems, a chaotic tool chain and nobody caring about anything but the next tape-out. 3dfx had written clean, well-commented code, and "in the time we spent making it clean, we went out of business." He came to admire the method ("iterate, iterate, iterate, execute, execute, execute"): tech debt is the battle scar of the survivor.
Culture of the grown-up company
With the 3dfx hires Nvidia passed 600 employees, up from 35 four years earlier, and moved to an eleven-acre campus in Santa Clara: curved glass-and-steel buildings linked by skyways beside the expressway. It no longer smelled of takeout; it smelled of nothing. Modernist respectability had arrived.
- Office Space. Mike Judge's 1999 satire of cubicle life drew on his time at a rival graphics-card start-up a couple of miles away. Nvidians, Witt found, felt no such ennui: corporate life suited them. Huang hunted down bureaucratic waste; Witt imagines a manager fussing over TPS cover sheets being "crucified" in the cubicle pit.
- Work hard. 3dfx's motto had been "Work hard, play hard"; at Nvidia it was just "Work hard." The six-month cycle meant "almost nonstop deadlines and a perpetual sense of being behind schedule." Others valued knowing the place would stay in business.
- Dress and decency. Game studios prized slobbishness (Karen Huaulme, a 3dfx hire, once found id Software's office by following the worst-dressed man in sight). Nobody wore flip-flops at Nvidia. Huaulme, used to men questioning her credentials or getting handsy elsewhere, found Nvidia, with few women then, a place of relative safety.
Kirk's interviews
David Kirk had once laid off 100 engineers at his own start-up, so painful he quit days later; he resolved to avoid layoffs by hiring selectively. Nvidia's multi-round consensus interviews had decayed into stock questions ("greatest weakness?", "why are manhole covers round?"). So Kirk, normally gentle, deliberately gathered his engineers and screamed at them: they had hired someone to do half their work without checking he could. Then he re-interviewed the candidate in front of them.
- Start soft. "Are you familiar with how to draw a triangle?"
- Escalate. How do you draw its edges? What if a coordinate is zero and you cannot divide by zero?
- Find the limit. Keep raising difficulty until the candidate's knowledge runs out, then ask a question they cannot know.
- Grade the response. Lying or bluffing: automatic fail. "I don't know": usually a fail. Pass: realise it is a Socratic dialogue, walk back through earlier answers and reason forward to a new one during the interview.
- Payoff. Fifteen minutes revealed more than eight hours of structured interviews, and taught the staff how to ask.
Kirk, like Diercks, thinks Huang's public yelling is premeditated: "It annoys people, but it does work." The audience matters, he says; Huang never yells at someone in a hallway, because the point is for everyone to learn the lesson.
No jobs for family
Huang's parents pressed him to hire his brothers; he refused, saying he could not justify it and they did not fit the culture, which caused tense conversations.
- Jeff (older). Held various engineering jobs; Horstmann recalls him "supervising" while Jensen and Horstmann built a deck: "He couldn't even hold a nail gun."
- Jim (younger). Followed Jensen to OSU for the same EE degree, then played it safe at Intel for thirty years building software tools; solid and hardworking, but in Horstmann's view without the appetite for entrepreneurial risk and failure.
Instead he gave them real estate. In the early 2000s he sold some Nvidia shares, bought land in Los Altos Hills (now the third-wealthiest US zip code) and, with Lori overseeing, built a 6,000-square-foot, five-bedroom, seven-bathroom mansion with a pool. He bought two Ferraris (his and hers), began collecting expensive whiskey, and gave the old house, deck included, to Jeff. The "just so" streak of the very rich showed when he noticed the garden doors did not line up with the pool house, and had the pool house moved eighteen feet at great expense.
Market leader, and still paranoid
Expansion and the bubble
- Quadro. Alongside GeForce, Nvidia launched Quadro professional GPUs for modelling and animation: the Christensen move against Silicon Graphics, as foretold.
- Old bosses. In 2000 Huang hired Tommy Lee, his first manager at LSI.
- Real business, frothy market. Unlike dot-coms, Nvidia shipped products and made real profits, but it traded in a tech bubble. In early 2000 a deal to build the chip for Microsoft's unnamed console sent the stock above $100, triggering dares from an offsite: Malachowsky pierced an ear, Priem shaved his head bar a patch dyed green into the Nvidia logo, and Huang had the logo tattooed on his upper arm (and complained of the pain for years).
The duopoly
By 2001 Nvidia sold $1 billion of GPUs a year. Only ATI, from suburban Toronto, kept pace with its Radeon: also fan-cooled, also parallel pipelines, made in the same TSMC fab, led by cofounder Kwok Yuen Ho, a fiercely competitive immigrant whose firm had also nearly gone bust. The market settled into a static duopoly, just as Huang's 1998 "rule of three" anticipated, with GeForce and Radeon trading the lead for two decades (GeForce has since pulled well ahead).
Huang stayed wary. S3 led in 1996 and was gone by 1999; 3dfx led in 1998 and was gone by 2000. Nothing guaranteed Nvidia would not follow. Andy Grove's Only the Paranoid Survive sat in his office.
- Fashion, not Coca-Cola. Coke protects a winning formula; in chips, if today's product resembles yesterday's you have failed. Design software, photolithography machines and architectures are all reinvented every few years (Witt quotes Marx's "all that is solid melts into air" and Grove on exposure to "the winds of change").
- Rising antes. Nvidia's first chip had about 1 million transistors; by 2000 its chips had 20 times that, fan-cooled and in half the space. Each new manufacturing generation raises the cost of staying in.
- All-in or bleed out. Standing pat lets your stack dwindle; survival means finding a promising hand and shoving everything in, again and again. Every executive in the sector is a gambler.
- The merchant's exposure. An asset-light firm is "just a collection of engineers" in an office park. Without hard-to-copy technology, Asian manufacturers knock off the chips and the firm disappears.
The programmable-shader gamble
Graphics of the day often looked plastic or rubbery. Kirk wanted programmable shaders to give developers better lighting tools, which meant sacrificing profit for infrastructure few developers would use at first; his pitch was that it would cost more for a while, then everyone would want it and rivals would have to chase.
Huang was sceptical, as he had been about parallel pipelines, until he weighed the cost of not doing it. The one certainty was that more transistors were coming, making today's graphics cheap and easy for anyone; only continual reinvention would prevent Nvidia being "commoditized out of existence." Hence the chapter's rule: not to gamble was the biggest risk.
The peak of 2001
- June 2001. First GeForce cards with programmable shaders.
- November 2001. Microsoft's Xbox launches with Halo, a shooter in the Quake and Doom tradition set on a ring-shaped world with naturalistic lighting rendered by Nvidia hardware.
- The payoff. Halo's success pushed Nvidia's value above $20 billion; two weeks later it joined the S&P 500, replacing Enron.
- Huang at 38. One of the index's youngest CEOs, eight years from a diner booth to America's top 500, having beaten or absorbed every rival but one, and briefly a paper billionaire. It would be fourteen years before he was that rich again.
Witt accepts Vivoli's claim that Nvidia coined "GPU" with GeForce. Sony had already called the PlayStation's graphics chip a "GPU" in 1994; what Nvidia did was popularise the label for PC cards and define it around on-chip transform and lighting. The account of the 3dfx war also leans on Nvidia-side sources (Kirk, Vivoli, Garlick after 17 years there), while the "nuisance suit" motive is reported as speculation, not established fact.
ATI was bought by AMD in 2006, so the GeForce-Radeon rivalry became Nvidia versus AMD. By 2025 analysts at Jon Peddie Research put Nvidia's share of discrete PC graphics cards at over 90%, confirming Witt's "pulled well ahead." Console work moved mostly to ATI/AMD after the original Xbox and Sony's PS3, but Nvidia supplies Nintendo's Switch (2017) and Switch 2 (2025).
The Compulsion Loop
From mid-2001 to fall 2002 Nvidia's stock fell more than 90% under four simultaneous blows: an SEC investigation, a broken Xbox partnership, slow uptake of programmable shaders, and then the "dustbuster" GeForce FX. Hardcore PC gamers, hooked on a deliberately addictive compulsion loop, rescued the company and funded Huang's next long-shot bets. Along the way Witt examines how Huang's mixture of fear, guilt and tenderness kept staff loyal, and why cofounder Curtis Priem sold every share.
The man at the table
Witt pictures Huang in 2002 grinding out forehand loops at a ping-pong table while everything at work went wrong: crashing stock, a defective fan, a collapsing Xbox deal, a fraud probe. He had returned to the sport after phoning Joe Romanosky, his old Paddle Palace partner, for the first time in nearly twenty years.
- Old memories. Romanosky recalled a college tournament in a state penitentiary where Huang, winning easily before an audience of prisoners, slipped in trick shots disguised as normal play, Harlem Globetrotters style.
- Back to the game. Huang sent a GPU to Romanosky's son (who built a PC around it), received his favourite paddle in return, and hired a former Olympian as coach.
- Home life. Romanosky, a Boeing engineer in San Diego, and his wife began visiting Jensen and Lori. In the garden by the relocated pool house Huang had installed a teppan grill and practised Benihana tricks (tossed rice, an onion-ring volcano, shrimp caught in a hat, mostly dropped). He shared his growing whiskey collection but declined a camping trip to the Sierras.
Romanosky never saw "Darth Vader": at dinner Huang talked family, kids and interests, never business, and came across as warm, authentic and mischievous. Witt reads this as an unusual ability to compartmentalise the obsessive executive from the man at home; Romanosky: if that side exists, "he's turned that side of himself off."
Four blows at once
The SEC and the market
- 90%+ fall in Nvidia's share price, summer 2001 to fall 2002, taking Huang's wealth with it.
- January 2002: the SEC opens an accounting investigation, in the climate of the WorldCom and Enron frauds.
- July 2002: Nvidia restates three years of earnings; CFO Christine Hoberg is let go.
- $3 million: unrecorded expenses the SEC later accused Hoberg of hiding from auditors; she paid a $600,000 fine without admitting wrongdoing.
- ~50%: the S&P 500's loss amid the dot-com bust, 9/11 and Enron's bankruptcy.
Board member Tench Coxe says the board never lost confidence in Huang and calls the probe "a fishing expedition." Witt grants he had a point: the restated earnings showed Nvidia more profitable than reported.
The Microsoft rift
The official causes were pricing and intellectual property, but Witt adds Nvidia's growing sense of entitlement. Its staff were openly elitist; they really were the best, but the pride could sound like narcissism.
At Microsoft's pre-launch Xbox banquet, with Bill Gates speaking, Nvidia's technicians were seated at the back alongside the makers of the rubber feet that stop the console sliding off shelves. Kirk, "joking" decades later, still sounded like a slighted bridesmaid; Witt notes that Kirk's jokes, like Huang's, carry barbed truths. Whether or not it was a planner's slip, Kirk took it as a signal that Microsoft would not be held hostage by a hardware supplier. Soon Microsoft demanded large volumes and price cuts, Nvidia said its contract did not oblige it, the dispute went to arbitration, and Microsoft used ATI for the next Xbox.
The shader gamble stalls
The worst problem was slow adoption of programmable shaders: the bet only paid if developers learned a new coding language, Cg. Nvidia promoted it with "Dawn," a scantily clad CGI fairy with butterfly wings who graced the cover of The Cg Tutorial, a textbook of ten chapters on vertex transformations, pixel pipelines and sample code, with homework. Uptake was slow.
The dustbuster
In early 2003 Nvidia shipped the GeForce FX: slow at rendering and nicknamed "the dustbuster" for its overactive fan. Reviewers and buyers panned it, including Huang's thirteen-year-old son Spencer, who left a harsh magazine review for his father with a Post-it: "Dad, I think you need to kick it up a notch."
Fear, guilt and tenderness
The post-mortem
Huang had product managers present, to a few hundred people, every decision that led to the FX fiasco, then screamed at them for nearly an hour. QA engineer Sharon Clay: "terrifying but cathartic."
- Guilt, not only fear. Huang often spells out that letting customers down means letting colleagues' families down. "I think I'm driven as much by guilt as anything else," he told Witt.
- The Black Spot. Clay expected her next review to say "RI" (Requires Improvement), Nvidia's equivalent of a death sentence.
- The fix. For the FX she had run four or five quality tests; for its successor she ran a hundred, then thousands, a solution she could not have imagined until "properly, uh, pushed."
- No scapegoats. Nobody on the QA team was demoted. Marketing released a self-mocking video turning the card into a leaf blower, and the revised GeForce shipped to acclaim six weeks later.
Many staff compare Huang to a general or a pro football coach. One: he may not yell more than other Fortune 500 CEOs; "it's not really his job to be your friend. It's his job to push you beyond where you think you could ever go."
The other side
Of the 100+ current and former employees Witt interviewed, almost all had a tender story about Huang.
- Tim Little. Exhausted after weeks away from family, he resigned in response to an email titled "Drag Your Sorry Ass Across The Finish Line." Days later, around 2 a.m., Huang sat beside him at the simulator, talked about his own sacrifices and late nights, admitted he was not sure it was all worth it, offered the job back, and thanked him when Little declined. Little calls it the high point of his time there.
- The "refund" employee. The man once told to repay his whole career salary was later diagnosed with a serious illness; Huang offered to pay for his treatment out of his own pocket.
- Ben Garlick. Leaving for a start-up as a frontline manager of ten in a company of thousands, he got a knee-to-knee personal plea to stay from a CEO he did not think knew his name.
Witt's synthesis: Huang's mix of public humiliation, private kindness and moral guilt is a seductive and powerful motivator. Clay put it simply: "You felt like you couldn't let him down."
Priem cashes out
Priem's responsibilities had been minimal for years; eventually he stopped coming in (Kirk: after a clash with another executive, "I think he decided he was fired").
- Philanthropy. He sold stock to fund gifts mainly to Rensselaer Polytechnic Institute, joining its board; the money built a $200 million performing arts center and later bought a quantum computer. He says he is on track to give away half a billion dollars in his lifetime.
- The ranch. A $6 million ranch in the Diablo Range east of Oakland, with cattle and a view of all five major Bay bridges; a Gulfstream jet for trips to RPI, offset by green-tech investments and taking the ranch off the grid; a messy divorce; a search for a way to "repair the earth."
- The sale. In chunks between 2004 and 2006 he sold every Nvidia share: "That's why the stock flatlined... We basically sold into strength."
The forgone fortune. Held untouched, the shares would make him worth more than $100 billion, among the richest people alive, by Witt's reckoning.
Priem's view. No regrets: holding meant 99.9% of his net worth in one volatile stock of a firm he no longer worked for. Channelling George Bailey, he argues the gains did not vanish but went into pension plans and people's houses, "like I contributed $100 billion to our economy," and he would not know how to give away a $100 billion foundation anyway.
Saved by the enthusiasts
The Golden Age of PC gaming
Even with the stock in the toilet, Nvidia was shipping some of the most complex silicon ever made. Combined with home broadband and mature multimedia PCs, it enabled what critics call the Golden Age of PC gaming: Call of Duty, Half-Life 2, The Sims, World of Warcraft. One nostalgic commenter dates the peak to 2000 to 2005.
Witt asks whether this was the best use of such technology, and does not romanticise it:
- Toxicity. PC gaming culture spawned 4chan and later the Gamergate harassment campaign; PC players called console players "peasants" and themselves the "PC master race."
- Arrested aesthetics. Graphics pioneers despaired that an extraordinary tool was used mostly for monsters, gunfights, car chases and gore. Peddie laments "the triviality of some of the results."
The compulsion loop
Commercially, though, PC gamers were the best customers: in Witt's blunt word, addicts. Games, like casino slot machines, hand out rewards on a randomised schedule.
- The loop named. In 2001 John Hopson, a researcher at Halo's studio, described the "compulsion loop": upgrade the character, send it on a quest, reward it with loot, repeat.
- Addiction signs. Researchers saw hardcore gamers binge, suffer withdrawal, lie to family about time played, and delete games only to reinstall them the next day.
- Medical recognition. In 2013 the American Psychiatric Association's manual added "internet gaming disorder," noting young men's vulnerability, with symptoms such as failing at school or work and declining personal hygiene.
- The other view. For many the games were rich worlds of meaning and challenge; World of Warcraft players made friends across the globe.
Enthusiasts and their rigs
About a quarter of gamers played two hours a day or more. Nvidia called them enthusiasts: often raised on Nintendo, graduating to PCs as adults. The PC market had fewer customers than consoles but they spent far more. With Nvidia's encouragement the custom gaming PC, or "rig," became what the muscle car was to gearheads: thousands of dollars of aftermarket parts, transparent cases with coloured lighting, photos posted to forums boasting of overclocked motherboards and GPU speeds.
Hard to value, easy to doubt
As the market matured the stock recovered; in fact Nvidia never stopped growing. By early 2004 it had 1,000+ employees and record earnings.
- The analyst's headache. Past figures said little, because Huang reinvested every dollar in speculative technologies that would either revolutionise computing or flop.
- A mixed record. By the mid-2000s his bets looked slightly worse than breakeven: wins with the GPU and parallel processing, misses on several other initiatives, leaving a company successful on paper whose stock chronically underperformed.
- The real asset. What mattered was not profit or revenue but the obsessive CEO and his long shots: "Either you believed in him or you didn't." Witt ends by flagging that Huang was about to make his craziest bet yet (CUDA, in Chapter 9).
The 2013 APA entry placed internet gaming disorder in a section for conditions needing further study, not as an official diagnosis; the WHO did recognise "gaming disorder" in ICD-11 (adopted 2019). Witt's "compulsion loop" critique also sits awkwardly with his own evidence that the same enthusiasts later bankrolled AI; he notes the irony but does not resolve whether Nvidia bears any responsibility for the addictive design it profited from.
Witt's "more than $100 billion" was computed near Nvidia's $3 trillion valuation. With the company passing $5 trillion in October 2025, the same stake would be worth roughly $170 billion, more than most of the world's billionaires own.
CUDA
In the mid-2000s Huang turned the gaming GPU into a cheap scientific supercomputer through CUDA, a free software platform launched in late 2006, betting on a market that had no customers yet. CUDA rested on John Nickolls's prediction that transistor scaling would stall around 2005, forcing programmers off serial CPUs and onto parallel chips, and on getting gamers to pay the development bill. The bet cost Nvidia money, focus and Wall Street's patience just as a manufacturing scandal pushed the company into its first loss.
Ian Buck's wall of Quake
Where deathmatch players such as Johnathan Wendel lowered their resolution for frame rate, Stanford graphics student Ian Buck went the other way. In 2000 he split the rendering of Quake III across 32 GeForce cards feeding eight projectors, a wall-sized display that he calls the first 8K gaming rig.
- The arithmetic. Rendering thirty 8K frames by hand, working around the clock, would take a person about 16,000 years; the array did it every second.
- The price. The rig cost about $20,000, trivial by high-performance computing standards. Buck had built a low-budget supercomputer by accident.
- The obstacle. Nvidia's bundled software spoke only "triangles." To use the horsepower for anything else Buck had to hack it, so he mastered Nvidia's shading textbook (the one with the pixie on the cover) and in doing so lost interest in graphics altogether, drawn down into the circuits beneath.
- The awe. Witt portrays Buck as intense and permanently astonished at the volume of arithmetic: 60 gigaflops (sixty billion operations a second) from a single card.
Brook (2003)
With a DARPA grant Buck formed a research group and in 2003 released Brook, an open-source language that let scientists disguise mathematical workloads (galaxy formation, the ignition of a nuclear weapon) as graphics jobs. The visible output was meaningless triangles; the useful output was the scientific calculation the GPU performed while drawing them. It still required real graphics expertise ("you really had to understand computer graphics to be able to hack those triangles").
Brook nonetheless made parallel computing accessible. Academics bought GeForce cards in bulk and chained them for financial modelling, weather simulation, high-energy physics and medical imaging. The gaming card had become a jury-rigged lab instrument, and Huang noticed the new customer. Around 2004 he invited Buck to Nvidia to "do it for real."
Nvidia becomes a laboratory
Buck's path was typical. In 2005 Silicon Graphics, beaten by Nvidia's Quadro workstation line, was delisted from the New York Stock Exchange; its headquarters became the Googleplex. As with 3dfx, Nvidia absorbed the refugees and doubled to about 2,000 employees, of whom 1,200 (60%) were classed as R&D. Outsiders still saw a slightly comic gaming-hardware maker; insiders felt they worked in a scientific laboratory.
Bill Dally, chair of Stanford's computer science department and a career-long advocate of parallel computing, had been watching the rising "arithmetic intensity" of Nvidia's chips and wrote papers praising them as a possible backbone for a new kind of computer. In 2003 Huang came to his office to offer a consulting role, reversing years in which Dally had pitched parallelism to uninterested executives. Dally rates Huang's technical grasp as deep enough to be "a step ahead of you" on your own speciality. He had also heard about Huang's volatility and Nvidia's harsh culture; Huang, apparently anticipating this, brought a signed paper check made out to Dally to their first meeting. Dally signed on.
The secret project
Buck, Dally and dozens of others were being recruited to a secret effort Witt names Compute Unified Domain Architecture, a deliberately vague title. The idea: open the gaming GPU's parallel circuits to scientists directly, ending the need to smuggle maths through triangles. Dwight Diercks pictures a game card with a switch on it: flip it and the card becomes a supercomputer.
- John Nickolls. Inventor of CUDA; ex-cofounder of parallel start-up MasPar; ran the GeForce science pilot.
- Ian Buck. Brook's creator; built CUDA's machine-code layer and later held patents on assembly-level techniques.
- Bill Dally. Stanford CS chair, parallel-computing evangelist, Nvidia consultant from 2003.
- Arjun Prabhu. Director of hardware engineering; laid out the CUDA-core chips.
- Bas Aarts. Dutch developer who wrote the first CUDA compiler.
- Brett Coon. Early CUDA engineer; source of the "gamers pay" verdict.
- David Kirk. Chief scientist; cowrote the CUDA textbook; went part-time in 2007.
John Nickolls and the death of Dennard scaling
The inventor was John Nickolls, a downhill skier and model-train hobbyist whose office walls held framed microchips. At MasPar Corporation, one of the failed "massively parallel" start-ups, he had tried to impose TSMC's 996 schedule (twelve-hour shifts, six days a week). After MasPar sank he remained convinced that physics itself would eventually force the industry to go parallel.
The physics was Dennard scaling, the rule that had made computers faster every year: as transistors shrank, they kept handling power efficiently. Nickolls calculated that it would break around 2005.
- Scale. The next generation of lithography machines would print transistors about 100 atoms wide: some 6,000 times thinner than a hair and 700 times thinner than a red blood cell.
- Failure mode. At that size conductivity degrades and transistors leak current into neighbouring circuits, so clock speeds can no longer climb.
- Industry denial. Intel was still forecasting smooth gains all the way down to single-atom components, which Nickolls thought impossible.
In early 2003 Nickolls sent Huang an unsolicited letter explaining, from electrical first principles and without exaggeration, why Intel's long dominance was ending. Witt's footnote gives the formal rule: Dennard scaling (Robert Dennard and IBM colleagues, 1974) says power density stays constant as transistors shrink. Huang later said it was Nickolls who "convinced me that Moore's Law was truly dead," and credits him heavily for what Nvidia became.
The man
Huang hired him to lead a pilot project building scientific applications for the GeForce. Two weeks into the job Nickolls was diagnosed with malignant melanoma; he kept working 72-hour weeks through treatment, hiding his discomfort from family and colleagues. The cancer went into remission as the first versions of CUDA went live.
- No gamer. He cared nothing for games or even graphics, only for making chips faster.
- A yeller, in the house style. His son Alec recalls phone meetings full of shouting, not toxic, but demanding that people know what they were doing.
- Relentless at home too. He tricked Alec onto black-diamond runs, preferred laying track to socialising at the train club, and kept the boy in the pool until he could inflate his trousers as a Boy Scout life preserver.
- Faith. Asked why he worked for a games company, he said he was building one of the most important technologies ever, a platform that would make other computers look like calculator watches: "Few inventions will have the impact on the world that CUDA will ultimately have."
Witt calls this mostly faith. By the late 2000s computers were fast enough for consumers and nobody obviously needed CUDA; bundling it with a retail graphics card was like bolting a minivan to a rocket sled and marketing it to commuters. Nickolls wanted to smash Moore's Law, not detour around it.
More transistors per beat
His lever was arithmetic intensity. A chip's internal clock was pulsing hundreds of millions of times per second by the mid-2000s, and the wiring could not take a faster beat. Parallel computing does not speed the clock; it makes more transistors act on each pulse. An Intel CPU fired a handful at a time; a gaming GPU fired thousands.
The zero-billion-dollar market
The first users were two breast-cancer researchers at Massachusetts General Hospital who wrote proposing to upgrade their mammogram scanners. Huang made the hospital CUDA's alpha tester and spent several million dollars on a pilot that sold exactly two graphics cards. Diercks says Huang loved it.
Witt explains why. Hardware advantages were easily copied:
- Hobbyist die shots. Enthusiasts pulled chips from retail boards, dissolved the casing in boiling sulfuric acid, and posted metallurgical-microscope images of the circuits.
- Professional teardown. Rival labs reverse-engineered competitors' silicon as a matter of routine ("it's not even black ops").
- Weak legal protection. The layouts were patented, but the 3dfx fight had shown lawsuits were futile.
To stand apart, Huang needed a move so contrary to business logic that ATI would not copy it: an exploratory product such as a $300 entry-level scientific supercomputer with no rivals and no obvious buyers. A "zero-billion-dollar market" is one only its creator can see. In Witt's image, Huang built a baseball diamond in a cornfield and waited for players.
The logic, in short: hardware could be cloned in months, but a market nobody else believed in could not be contested. Huang's differentiation came from committing early to customers who did not yet exist and accepting years of losses a rational rival would refuse.
Serial versus parallel, in three analogies
Witt explains parallel computing (split a big problem into small pieces and solve them simultaneously) through three pictures, each aimed at a different audience: the circuits, the programmers, and the end-user scientists.
| Analogy | Serial CPU (Intel) | Parallel GPU (Nvidia) | What it teaches |
|---|---|---|---|
| Dance floor (circuits) | Most dancers frozen, each moving only on its assigned beat; the DJ speeds up the tempo with diminishing returns | The DJ choreographs thousands of dancers on every beat; the floor shakes and heats up | Parallelism raises work per clock pulse, at the cost of heat and power |
| Delivery (programmers) | One van, one package at a time; slow but you just throw packages in the back | A fleet of motorcycles delivers everything in half an hour, but every package must be assigned and routed | Parallel code is far faster and far harder to organise |
| Kitchen (scientists) | A Wüsthof knife: any cut, beautifully, one vegetable at a time | A Cuisinart: loud, crude, power-hungry, but minces a pile fast | The GPU is a specialist add-on; the CPU stays the primary tool |
- Bottom-up change. Reworking the circuits alters everything above them; programmers must learn to think about problems in a new way. Nickolls's bet was that once the "van" hit the traffic jam of electromagnetic physics, programmers would be forced to learn to run the motorcycle fleet.
- Programmers were intermediaries. The real paying customers, if they came, would be doctors, astronomers, geologists and other domain scientists, often unable to code at all.
- The GPU needs a CPU. You cannot cook with only a food processor, and you cannot run a computer on a GPU alone; at first purists sneered at the gadget.
- Big Data is the truckload. By the mid-2000s astronomical, geo-engineering, medical, government and financial data, plus the human-generated web, arrived in volumes exponentially larger than before. A scientist who once got a crate every couple of weeks now got several shipping containers a day. Elegant cuts no longer mattered; speed did. The GPU was "data's Cuisinart."
Building the CUDA stack
Under Nickolls, Nvidia divided its chips into CUDA cores: arrays of circuitry that execute the same instruction simultaneously across many groups of data. On top sat layers of software, together known as the CUDA stack, which turned ideas into electricity and electricity into results.
- Circuits (Arjun Prabhu). The hardware director compares chip layout to urban planning, with zones for different tasks, and says his best ideas come on Friday nights when he literally dreams of transistors, like a Tetris player.
- Machine code (Ian Buck). The lowest layer, "whispering directly to the metal," which most programmers find numbing and Buck loved. With numerics specialists largely from Moscow State University ("a lot of Pyotrs, a lot of Borises") he rewrote differential equations and higher-dimensional matrices as primitive plus, minus, times and divide, run in parallel over many datasets. Buck's point: humans think linearly, giving directions to one Starbucks, not to every Starbucks from everywhere.
- Compiler (Bas Aarts). The translator from languages such as C++ and Python into machine code. Aarts would vanish into his head for weeks, conceding that some think him "one-dimensional."
- Strategy (Huang and Nickolls). Hug the scientific customers tightly: make the speed-ups so large and obvious that users build whole academic disciplines on the platform. Aarts: "It's vendor lock. There is no out."
The CUDA tax
Shipping dual-purpose chips was internally contested because the extra circuitry pushed GeForce production costs above ATI's Radeon, a premium staff called the "CUDA tax." Huang gambled that gamers absorbed in Half-Life 2 would not notice they were subsidising a speculative venture into supercomputing.
Early CUDA engineer Brett Coon puts it bluntly: the "genius" of CUDA was getting gamers to pay for the massive chip development costs. His point is that most CUDA-core ideas were old supercomputer and specialty-processor techniques that had been too expensive for their small markets. The gaming mass market, not technical novelty, made them affordable.
A slow, costly launch
- Late 2006: public launch; free to download, but runs only on Nvidia hardware.
- 13,000 downloads in 2007, its first full year: under 0.01% of the hundreds of millions of GeForce owners.
- 100 million+ lines of code in the full GeForce software package soon after, more than some versions of Windows.
- Wall Street did not merely doubt CUDA; one insider says investors thought it had "negative value."
Why adoption stalled
- Hard to program. Users had to break tasks into hundreds of threads, feed them carefully into the CUDA cores, juggle multiple memory banks and avoid timing mismatches that silently produced wrong answers. Physicists and doctors rarely had the computer-science training.
- The textbook flopped. Kirk tried to repeat his shader-era tactic with Programming Massively Parallel Processors, cowritten with Wen-Mei Hwu. It argued architecture had not changed since John von Neumann's 1945 design and that expecting programs to speed up with each new chip was "no longer valid." Few professors adopted it; Witt jokes that questioning von Neumann was beyond heresy.
- Industrial deals faded. Nvidia courted stock traders, oil prospectors and molecular biologists, and even signed General Mills to simulate the thermal physics of baking frozen Totino's pizza. Most deals died within a couple of quarters.
- Internal friction. Nickolls had to fight, sometimes forcefully, for resources. Coon: it was easier to sell hardware designers on faster Unreal or Doom than on faster matrix multiplication or Fast Fourier Transforms.
Bumpgate
Witt's biggest hidden cost of CUDA was distraction from the core customer.
- The fault. From late 2006 gamers reported notebook GPUs dying after a few weeks. By the time Nvidia acknowledged it, forums accused the company of attaching chips badly to the solder "bumps" on the board.
- The market. The stock lost close to 90% for the second time in six years; board members said the Bloomberg chart looked like an EKG of a heart attack. Radeon loyalists alleged a cover-up.
- Dell. In early 2009 Dell dropped Nvidia as preferred supplier for its gaming laptops; a columnist called it the price of Nvidia's "abject stupidity."
- The reserve. Huang set aside $200 million for refunds, erasing the year's profit: Nvidia's first loss since going public. At a press Q&A, in glasses, black jeans and a gray athletic T-shirt showing a physique built by lifelong push-ups, he sounded calm; only veterans sensed the anger underneath.
To save money after Bumpgate, the 2008 strategy offsite moved into the company cafeteria. Huang stood in one corner and a long-serving, highly skilled chip architect in the other while about 150 senior executives lined the walls; hardware engineer Sameer Halepete recalls 90 minutes, maybe two hours of nonstop berating, the loudest he ever heard Huang. The architect kept his job. Halepete's reading: Huang rarely acts on execution failures because he fears chilling risk-taking, so the tirade replaces dismissal: "He's never going to fire you."
Deb Shoquist and the speed of light
To restore confidence Huang leaned on Deb Shoquist, head of Nvidia's worldwide supplier network (Guadalajara, Hanoi, Bangalore), a twenty-year operations veteran who did not back down.
Soon after she joined in 2007, Huang asked Shoquist to shorten lead times at a Taiwanese packaging vendor downstream of TSMC. She thought it impossible and told him in a meeting, "You don't understand how this works," while colleagues messaged her laptop to stop. Huang erupted ("You don't know ops!"). She flew to Taiwan to prove him wrong and spent a week on the vendor's unit economics. Lead time was three weeks, but actual cycle time (person-hours to encase the chips) was only 36 hours; matching the two was possible at $1,000 per chip instead of $8. Huang was right. She presented privately and he said, "That's the right answer." She then built expediting cost schedules for hundreds of suppliers and cut Nvidia's cycle time from months to weeks, to a record 13 days.
- Define the limit. Ask how fast a task could possibly be done with unlimited budget and everything going right (New York to London: perfect weather, no traffic, a supersonic plane).
- Work backward. Set realistic but still ambitious schedules from that unreachable constant.
- Why it relieves pressure. Shoquist: once you know the physical limit, you know competitors cannot go faster either.
- Not "move fast". Witt notes nearly every Nvidia employee he interviewed invoked the phrase; it means benchmarking against physics, not haste.
Huang applied the ideal to himself, working until exhausted so nothing could keep him awake at night, "the only thing I can control." Others burned out. In 2007 David Kirk, weary of commuting, cashed some of his 1990s shares, moved to Telluride and later Hawaii, and reverted to part-time consulting. He had been running Nvidia Research, then 30 scientists on advanced graphics. Huang wanted something more ambitious to exploit CUDA and told Kirk to find his successor; Witt previews that the recruit would make it the most successful corporate R&D group in the world.
Witt expands CUDA as "Compute Unified Domain Architecture." Nvidia's own expansion was Compute Unified Device Architecture, though the company has since dropped the acronym's meaning. His "first 8K rig" and "thirty-two cards" are Buck's own recollections, and the claim that CUDA alone caused Huang to neglect manufacturing is Witt's interpretation; Nvidia itself blamed a weak die and packaging material set in a generation of notebook chips.
The "no out" lock-in held. At GTC in March 2025 Nvidia added native Python support to CUDA, opening it to far more programmers than C++ ever did, and in September 2025 it began distributing CUDA directly through third-party Linux platforms such as Canonical and SUSE. Nvidia was claiming over 5 million CUDA developers by 2024, against 13,000 downloads in 2007.
Resonance
From 2009 to 2013 Huang kept pouring money into CUDA while downloads fell, profits stayed flat, the stock went nowhere for a decade and the activist fund Starboard Value circled. He hired Bill Dally as chief scientist, dropped a failed modem venture to placate investors, and refused to drop CUDA. Witt calls the decision to double down on CUDA against Wall Street the riskiest of Huang's career, "the bet that made Jensen Jensen." Huang's defence was half Christensen, half instinct: a sense, which Horstmann calls resonance, that some neglected science was about to need his chips.
Bill Dally, chief scientist
Kirk's replacement as head of research was Bill Dally, whom Witt presents as a man for whom every waking hour serves either engineering or adventure: cycling to collapse, rowing on Lake Tahoe, downhill ski racing, sailing nonstop from Grenada to Antigua. His output dwarfed even Kirk's and Nickolls's.
- 250 technical papers, 4 textbooks, 120 patents, from circuit architectures to power-supply chips.
- High-school dropout (he skipped history class) and former auto mechanic, admitted to college on test scores.
- B.S. Virginia Tech, M.S. Stanford, Ph.D. Caltech; tenured at MIT by his early thirties.
- Chair of Stanford computer science at 40.
- Nvidia consultant from 2003; chief scientist from January 2009.
The crash and the Jellybean
- September 1992. Flying his single-engine Cessna over Long Island Sound at about 6,000 feet, Dally saw the oil light, lost the engine minutes later, glided and ditched in the water. He broke his nose on the yoke, had about twenty seconds to smash the window and get out, floated on a seat cushion and was picked up by a passing sailboat. He later rebuilt the flight in a simulator; after ten tries ending in the water, he found one that reached Groton-New London airport. He was back at work within days.
- The Jellybean machine. His MIT team built a person-high experimental parallel computer, with bespoke processors, boards, networking and applications, partly held together by duct tape. Benchmarks beat conventional machines, but no commercial partner would build it, because existing computers were fast enough and parallel devices had no market. It is the same lesson as Nickolls's MasPar.
Saying no, then yes
When Kirk offered the full-time job in 2009, Dally first declined. The case against was strong:
- Losses. Nvidia lost money in fiscal 2009 and 2010, and the stock was depressed.
- Reputation. Neither Fast Company's nor Businessweek's 2010 most-innovative-companies list included Nvidia; leaving Stanford looked lateral or worse.
- Intel's counter-story. Intel, ten times Nvidia's size, kept telling investors it could engineer around leaky transistors and that Moore's Law was alive. It also made Dally a richer offer.
He took Nvidia anyway, accepting in January 2009 as the stock fell into single digits. Jim Plummer, Stanford's engineering dean and an Intel board member, told him he was crazy: "Intel is going to crush Nvidia." Dally's reason was Huang himself, a natural leader you want to follow.
Rebuilding Nvidia Research
Dally grew the group from 30 to more than 300 people. His selection rule followed Nvidia's structural edge, exponentially more computing per dollar over time: back projects that would intersect that rising curve. The portfolio eventually spanned robotics, automobiles, climate modelling and biochemistry. Bound now to the six-month GPU release cycle, he found to his surprise that it involved less bureaucracy than academia, because every employee's livelihood depended on the chip shipping on time.
GTC 2009: the Woodstock of HPC
Dally pushed for more outreach to academics, and within months Nvidia staged the first GPU Technology Conference (GTC) in 2009 at the Fairmont Hotel in San Jose. Huang, mid-makeover from a wardrobe Witt says looked bought "with Kohl's Cash," appeared in a tight black shirt, boot-cut jeans and silver-buckled shoes and handled the crowd like a stage hypnotist: "Welcome to the Woodstock of high-performance computing."
- Spectacle as teaching. Huang had earlier hired Mythbusters hosts Adam Savage and Jamie Hyneman to dramatise serial versus parallel: a robot firing one paintball at a time took about a minute to paint a crude smiley face, while an array of 1,100 paintball cannons fired at once and produced a pixelated Mona Lisa in a split second.
- The programme. A masquerade charity ball for a local school opened proceedings (attended mostly by middle-aged men in lanyards); talks covered quantum chemistry, augmented reality and black-hole modelling.
One talk was "Unlocking Biologically-Inspired Computer Vision: a High-Throughput Approach" by MIT's Nicolas Pinto. He had gathered many image-recognition programs, several of them neural networks, optimised them with CUDA, and asked them to identify characters in clips from Law & Order DVDs. Blended, the best models recognised actor Jerry Orbach with almost 90% accuracy even on unseen clips, and Pinto's paper measured the Nvidia chipset at 1,356 times the performance of a similarly priced Intel CPU. It drew little attention: GTC 2010 and 2011 did not build on it, and the paper collected fifteen citations in two years.
Witt's gloss is evolutionary. Image recognition is a foundational AI problem because it retraces how intelligence evolved. Around the same time paleontologists in Greenland found a 500-million-year-old trilobite-like fossil (eye stalks, horseshoe-crab body) with preserved neural tissue: a rice-grain clump at the end of its optic nerve, which Witt calls the earliest brain ever found. Vision drove biological intelligence and, he argues, was about to do the same for machines. In 2009, though, neural nets were a neglected branch and nobody cared.
CUDA in the doldrums
John Nickolls's melanoma returned in the late 2000s and he died in 2011, working to the end; his son Alec thinks those were among his most productive years. Dally, his ski-racing partner, was devastated, and twelve years later Huang still could not talk about him without visible emotion. Nvidia endowed a scholarship in his name at the University of Illinois at Urbana-Champaign.
| Year | CUDA and Nvidia |
|---|---|
| 2007 | 13,000 CUDA downloads |
| 2009 | 300,000+ downloads, the peak |
| 2010-11 | Three straight years of decline begin |
| 2012 | Bottom: just over 100,000 new installs; stock flat for a decade; revenue and headcount up but profits flat |
| 2012 | Starboard visits headquarters, urges focus |
| Early 2013 | Starboard Value letter to the board |
The scientific market looked saturated, and investors grumbled. Ben Gilbert, cohost of the Acquired podcast, frames the critique: Nvidia was spending many billions on chip architecture for an obscure academic niche worth far less. Witt's verdict: Huang was bringing supercomputing to the masses, and the masses did not want it.
Starboard and Jeff Smith
Starboard took a small stake and wrote to the board in early 2013. The letter was formally neutral on CUDA but questioned whether Huang's strategy made sense; other analysts argued the stock would rise if CUDA spending were returned to shareholders, and some doubted Huang's fitness to remain CEO.
- Jeff Smith. Starboard's CIO, 42, boyish and media-savvy; demanded board seats and, when resisted, usually tried to oust the CEO. By 2014 he had forced 80 board replacements at 30 companies in three years, and Fortune named him "the most feared man in corporate America."
- Jim Gaither. The founding lawyer (to whom Huang once handed all the cash in his wallet), now a director, who organised Huang's campaign to keep his job.
- Fidelity. Nvidia's largest holder, managing over $1 trillion and owning more Nvidia shares than Huang; in Boston it "beat the crap out of us," says Gaither.
- Tench Coxe. Founding-era director who admits the company was "going sideways."
- Dawn Hudson. Former NFL marketing executive, added to the board just after the letter; she found "a distinctly flat, stagnant company."
Smith's reputation rested on Darden Restaurants, where he replaced the entire twelve-person board while holding under 6% of the stock, armed with a 294-slide turnaround plan for Olive Garden. Slide 104 attacked breadstick policy (servers had drifted from one stick per guest plus one for the table, refilled on request, to dumping piles that filled diners up and cut orders); slide 163 noted the chain had stopped salting pasta water to make cookware last longer. The signal to Nvidia: Smith interrogated operations in excruciating detail and usually won.
What Huang cut, and what he kept
Witt concedes the money-pit view of CUDA was not obviously wrong, and Huang had killed projects before:
- Northbridge chips (early 2000s). Nvidia made motherboard memory-controller chips for several cycles, saw it was a race to the bottom with Intel, scrapped them and told staff he had erred.
- Mobile graphics. A defensible bet on a huge market; in early-2010s interviews Huang talked more about phones than supercomputing.
- Icera (2011). Nvidia paid $367 million for this cellular-modem maker, entering a mature market dominated by Qualcomm. This, not CUDA, triggered Starboard: CUDA plus graphics plus mobile chips plus modems was too many balls in the air. Smith's team visited in 2012; the meeting was cordial, with a proxy fight implied.
Huang eventually agreed and abandoned modems: fighting to the death in mobile meant not doing something else, and he had been scarred by earlier fights to the death. CUDA he never considered abandoning, because in that unpopular market Nvidia was the only supplier.
Supercomputing customers were the worst kind: cash-starved and fickle. Academics lived on unpredictable grants, corporate R&D faced sceptical CFOs, and government programmes announced ten-year budgets that dissolved into wrangling. Fellow chip executives thought CUDA unwise. Holding on through the Starboard episode is, in Witt's ranking, the riskiest decision of Huang's career.
The roadshow went badly. After Fidelity, Huang met about half a dozen institutional investors in New York and was grilled by portfolio managers; Gaither admits there was no clear path to a breakthrough. The founding-era board stayed loyal, but the first whispers of discontent were audible.
Empty-mansion syndrome
At home the children had gone: Spencer to art school in Chicago to become a photographer, Madison to culinary school in Paris (Le Cordon Bleu); the dog Sushi had died. After years of eighty-hour weeks Huang admits Lori did "ninety percent of the parenting"; on his one weekend day with the kids he was often distracted (Horstmann recalls him sending them repeatedly onto a roller coaster so the two men could talk shop). Horstmann suspects both families' children avoided tech careers because they saw "there's got to be more to life than this." Huang adopted two more dogs and took up cooking, but around the Fidelity grilling he botched a dish in his million-dollar kitchen, with his Cordon Bleu-trained daughter beside him, and screamed at the stove until everyone left.
Why Huang held on
The mad scientist in Taipei
Huang often cites a visit to Ting-Wai Chiu, a physicist at National Taiwan University simulating matter's evolution after the Big Bang. Chiu had built a home-brew supercomputer from GeForce cards, the lab littered with their boxes and cooled by oscillating desk fans; he credits Huang with making his life's work possible. Chiu was the ideal customer: by shipping CUDA on every retail board, Nvidia reached even the poorest researchers, "mad scientists" whose fields were too unfashionable to afford a workstation.
- Disruption starts with hobbyists. New technologies grow from "bootlegged resources" and "off-the-shelf components" used for unintended ends, start crude and improve fast on attributes incumbents ignore.
- Incumbents are told not to. Established firms struggle to serve emerging markets because their investors and customers tell them not to, exactly Starboard's message.
- How not to fail. Witt's reading of The Innovator's Dilemma: it is less a start-up how-to than a counterinsurgency manual for managers of stagnating firms. Thirteen years in, Huang feared Nvidia was becoming one.
- Risk both ways. Huang: shipping CUDA was risky, but so was not shipping it. The danger was a hungry start-up, in an office beside a Chinese restaurant and a much-robbed bank, serving marginal academics for years and doing to Nvidia what Nvidia did to Silicon Graphics.
- Huang's correction. Christensen "got as much wrong as he got right." His marginal customers were dirt bikers and trench diggers; Huang's were scientists, any one of whom might change the world.
Spectacles to telescopes
Witt's historical parallel is the lateral technology transfer of the early 1600s, when Dutch spectacle makers (one story credits two children looking at a weather vane) rearranged eyeglass lenses into telescopes and flooded the patent office with designs. Within a year Galileo had one, and with less magnification than modern birding binoculars he described the phases of Venus, Jupiter's moons and Saturn's rings. Cheap supercomputers for mad scientists were meant to do the same.
Elon Musk. Starts with a vision (himself standing on Mars) and works backward to the technology needed to get there.
Jensen Huang. Starts with the circuits in front of him, projects forward as far as logic allows, and only at that frontier takes a single step on instinct.
Resonance
Horstmann's word for the instinct is resonance, a step "beyond focus." Huang generated it through constant contact: scientists in the front rows at his conferences and press at the back, with attention allocated to match; frequent visits to junior staff to feel the company's pulse. Horstmann compares it to LSI days when they almost read customers' minds. Huang sensed a breakthrough from researchers' astonishment at their speed-ups and from the zeal of Dally, Nickolls and Buck, strongly enough to sacrifice profits, burden his core product and risk his job. He was sure some graduate student would skip the grant circus, buy a GPU with a housing stipend and prove CUDA right. He had no idea which neglected branch of science it would be.
Pinto was a doctoral researcher at MIT in 2009, not a professor, and the Jerry Orbach work was academic face recognition rather than a deployed system. The "first brain ever found" is Witt's superlative; claims for the oldest preserved Cambrian nervous tissue have shifted with new finds. Gaither's and Coxe's memories of the 2013 roadshow are recollections a decade later, after CUDA had been vindicated, which flatters the bet in hindsight.
The "Woodstock" outgrew the Fairmont: GTC 2026 ran 16-19 March 2026 in San Jose with Huang's keynote staged in an arena and streamed worldwide, a fixture now treated by markets as a product-roadmap event. Dally remained Nvidia's chief scientist and SVP of research into 2026, still running the group he took over in 2009.
AlexNet
In 2012 Toronto graduate student Alex Krizhevsky trained a neural network on two $500 GeForce gaming cards in his parents' house, and it crushed the ImageNet image-recognition contest. AlexNet proved that GPUs could train neural networks hundreds of times faster than CPUs, finally giving CUDA its killer app and ending the long exile of neural nets. Within weeks the three-man team was sold to Google for $44 million, and machine learning had become the computing workload with no ceiling.
Hinton's exile in Toronto
The recruit
Witt draws Alex Krizhevsky as almost pathologically averse to attention: slight, pale, red-haired, revealing little even to long-time colleagues. Born to a Jewish family in the Soviet Union, in what is now eastern Ukraine, he emigrated to Canada as a child; his English was flawless but sparing, and Geoffrey Hinton likened him to a veteran spy. Hinton's one firm judgment: "probably the best programmer I ever met." In the late 2000s, still living with his parents and studying at the University of Toronto, Krizhevsky appeared uninvited at Hinton's office: he was the top software-engineering student, it was boring, could he join?
Why neural nets were outcasts
Hinton was a coauthor of the seminal 1986 backpropagation paper and had pushed neural networks for decades against indifference and hostility. That crusade had left him in snowy, underfunded Toronto, far from Stanford and MIT; he warned Krizhevsky the field was deeply out of favour, and the group's competitive results were often rejected for publication. "Neural nets were regarded as nonsense."
- An "ideological" bias. Hinton spat the word the way Huang says "political." The field held that working was not enough: AI should unlock the secrets of intelligence in mathematics.
- The textbook. The standard 1,100-page AI text surveyed probabilistic reasoning, decision trees and support-vector machines; neural nets got ten pages, led by a backgammon example.
- The pedestrian paper. A colleague's net beat state-of-the-art pedestrian detection and could not get into a conference, because it did not explain how the computation worked.
- Hinton's rebuttal. Nobody could describe mathematically how brains process language either; trying to "solve" the brain was like trying to solve for the function of the kidney. It won no converts.
- The disguise. To get funding, neural-net researchers called their work "machine learning" or "deep learning", anything but AI.
- Lingering myth. Intro courses still claimed nets could not compute simple logic functions, a limit backpropagation had solved decades earlier.
- Geoffrey Hinton. PhD adviser; backpropagation coauthor; set the goal of obliterating rivals, beyond merely beating them.
- Alex Krizhevsky. The programmer; mastered GPU programming with no prior background.
- Ilya Sutskever. Another Russian-speaking Jewish émigré, but athletic, bushy-browed and audacious; Hinton's most fervent disciple, who already argued nets would outstrip human intelligence, which even Hinton did not then claim.
- Abdelrahman Mohamed and George Dahl. Students whose GPU speech recogniser (2008-09) first brought CUDA into the lab.
- Fei-Fei Li. Stanford creator of ImageNet and its annual contest.
- Wojciech Zaremba. Polish Google researcher who replicated the network as "WojNet."
CUDA arrives, Nvidia shrugs
In 2008 Hinton set Mohamed and Dahl to build a speech-recognition module on an expensive Nvidia server; by early 2009 their net rivalled the best mathematical models. At that year's NeurIPS Hinton told the audience parallel processors were the future of AI and they should drop everything and buy Nvidia GPUs. He then emailed Nvidia: having told 1,000 machine-learning experts to buy its cards, could he have one free? Nvidia declined. It had not one dedicated AI researcher, machine learning was absent from the application list in Kirk's textbook, and the CUDA group sometimes ignored Hinton's emails. Hinton concluded that only a crushing victory would get Nvidia's attention.
Sutskever did not count the private Krizhevsky as a friend, but they were on one wavelength: asked a question, they would confer in Russian and return with the right answer, leaving Hinton struggling to keep up.
Teaching a computer to see
Hinton's assignment: use Nvidia GPUs to build an image recogniser based on a convolutional neural network, which applies mathematical filters to pick out key details of a picture. Krizhevsky took to parallel programming instantly, as if his brain natively thought in Buck's "every Starbucks from everywhere at once"; Hinton calls him a wizard at making GPUs run convolutional nets.
- The 30-second test. In early 2012 he ported a teaching-exercise network to CUDA and it trained in 30 seconds, against an hour on an Intel machine and, Witt adds, a hundred thousand years for biology. Sutskever saw speed hundreds of times beyond anything he knew.
- Sutskever's syllogism. If an artificial neuron is roughly like a biological one, nets should do everything we can; if they can be accelerated, "then you're training brains."
- Scale to the maximum. Sutskever immediately grasped that the computing behind this "synthetic evolution" had to be pushed as far as possible, the insight Hinton says others take years to see.
The hardware
Pooling their money, the students could afford just two GeForce GTX 580s at about $500 each online. Each weighed about three pounds, looked like an Alien prop (black casing, slime-green trim, a round fan vent) and held a chip of 3 billion transistors in 32 parallel cores. Krizhevsky put them in the desktop in his bedroom and ran them for a week; his parents paid the considerable electricity bill. Witt's point: this broke, reclusive iconoclast was precisely the "mad scientist" customer Huang had imagined when he put CUDA on every retail card.
The data
The training set was ImageNet. Dissatisfied with small online datasets, Li had hired workers on Amazon's Mechanical Turk to hand-label 15 million+ images in 22,000 categories, hundreds of times larger than any comparable set, over the doubts of her own advisers. Krizhevsky's network had about 650,000 neurons (roughly a honeybee's count), and a model that size needed that much data.
How the network learned
- Random start. Connections began random. Shown its first image (a stingray, a terrier, a golf cart) the net could only guess wrong, but each miss carried a scrap of information about what the image was not.
- The hard step. Backpropagation adjusted the connections through repeated matrix multiplication, likened to solving an unimaginably large Rubik's cube. Every earlier attempt had stalled here; CUDA's parallelism finished each round in a fraction of a fraction of a second.
- First success. Within minutes, by chance, it labelled something right (say a flowerpot), triggering a burst of updates about what separates flowerpots from chimpanzees or dump trucks. The gain was tiny (a garbage can still looked like a flowerpot), but now a whale shark could be told apart.
- Layers. Over millions of repetitions the layers specialised: one on shape, one on colour, one on symmetry, with pathways weaving them into a whole.
- No light involved. The net saw only digits encoding pixel positions and colours, updating grids of numbers representing synaptic weights. Sceptics later called it "just doing math"; Witt answers that this is like saying the retina is "just interacting with photons."
- The curve. Accuracy rose 0% → 1% → 10% → 40% → 60%, then flattened near 80%, against a previous best of about 70%. The two fans hummed at about 44 decibels, enough to keep Krizhevsky awake. Weakness: it confused human tools, unable to tell a spatula from a hatchet.
- Sanity check. Run on the previous year's ImageNet contest data, which it had never seen, it beat every entrant.
Training on labelled data is supervised learning, so Krizhevsky named the net SuperVision.
- 650,000 neurons, 60 million parameters, 630 million connections.
- 2 × GTX 580, about $1,000 of gaming hardware, one week of training.
- 3 trillion operations per second at peak; quintillions of steps in under a week, over 100 billion years of human arithmetic.
- ImageNet contest entrants: 35 (2010), 15 (2011), 7 (2012), 40+ (2014, all neural nets).
- 150,000+ citations for the paper, among the most cited in computer science.
Sutskever still glows a decade later ("GPUs showed up, and it felt like a miracle"); Hinton says machine learning without CUDA "would have just been too much trouble." Before the contest the team paced the lab sure of winning, the first to feel the now-familiar thrill of sitting on embargoed AI that would shock the world. They imagined autonomous robots, self-driving cars and self-coding computers, and saw AI as purely good, "at the time, at least." They also saw the corollary: Nvidia would gain more from SuperVision than SuperVision gained from Nvidia, because the appetite of neural nets for parallel compute had no foreseeable limit. Hinton: it was obvious a large share of scientific computing would become machine learning.
Krizhevsky invented several programming techniques, but the finding that mattered was simple: a GPU trains a neural network hundreds of times faster than a CPU. That turned CUDA from a subsidised science project into the engine of the AI industry, and turned neural nets from a fringe into the field.
Florence
Fei-Fei Li had built the contest to prove ImageNet's worth to her advisers, and it was dying: 35 entrants in 2010, 15 in 2011, 7 in 2012. Now one entry beat the state of the art by 10 points in a field that measured progress in fractions of a point, and it was a neural net, which she regarded as a museum piece. Her image: a Honda Civic breaking the land-speed record by 100 mph.
- Li's background. Born in China, an only child who moved to New Jersey as a teenager, helped support her parents, and with a math teacher's mentoring won a full scholarship to Princeton (physics), then a PhD in electrical engineering aimed at machine vision.
- Disbelief. Her staffer who computed the results suspected a bug; her thesis adviser asked Hinton whether test data had contaminated training. It took several checks to convince them.
- The conference. Results were published in October 2012 and discussed later that month in Florence. Li, who had just given birth, decided she had to go. Krizhevsky ignored her texts, then turned up unannounced in an oversized zip-up sweatshirt, looking like a teenager.
Krizhevsky spoke last. His voice cracked, he coughed, kept his eyes on the podium and rushed through black-and-white slides, one listing the 650,000 / 60 million / 630 million specs. "It was trained, actually, in my bedroom," he said, then ended with "That's all I have." The questions were hostile. Witt's reading: the talk implied that decades of elaborate AI mathematics belonged in the bin, and the audience was venting at a man who had cracked the hardest problem in computer science with a gaming rig in his childhood room.
Conversion and the auction
The field turned quickly. Senior researchers told Hinton they had been wrong; 2013's contest was swamped by neural nets, and by 2014 all of the 40-plus entrants used them. The paper, credited to Krizhevsky, Sutskever and Hinton, passed 150,000 citations.
- The rename. At Google, Wojciech Zaremba replicated the model and his "WojNet" spread. Fearing his modest student would lose credit, Hinton persuaded a sceptical Krizhevsky ("not into things like branding") to rename SuperVision AlexNet.
- DNNResearch. With acquisition offers pouring in, Sutskever pushed Hinton to incorporate DNNResearch, split in equal thirds: no customers, board, revenue or website, only three brains.
In December 2012, at a research conference, Hinton ran an email auction from a seventh-floor hotel room at Lake Tahoe. Microsoft and London start-up DeepMind bid early and dropped out; Google and Baidu fought on, and past $20 million the three would wander to the window to look at the snowy Sierra Nevada. At $44 million, with his students' agreement, Hinton stopped the auction and took Google's money, judging it the better cultural fit. Witt ranks AlexNet with the Wright Flyer and Edison's bulb; Hinton: "That was a kind of Big Bang moment."
Witt's "80% versus 70%" simplifies the contest metric. AlexNet's top-5 error was about 15.3% against 26.2% for the runner-up, a gap of roughly 11 points. Stopping the auction at $44 million rather than taking the highest bid is Hinton's account; Baidu's final number was never published. The three then saw AI as purely positive; Witt flags that two of them, Hinton and Sutskever, later changed their minds.
Hinton quit Google in 2023 to warn about AI risk and shared the 2024 Nobel Prize in Physics with John Hopfield. Sutskever cofounded OpenAI, left in 2024 and started Safe Superintelligence, valued at about $32 billion in April 2025. Krizhevsky left Google in 2017 and has stayed out of public life. Li cofounded World Labs, which released its first commercial 3D world model, Marble, in November 2025.
O.I.A.L.O.
Nvidia's first dedicated AI researcher, the long-haired Russian-literature graduate Bryan Catanzaro, built cuDNN, the library that tied neural networks to CUDA, and in 2013 Huang reorganised the whole company around it after one weekend of reading. Witt's claim is that deep learning was as much a hardware revolution as a software one: two discredited technologies, neural nets and parallel computing, that only worked in combination. By late 2014 Google was ordering 40,000 GPUs at a time.
The misfit with a theory
Witt sets Bryan Catanzaro apart from Nvidia's engineering monoculture: long hair, statement glasses, loud shirts (a rainbow tee printed with JPEG artifacts, an owl sweater), a soft voice, and the only humanities degree Witt met at the company.
- Siberia. Raised Mormon, he spent two years as a missionary in Siberia speaking only Russian and read Crime and Punishment in the original.
- Two degrees. At Brigham Young University he took Russian literature (favourites: Dostoyevsky, Tolstoy, Pushkin) alongside computer engineering, aware of which degree would pay.
- The Intel exercise (2001). As a summer intern he was asked to design a chip pulsing at 10 billion cycles per second. His maths said it could not be built; his supervisor said he must have erred, since it was on Intel's road map. He found no mistake and concluded that transistors were too small, Moore's Law was ending, and Intel was ignoring it.
The killer-app argument
Catanzaro cofounded the UC Berkeley Parallel Computing Lab in the mid-2000s and catalogued existing parallel applications. He found the field's business flaw: even the hungriest customers had capped demand; sell an oil prospector a supercomputer and he is satisfied for years. Parallel computing needed an application like 3D graphics, whose appetite grows each time it is fed. His answer was AI, reached "from a circuits perspective": the most important computational workload, inevitably.
His Berkeley advisers disagreed. Looking for AI was like hunting Bigfoot, and neural nets were regarded with contempt as an eccentric old idea that did not work. Steered away, he published a few dull papers on fancier maths. Witt notes the frustration of someone who saw the AI-plus-parallelism combination first and was denied the right tool.
With a wife and three children in the Bay Area on a student income, Catanzaro chased every corporate fellowship and did eight internships in six years, each just enough to avoid eviction, which gave him a tour of the chip giants.
- Intel. Back again, he worked on Project Larrabee, the internal "GeForce killer," which infighting delayed and then killed before launch. To management, he says, Intel "might as well have been a machine that made soap."
- Qualcomm. Well run and well paid, but staff kept telling him Nvidia was a horrible place run by a tyrant.
- Nvidia. His own internship there contradicted that. Huang felt like a finger in an electric socket, but he was no soap seller; his passion for computing was unmatched, and he was the person most likely to grasp the AI-parallelism intersection. After his 2011 PhD, Catanzaro chose Nvidia.
Inside Dally's lab
Catanzaro joined Bill Dally's Nvidia Research, which he likened to graduate school "only bigger and better." Dally ran it like an academic department: researchers followed their interests, and he published discoveries openly with no financial return, often coauthoring with AMD and Intel engineers. Some inside Nvidia pursed their lips; Dally's logic was recruiting: strong publications attract the best academics, and the quality speaks for itself.
Hired to study programming languages, Catanzaro became Nvidia's first dedicated AI researcher once Dally, hearing whispers about neural nets, decided the area could not be ignored.
In 2012 Dally lent Catanzaro to his former colleague Andrew Ng, a Stanford professor working at Google. Ng had built AlexNet-like technology on conventional machines: a cluster Witt puts at 2,000 CPUs digested thumbnails from 10 million YouTube videos and taught itself an internal image of a cat, which ran in The New York Times in June 2012. The power bill was ruinous. Dally cared less that it found cats than how much computation it took. Catanzaro repeated the experiment on 12 GPUs.
The parking lot
Events accelerated. After the Google sale, Krizhevsky, Hinton and Sutskever moved to Mountain View and started what Witt calls a parallel-computing insurrection. Offered Google's giant CPU cluster, Krizhevsky instead bought a commodity PC with a couple of retail Nvidia cards and put it in an office closet. Other Googlers began unplugging from what was probably the world's largest private computer network to run gaming hardware under their desks.
Catanzaro asked Nvidia for more resources and was refused. His reviews and pay were poor; his sensitivity and humanities background did not fit. He began building cuDNN, a library to accelerate neural-network development on CUDA, alone, while a fourth child arrived, sleep vanished, and medication for health problems left him feeling "kind of dumb." In early 2013 Nvidia's software team panned his prototype, and he doubted his managers thought the work mattered.
Going to Huang
He took the case straight to Huang, for whom machine learning seemed off the radar: at GTC in March 2013 Huang talked weather modelling and mobile graphics, never neural nets (and wore a leather jacket on stage for the first time, a bulky ugly one). Yet Huang was instantly intrigued, cleared a whole weekend to read about AI, and at the next meeting knew as much as Catanzaro, perhaps more.
- Atlantis. Witt's image: after a decade at the prow scanning for land, Huang had found Atlantis, and by mid-2013 he was vibrating with resonant intensity.
- The declaration. He called Catanzaro into the conference room he used as an office and named cuDNN the most important project in Nvidia's twenty-year history. The whiteboard held one acronym in perfect lettering: O.I.A.L.O., "Once in a Lifetime Opportunity."
- The thought experiment. Imagine all 8,000 employees marched into the parking lot: Catanzaro could pick anyone for his team.
Kirk notes that Huang needed long persuasion on parallel computing and on CUDA, with a full business case. AI was different: he "got it immediately, before anybody." Huang calls it first principles: if a network can solve computer vision, which is completely unstructured, the obvious next question is what else it can learn, and the answer looked like everything.
Huang concluded neural networks would transform society and that CUDA could let Nvidia corner the hardware. He bet the company. Vice president Greg Estes: a Friday-evening email said everything was going to deep learning and Nvidia was no longer a graphics company; "by Monday morning, we were an AI company."
A company built to reorganise
Huang had turned 50 a few months earlier, graying but boyish, prowling the halls to quiz junior staff. He now held quarterly all-hands talks, speaking off the cuff for over two hours on recurring themes: the speed of light, the zero-billion-dollar market, and above all the danger of creeping bureaucracy.
| Feature | How Nvidia was organised (2013) |
|---|---|
| C-suite | Effectively Huang alone: no COO, CTO, CMO, no clear deputy |
| Chief of staff | None |
| Direct reports | 30+, mostly "vice presidents" with fluid remits |
| Divisions | No fixed divisions or hierarchy |
| Operating creed | Be ready: anyone might suddenly become the most important person in the company |
The parking-lot exercise expressed that belief: the company must be able to reorganise itself kaleidoscopically at any moment. In early 2014 the most important person at Nvidia was a misfit with consistently poor reviews.
Building cuDNN
Now leading a team, Catanzaro stripped cuDNN to its essential job, which Witt calls accelerating evolution. A human neuron keeps about 1,000 synaptic connections and changes them chemically; a neural net changes its connections by matrix multiplication, combining two grids of numbers into a third. The rules are simple, but the number of operations explodes as matrices grow, which makes the task ideal for parallel hardware. Before neural nets, Nvidia had never prioritised it.
- Philippe Vandermersch. The cantankerous, bike-commuting Frenchman who had rescued CUDA's orphaned, unfinished matrix-multiplication library, then spent years failing to persuade climate researchers to port ancient Fortran simulations to it ("they could be a little lazy").
- No switching cost. Neural-net code was being written for the first time by motivated programmers who prized speed. They adopted CUDA at once and became lifetime Nvidia customers, the power users Huang had always imagined.
- The ace. Picked from the "parking lot," Vandermersch optimised his library for AI and, by his own account, did much of the technical work: "Catanzaro was the Jobs of cuDNN. I was the Wozniak."
- Clip the range. Most weights cluster between plus and minus one, so values outside can often be truncated, saving time and memory.
- Accept fuzziness. Within that range weights need little precision; like brain synapses they are loose, and sometimes getting the sign right is enough.
- Tolerate bad weights. AlexNet spread 630 million connections over 650,000 neurons; one wrong weight is noise, whereas in serial code one misplaced semicolon can crash an operating system.
- Serve inference too. As nets grew, users needed fast answers (an oracle speaking one word an hour is useless). Inference needs less compute than training but became a large part of the library.
Management by loyalty and fear
Huang met Catanzaro often, apologising repeatedly for mispronouncing his name. Most firms would have handed the flagship product to a seasoned product manager; Nvidia left it with a researcher who had never shipped commercial software or managed anyone. As the deadline neared and Catanzaro questioned himself, Huang, having rewarded innovation with loyalty, now rewarded lateness with abuse. Catanzaro: "I've been afraid of Jensen sometimes. But I also know that he loves me."
He privately feared Huang was falling into AI's old trap. Since the 1950s AI had cycled through hype and embarrassing busts, and he expected investors to be disappointed again. He kept quiet; the opportunity was too good. Wall Street, nearly three decades past the 1980s expert-systems crash, remembered nothing. The stock rose on neural-net buzz, and Starboard Value stopped writing. Tench Coxe: "They doubled their money and disappeared."
GTC 2014 and Google's appetite
June 2012 Ng's cat network makes The New York Times.
Late 2012 AlexNet team joins Google; GPUs appear in closets and under desks.
Early 2013 Software team pans cuDNN prototype; Catanzaro goes to Huang.
March 2013 GTC keynote never mentions neural nets.
Mid-2013 O.I.A.L.O.; the Friday email makes Nvidia an AI company.
Early 2014 Google buys DeepMind.
March 2014 GTC: Nvidia publicly tied to AI for the first time in its 21-year history.
Late 2014 Google's Project Mack Truck: 40,000+ GPUs, over $130 million.
At GTC 2014, in black slacks and a dark-blue polo open at the chest (no jacket; Witt reads the wardrobe as growing power), Huang showed Ng's network and its internal cat and human face, walked through early Nvidia AI projects, then (mispronouncing the name) handed over to Catanzaro. A live AlexNet variant labelled dog breeds posted to Twitter in real time: a Dalmatian, a German shepherd, a vizsla and a cairn terrier ("I didn't even know what a cairn terrier was"). Witt marks this as the end of the long AI ice age.
- Beta partners. The final slide listed Adobe, Facebook and Netflix as cuDNN testers. Google was absent, too important to name.
- DeepMind. Weeks earlier Google had bought the London firm of Demis Hassabis, Mustafa Suleyman and Shane Legg, which aimed at the first artificial general intelligence (AGI). It was building AlphaGo, which in 2016 beat grandmaster Lee Sedol at Go, a long-standing AI milestone. With Ng's group, the AlexNet team and DeepMind, Google held an early AI monopoly.
- Project Mack Truck. As researchers demanded GPUs, Google secretly began building the world's most powerful parallel computer in late 2014: 40,000+ Nvidia GPUs for $130 million+, Nvidia's largest order ever by far, and only the beginning.
Witt's conclusion is that the deep-learning revolution, rarely understood this way at the time, was a hardware story as much as a software one: two unpopular, cash-starved, discredited technologies that revealed their ideal form only together, twin strands of DNA for an organism set on consuming all the world's data.
Witt calls Go "the Japanese game"; it originated in China, though Japan shaped its modern professional form. Ng's cat network is usually described (by the paper and the Times headline) as using 16,000 processor cores on about 1,000 machines, not 2,000 CPUs; the GPU replication with Ng's group used 3 servers with 12 GPUs. cuDNN itself shipped publicly in September 2014, months after the GTC demo. Vandermersch's Wozniak claim is, as Witt notes, his own telling.
The cuDNN trick of trading precision for speed became Nvidia's main hardware lever: FP16 tensor cores in Volta (2017), FP8 in Hopper (2022) and FP4 in Blackwell (2024-25), with Nvidia's own NVFP4 format promoted for training as well as inference in 2025. Catanzaro remained at Nvidia as VP of applied deep learning research, a public face of its open Nemotron model family.
Superintelligence
From 2014 Huang dropped every hobby and worked only on AI, while cloud giants bought GPUs by the tens of thousands, venture money flooded in and Nvidia's stock finally passed its 2001 peak. As Nick Bostrom and Elon Musk warned of existential risk and Musk cofounded OpenAI, Huang built the machine to train it: the DGX-1, hand-delivered in August 2016. Witt's sharpest contrast is between two engineer-visionaries: Musk saw AI as a possible extinction event; Huang saw no risk in it at all.
Only work, only AI
Jens Horstmann noticed his friend change. Huang had always kept a gentler life outside work:
- Possessions. Dogs, a whiskey library, expensive cars including a Tesla Roadster and a Koenigsegg supercar, and a beachfront house in Maui with a sunset view, where he hosted friends such as TSMC's Morris Chang.
- Cooking. For one birthday friends arranged a stint under a Michelin-starred chef at the Four Seasons. The chef hazed him ("he'd done Denny's, but this was something else"); Huang worked a twelve-hour shift, was yelled at by a boss for the first time in years, served his friends dinner, and fell asleep in the car home.
- Movies. He liked watching big-budget films alone, especially the Avengers series, grading the CGI as much as the plot. Marvel fans were often gamers who knew his face from the Nvidia subreddit; selfie requests pushed him to 10 a.m. showings, then to stop going.
With AI, it all stopped: no table tennis, a cold teppan grill, unanswered texts, no conversation but work. "O.I.A.L.O." opened every meeting. Having worked 12-hour days, six days a week for three decades from age twenty, Huang, now past fifty with grown children, worked harder still. From 2014, Witt writes, there was only AI.
The GPU becomes a utility
- Stock up 30% (2013), 27% (2014) and 66% (2015), finally clearing for good the peak set in 2001, the day after index funds were obliged to buy on Nvidia's entry to the S&P 500. Holders had waited fourteen years.
- Revenue still came mostly from GeForce gamers, but Wall Street now valued an AI company.
- AI venture investment: close to $0 in 2010, $5 billion by 2015 and climbing.
- Rented compute: about $3 per GPU per hour for fresh Nvidia chips, against a few cents for comparable Intel CPUs.
- At that rate AlexNet would have cost about $500 to train; new models with billions of parameters cost millions.
Google's Mack Truck order was copied by Amazon, Oracle and Microsoft. These cloud-service providers built hyperscale data centers of tens or hundreds of thousands of GPUs and leased the capacity to companies, selling computing like water or electricity.
Inside a data center
- Setting. Unmarked industrial warehouses worldwide; through security and an airlock onto a climate-controlled floor.
- Racks and trays. Rows of seven-foot racks, like library stacks, each holding horizontal trays with one or two GPUs; modular, so easy to upgrade.
- One computer. Cabling lets hundreds of GPUs act as a single machine; a small core of CPUs administers it and a thick fibre bundle connects it to the world.
- Nvidia's share. Nvidia made neither racks nor trays and usually not the buildings, only the chips, but there it held a virtual monopoly.
Nvidia named architectures after scientists: Curie, Tesla, Fermi, Kepler, Maxwell, Pascal, Turing, Volta, Ampere, Lovelace, Hopper, Blackwell. Later generations added dedicated AI circuits, so data-center trays needed constant replacement, and the six-month upgrade cycle set off panic buying and huge profits.
Why buyers kept buying
Training paid for itself. Google used a neural net to optimise its server network's power draw and saved hundreds of millions of dollars a year in electricity, recouping its AI spending almost at once; it also launched photo recognition, automatic picture tagging and AI-improved search. Instagram, Facebook and Twitter turned to "algorithmic" feeds, which Witt calls a euphemism for machine-learned engagement that kept users doomscrolling on rage bait. Returns were so direct that Huang revived the oldest retail line: "The more you buy, the more you save."
Venture money followed into image and speech recognition, health care, self-checkout, self-driving cars and education. In early 2016 Marc Andreessen said every deep-learning start-up his firm backed built on Nvidia, and in the firm's game of picking public stocks as if it were a hedge fund, "we'd put all our money into Nvidia."
Talent prices
Engineers now judged offers by "total comp" (salary, bonus, options, benefits), which could exceed seven figures a year. In late 2014 Bryan Catanzaro left for Baidu to work with Andrew Ng when it tripled his salary, though he says he would have made more by keeping his Nvidia options. Bas Aarts, author of the first CUDA compiler, also left. Within a few years both were back: "There was only Nvidia."
The doomers
- Children with a bomb. The Swedish Oxford philosopher Nick Bostrom likened humans building machine intelligence to "small children playing with a bomb."
- The last invention. A computer with general, abstract intelligence would probably be the last invention humans ever needed to make.
- Fast take-off. Such a machine might self-improve, perhaps within seconds, into a superintelligence that would seem omnipotent to us.
- The gorilla analogy. It might treat humans as humans treated gorillas: take the planet, destroy the habitat, keep a few as mascots in reserves.
- The paper-clip maximizer. His earlier thought experiment: an AI whose only goal is maximum paper clips will want no humans, since humans might switch it off and their bodies contain atoms that could become clips. The idea spread through the online "rationality" community and among tech executives.
A few months after the book, Elon Musk posted on Edge.org that progress in general AI was "close to exponential," visible only to those near groups like DeepMind, and that something seriously dangerous could happen within five years, ten at most. He deleted it within minutes; the screenshots spread. Witt judges the timing exaggerated, as with many Musk predictions, but the long-term risk real.
GTC 2015: the jacket, the captions, the demon
At GTC 2015 in the San Jose Convention Center Huang settled his signature look: black leather jacket over a dark-blue polo, black slacks, black shoes, mostly high-end labels. He credits Lori and daughter Madison, who soon left cooking for the luxury group LVMH. In 2023, asked about it, he opened his jacket to check the label: "Tom Ford, maybe?"
Slovak researcher Andrej Karpathy, from Fei-Fei Li's Stanford lab, had glued an AlexNet-style image recogniser to a second net that wrote natural-language descriptions. A bird became "a bird perched on the branch of a tree"; a rear-view photo became "a man riding a horse drawn carriage down a street." Huang gleefully showed the failures: a flying fish as a white bird over water, two men on a toboggan as a man and child on a bench, an infant with a toothbrush as "a young boy holding a baseball bat" (human tools were still the weak spot). Karpathy admitted the net mimicked speech without understanding it and, at unfamiliar concepts, proudly produced nonsense, which he called hallucinations.
Then Musk came on, to a sea of raised phones. Huang was a Tesla fan who owned all three of its models; Tesla had used Nvidia chips in its dashboard displays since 2011. Huang opened with Musk's claim that AI was more dangerous than nuclear weapons ("I said 'potentially'") and that it was like "summoning a demon," which drew a glare. He moved on to self-driving, which Musk called almost "a solved problem," a few years away. Musk ended musing that it seemed strange to be alive just as AI arrived. The real conversation happened backstage: with Horstmann and cofounder Chris Malachowsky looking on, the two traded ideas so fast it sounded like a data link, and even Malachowsky was lost.
Two visionaries
Witt lists the obvious likenesses: immigrants, workaholics, visionaries, screamers, gamblers, world-class engineers, both able to make barren, start-up-littered markets bloom. The differences take a sharper eye.
| Dimension | Jensen Huang | Elon Musk |
|---|---|---|
| Vision | Forward from reality: today's circuits, projected | Backward from fantasy: Mars first, then the tools |
| Loyalty | Almost never fires; warnings and an improvement plan first; reassigns staff when units close | Fires arbitrarily; once the whole Starlink engineering team, on a Sunday |
| Tenure | Decades-long careers and boomerangs (Catanzaro, Aarts); at the 2019 founders' reunion Curtis Priem, back after sixteen years, was amazed who remained, including Jeff Fisher's children | High churn |
| Private life | Stable marriage; warm, funny, empathetic when not exploding | At least eleven children with at least three women; awkward, says he is on the autism spectrum |
| Public voice | Whiskey, no tweets, no political opinion in forty years; Witt found no federal donations by him or Lori | Smokes weed, "posts cringe," funded Donald Trump |
| AI risk | Zero | Potentially extinction-level, a view shared by Hinton and Sutskever |
OpenAI, and the one who walked away
By late 2015 Musk concluded the way to protect humanity from AI was to build it himself. OpenAI launched in 2015 as a nonprofit aiming to benefit humanity "unconstrained by a need to generate financial return." It raised $135 million in donations; Musk gave about $45 million, by far the most, alongside Reid Hoffman (LinkedIn) and Sam Altman (president of Y Combinator). Witt notes the structure and pledges would later "create trouble for all involved."
- Founding talent. Karpathy; Wojciech Zaremba, who had cloned AlexNet at Google; CTO Greg Brockman, a North Dakotan early Stripe employee; and, most important, Ilya Sutskever, present at AlexNet's creation and a guide of the field since.
- Krizhevsky opts out. Never a natural collaborator, he left Google in 2017 with just under $15 million from the auction. A Japanese TV crew found him in 2019 in a bare flat above a Vietnamese restaurant with a desk, couch, digital piano, television and a cat. Witt's "Orville Wright of the neural net" said that after about ten years in any speciality he loses interest.
The DGX-1
As OpenAI assembled its team, Huang set out to build it a computer ten times more powerful than Nvidia's priciest product, a desktop box for science and visualisation. The DGX-1 would focus on matrix multiplication, which Witt says was to AI what Quake had been to graphics.
FinFET transistors
- The shape. TSMC's new FinFET process made transistors that rise from the silicon like shark fins, about 200 atoms tall, towering like Soviet apartment blocks to a shrunken observer. They are sculpted with ultraviolet light at a precision engineers compare to hitting a quarter on an Arkansas sidewalk with a laser fired from the moon.
- The gain. Flat transistors leaked like an old garden hose; fins stopped the leaks and added, in effect, a nozzle with spray settings: the trinity of precision, efficiency and control.
- The indifference. It was the most important physical upgrade to transistors since the 1970s, and hardly anyone outside the trade noticed; Witt compares the industry to a dynasty sports team whose wins everyone takes for granted.
- Going vertical. Inside Nvidia the design "maze" grew from tennis-court to Rhode Island scale (in Witt's running hair-width metaphor); Daedalus, he says, would have wept.
Pascal, NVLink and the box
The first FinFET architecture was Pascal, after Blaise Pascal, inventor of the first mechanical calculator: a nod to computing's historic bottleneck, arithmetic speed, from gear-driven adders through ENIAC to Intel and IBM chips. The P100 (April 2016) calculated faster than data could be delivered to it, so the bottleneck moved from computing to feeding. Nvidia had been building the fix since 2014: NVLink, a "homework cannon" firing a million semesters of matrix exercises a second, which wired eight P100s into one super-processor.
April 2016 Pascal P100 released.
August 2016 Huang, in leather (now all he wore), wheels the first DGX-1 on a dolly into OpenAI's Mission District office: pebbled metal case, 134 lb, $129,000, a clothes dryer's power draw, a "data center in a box." Musk slits the packaging with a box cutter; Huang signs it "To Elon and the OpenAI Team! ... the world's first DGX-1!"
The next week A second signed unit goes to Fei-Fei Li's Stanford lab.
Days later The 2016 ImageNet contest: top multi-layer systems tag images at 98% accuracy against a human average of 95%.
Witt ranks the DGX-1 with ENIAC and the Apple II among the most important computers built, a machine meant for nothing except training ever more powerful AI. After six decades and tens of billions of dollars, a computer could finally tell a spatula from a hatchet; he dates the start of machine supremacy here.
Witt's Huang-Musk table is vivid but tilted: his access was to Huang, his material on Musk mostly public record. "Zero risk" is Witt's paraphrase of Huang's stance, not a measured position Huang published. The 98% versus 95% ImageNet figures are top-5 accuracy, and the human benchmark came from one researcher's own test (Karpathy's, about 5% error).
OpenAI's nonprofit structure did "create trouble." Musk sued in 2024, alleging Altman and Brockman abandoned the charity. On 18 May 2026 a unanimous Oakland jury found his claims time-barred after a trial before Judge Yvonne Gonzalez Rogers; Musk said he would appeal. Trial reports put his early funding at about $38 million, below Witt's figure. OpenAI was then valued at roughly $850 billion and preparing for an IPO.
The Good Year
In 2017 Nvidia's revenue doubled and profit tripled to $3 billion, as it stole the Nintendo Switch from its cousin-run rival AMD, powered both science Nobels, rode (and was later fined over) a crypto boom, opened a spaceship headquarters and began simulating reality itself. Witt's point is that by year's end Nvidia was the AI company, worth $100 billion, though almost nobody outside Wall Street knew it.
Cousin Lisa
The stock rose 224% in 2016, making Huang a billionaire again, yet it was not the S&P's best performer: AMD rose 309%. Huang hated second place and answered with what Witt calls one of the best corporate years ever: revenue doubled, profit tripled, products launched at dizzying speed.
AMD had been Nvidia's only real GPU rival since buying ATI in 2006 (GeForce and Radeon were the Coke and Pepsi of graphics). From 2014 it was personal: AMD's new CEO, Lisa Su, was Huang's relative.
- The family link. Huang's mother had at least eleven older siblings; one was Su's grandfather, so the two are first cousins once removed. The tie is thinner than blood suggests: as one of the youngest, Huang's mother was given away to her father's friend, an old custom, and grew up apart. Huang learned of Su only when she became CEO.
- Su's background. Daughter of a statistician and an accountant; came from Taiwan to New York at three; "tiger parents" offered engineer, doctor or concert pianist, and she chose engineering as the hardest. Mild New York accent, pantsuits, short hair, and, says AMD's Forrest Norrod, Huang's same conviction and courage in bad times.
- Public warmth, private rivalry. Huang calls her "terrific" and says "we're not very competitive," though Nvidia staff can recite relative market share from memory. Kirk believes Huang stopped trash-talking rivals in public after 3dfx's bankruptcy lawyers made him read his old insults aloud in a videotaped deposition, but that the old Jensen re-emerges behind closed doors. Analyst Hans Mosesmann: he does not want to lose, "but he especially does not want to lose to Lisa."
Jensen Huang. Temperamental and expressive, no poker face (though "he'd still find a way to beat you"). Sails toward the horizon to find empty markets and avoid fights.
Lisa Su. Reserved and stoic, "a great poker face." Prowls around the incumbent until it falters; revived AMD by taking Intel's CPU share, once thought impossible. In Witt's view braver, since she did not run from Intel.
AMD's long road to Su
Huang's history with AMD was already tangled: his first real job, a few AMD shares bought through the 1984 employee plan that he still holds, and a 2006 offer from AMD to buy Nvidia. He was open to it until he learned he would not run the combined company, then ended the talks. He had no exit plan and no intention of ever leaving the CEO chair.
AMD bought Toronto's ATI instead and blundered on: three CEOs in six years; a hilltop Austin campus finished in 2008 and grandly branded "The Summit at Lantana" (the hill was a few hundred feet); its sale and leaseback five years later to placate creditors. By 2014 the stock was $2. The board made Su, an internal VP aged 45, CEO expecting a hospice manager; she became the era's most celebrated chip CEO after Huang.
Now Su was prowling around Nvidia. In 2016 AMD released an open-source CUDA clone for Radeon; developers found it inferior, and AMD's cloud GPUs did not fetch Nvidia's rental premium, but she was willing to wait years. Norrod: "We think we can compete with Nvidia."
ATI had supplied several generations of Nintendo hardware, including the GameCube and Wii, and after the Xbox collapse Huang kept saying consoles, refreshed only every five years, were not worth his time. So everyone, Su included, was surprised when Nintendo chose Nvidia for the Switch.
- The safe. Nintendo was insular even by Japanese standards: run from conservative Kyoto by designer Shigeru Miyamoto (creator of Super Mario Bros. and Zelda in his early thirties, now in his sixties) and a small clique. Hardware and software teams sat side by side; Miyamoto once objected to the stitching on Mario's trousers in a US advert, yet pushed bold risks.
- The chip. Nvidia sold the Tegra, a repurposed phone part engineers called "floor scrapings." It is a system-on-a-chip (CPU, GPU and more on one die), unremarkable in speed but frugal with power, so the console could be undocked and played for hours under the bedcovers.
- The mystery. How Huang won remains secret; Witt suspects the real decisions came late at night in Kyoto izakayas. Analyst Jon Peddie had thought ATI's position safe; Nintendo's post-decision meeting was "incredibly friendly, but stony."
- The outcome. About 140 million Switches sold, by far Nintendo's most profitable product, and Huang barely mentioned it: ambushing his cousin was a side project.
Two Nobels for GPU-powered tools
Both 2017 science Nobels went for new tools, not new discoveries, and both depended on GPUs.
| Prize | Tool | What GPUs did | What it revealed |
|---|---|---|---|
| Physics | LIGO: twin pairs of crossing laser beams about 1,800 miles apart | Assembled billions of data points (like Lego bricks) into 3D models in parallel | Two black holes spiralling for millions of years and colliding, rippling space-time; Science's breakthrough of the year |
| Chemistry | Cryo-electron microscopy: specimens frozen in transparent amorphous ice and hit with electron beams | Reconstructed 3D structure from huge point clouds | Real biological gears in a single-celled creature, spinning to drive it like a motorboat; structures earlier crushed between slides |
Witt stresses the renders are computer reconstructions, not photographs, but the black holes and nature's clockwork had waited millions of years simply to be seen. The AI frenzy hid such contributions. Still, laureate groups in two fields in one year proved CUDA's value to humanity without proving it paid. Bill Dally doubts science alone would have been profitable: the lab-supercomputer market is worth "probably no more than what it costs to develop each generation of GPUs." AI arriving was good luck as well as vision.
Crypto, and the cost of silence
Also in 2017 a less reputable customer found CUDA. Cryptocurrency miners mint coins by cracking ciphers with brute force, processing similar data chunks millions of times in parallel, a natural GPU job; a well-tuned rig minted a coin in about four hours.
- Bitcoin passed $1,000 for the first time in January 2017, quadrupled twice, and topped $16,000 by year-end.
- GeForce cards sold out in shops and fetched about double list price on eBay.
- "Superminers" racked GPUs in bedrooms and garages, hot enough to melt roof snow; East Wenatchee, Washington, in "hydro alley" with reputedly the nation's cheapest power, became the mecca.
- Nvidia's stock moved in step with bitcoin for a while.
- The SEC later charged Nvidia over undisclosed crypto sales; it paid $5.5 million without admitting or denying.
Many staff were appalled, some on climate grounds; most engineers Witt met called crypto an idiotic distraction. Huang, reasoning from first principles, did not foresee crypto displacing government money, but as a businessman he neither discouraged nor endorsed miners. Nvidia's official stance was silence: earnings calls stressed self-driving cars and AI, played down crypto and refused to estimate miners' share; mining showed up only as a suspicious bulge in "gaming."
The SEC settlement came in May 2022, a date Witt omits, for disclosures in fiscal 2018. Nvidia then shipped a miners-only CMP card (a GeForce with video ports removed) and throttled mining on standard cards to protect gamers and scientists. After bitcoin crashed in 2018, purpose-built mining rigs displaced GPUs and the largest other coin (Ethereum) later dropped brute-force mining entirely; Nvidia gladly ceded the business.
Endeavor, the spaceship
In January 2017 Nvidia reported its first $1 billion annual profit and soon hired its 10,000th employee, outgrowing the dull modernist campus it had used since 2001. A neighbouring lot bought years earlier for a command centre had stalled after Bumpgate; in 2017 the new headquarters, Endeavor, opened.
- Form. A huge triangle with trimmed corners, the motif repeated in couches, carpets and even urinal splash guards; hospital white and matte black, multistory living walls, staff called it the "spaceship."
- Architect. Hao Ko of Gensler, a soft-spoken junior whom Huang made lead designer over his boss. Ko's explanation: Huang "will undress you" and wanted someone he could push; "ninety percent of architects would battle back. I'm more of a listener."
- The roof. Huang put Ko in a VR headset wired to a GPU rack to simulate light ("the first building that took a supercomputer to design"). Ko extended the roof like a hat brim and placed hundreds of triangular skylights along its crinkle-cut folds at mathematically optimised spots, so the floor is always naturally lit but never glared.
- Sight lines. One can see a couple of hundred yards edge to edge, giving Huang a 360-degree view of staff. He used Ko's third-floor executive suite to store books and ran Nvidia from an anonymous central conference room turned war room, at the centre as he had once sat by the fridge in Priem's house.
Ko watched Huang tear into engineers over the headset's render time, which then fell from five hours to ten seconds. Ko's point: at five hours he would settle for an adequate shade of green; at ten seconds he would find the best one. Speed changes the quality of decisions, not just their pace.
The building won awards, made Ko's career and gave Nvidia a landmark to rival the Googleplex and Apple's Infinite Loop. Perks were sparse: a top-floor bar named "Shannon's" after former general counsel David Shannon, who had questioned mixing staff and alcohol, but no gym, climbing wall, dog park, disc golf or ball pit. "You come here to go to work." Ko was already designing Voyager, a larger opposing triangle to the north (Endeavor + Voyager = NV), with a third, bigger building planned to keep tiling the valley. On the opening tour Huang fixated on drinking fountains beside the bathrooms, where building code required them: "I'm never satisfied. No matter what it is, I only see imperfections."
Leasing compute, and simulating everything
CUDA was downloaded 2.7 million times in 2017, nearly triple 2016 and 15 times the 2012 low, mostly for AI rather than crypto. Students drove much of it; Stanford's most popular course was CS 229, which Witt calls "Introduction to Deep Learning."
Many developers never bought hardware, running programs or whole companies on leased virtual machines. Amazon Web Services held 50% of the cloud market and in good years earned more than Amazon's e-commerce arm; Microsoft Azure, revived under Satya Nadella, chased it. Cloud purchases doubled Nvidia's data-center chip sales that year. Huang noticed that leasing beat selling and launched two leasing platforms.
- GeForce Now. Rented virtual graphics cards let gamers play high-end titles on laptops and cheap PCs. Physical GeForce sales were still the majority of revenue, but cloud play implied gamers needed only a monitor and broadband. Huang was preparing to kill his own 25-year core business before someone else did.
- Isaac. Huang watched USC researcher Yevgen Chebotar train a robot arm by reinforcement learning to hit an orange ball with a hockey stick, his team resetting the ball by hand thousands of times. Huang's career rule was to simulate whatever he could; now he decided to simulate everything: "We have to build an alternative universe," where the arm could take a billion shots, as Gerald Tesauro had simulated backgammon with millions of dice rolls.
- The physics problem. Game engines were nowhere near (strip the textures, Peddie says, and you have blocks bumping). A reality simulator needed true elasticity and density: a tennis ball must respond unlike a billiard ball, cloth must give, metal hold, a wet plate slip, fruit bruise. It would be costly and slow, with perhaps five or six customers; Huang loved it.
- Omniverse. Isaac grew into the Omniverse, an "industrial metaverse" or Earth's "digital twin," leased (not sold) to robot makers, carmakers, industrial designers and warehouse builders to test products before breaking real ones.
The AI company nobody saw
By the end of 2017 profit had tripled to $3 billion with growth in every line, and Nvidia was worth $100 billion. To the few members of the public who had heard of it, it was a graphics or crypto-mining brand. Only savvy investors saw that it was the AI company, with AI now quietly everywhere (face recognition, product recommendations, social feeds, phone voice quality) and nearly all of it, as Andreessen had said, on the Nvidia stack. The Fidelity managers who grilled Huang in 2013 now held an outsized stake in the market's best investment. Witt closes by warning that none of this prepared anyone for the metamorphosis coming next.
Nintendo kept Nvidia for the Switch 2 (launched June 2025, custom Nvidia T239 chip). Su's AMD became Nvidia's main merchant-GPU rival in AI with its Instinct MI300/MI350 lines and a multi-gigawatt OpenAI supply deal in October 2025. Two small slips: Su is about six years younger than Huang, not seven, and Stanford's CS 229 is "Machine Learning" (deep learning is CS 230).
The Transformer
In 2017 eight Google researchers published "Attention Is All You Need," an architecture so simple that its core ran to barely twenty lines of code and so well matched to GPUs that it got smarter the more data it was fed. Witt's point is that Google invented the transformer and then failed to ship it, leaving OpenAI, which was free to launch something bad and iterate, to scale it into GPT. By 2019 OpenAI had turned capped-profit, taken $1 billion from Microsoft, and set out to build a model bigger than a rodent's brain, on what Huang now called an AI factory.
Self-attention
Jakob Uszkoreit was, in Witt's phrase, born into computational linguistics: his father Hans spent his career teaching computers language. American-born, raised in Germany, long-haired and often man-bunned, Jakob wanted to do something else (perhaps biology), but after joining Google in 2008 he found language the most interesting problem and entered the family field, soon surpassing his father.
The problem with recurrent nets
- Grammar errors. However trained, early neural language models kept making elementary mistakes.
- Explicit rules don't scale. Teaching grammar like a Latin teacher could not keep up with real language.
- Memory hacks. Recurrent neural networks with built-in long- and short-term memories were finicky to program and, fed more text, sometimes untrained, forgetting what they knew.
Uszkoreit's bet (around 2014)
He wanted a net that improved the more it read, and reasoned: if more data gives better results, the processing structure should be as simple as possible. The biological hint: of the brain's roughly 100 billion neurons, under 1% handle language, which suggested to him that language evolved to exploit our cognition efficiently.
- Context only. Strip out all memory structures and replace them with a simple knowledge graph of links between words.
- Meaning is the map. A word alone is an arbitrary sound. Links to "hop," "green," "tongue," "flies" and "amphibian" pin down "frog," and that map should look the same in German, French, Swahili or Vietnamese; the letters are placeholders.
- Weighted trees. Each word is a tree of statistical weights. In "The orange ___ caught the brown mouse," frequent cat/mouse pairings make "cat" likely; "cat" links strongly to "caught" and "ate," weakly to "brown," barely to "the." With enough examples the model might learn that "orange" modifies "cat" with no grammar lessons.
- Tokens. Harder words are split into fragments: "un" + "happi" + "ness" (negating prefix, root, noun-forming suffix), each linked in the same weighted way.
- Built for the GPU. Recurrent nets fought parallel hardware, minimising data and maximising clever sequential code. Self-attention does the opposite: huge volumes of text pushed through a simple funnel of many simple computations in parallel, the statistical property GPU hardware rewards.
Google colleagues raised eyebrows at an idea that discarded every existing architecture; his father was sceptical too. Yet self-attention worked at once, and pieces of it went into Google's search and advertising products.
Building the transformer
The team assembles
- Illia Polosukhin. Champion coder at the Googleplex, drawn by language's uniqueness: thousands of species see, only one understands language. Watching Denis Villeneuve's Arrival (2016), where alien heptapods paint circular inkblots each holding a whole text, he saw that self-attention could link each word to its sentence and also to thousands of words across an entire text, so a clue paragraphs back could inform the next word.
- Ashish Vaswani. Joined the pair; by early 2017 the three had a crude English-to-German translator. (Uszkoreit and Polosukhin had earlier worked on "autobot," an internal project to write Wikipedia pages automatically.) The mechanism was now called a transformer.
- Four more researchers joined over the following months; by February 2017 the translator matched the best recurrent nets.
- Noam Shazeer. At Google since 2000, an expert coder fed up with recurrent nets, the eighth and final member. With Welsh programmer Llion Jones he turned the research code into production-quality software.
As data increased the results kept improving and beat public Google Translate. Shazeer: as you make it bigger, "it's clearly just kind of more intelligent," which had not been true of their earlier work.
What it learned
- One token at a time. Older architectures tried to build whole sentences or paragraphs; the transformer predicts exactly one next word and never looks further. Aidan Gomez: generating sequences in order forces the model to learn extremely complex behaviour, until it seems to understand culture and human interaction.
- Beam search. Always picking the top word made clunky prose, so Jones let the model pick among several candidates, like a writer with a thesaurus (top, second, third, top again). Hellish to implement, it first delivered state-of-the-art results.
- "It just fell out." Jones's visualiser drew word relationships as lines of varying thickness. Given "The animal didn't cross the street because it was too tired" and "...too wide," it linked "it" to the animal in the first and the street in the second, solving one of computational linguistics' oldest problems without anyone aiming at it.
- Unnamed structure. Gomez says the model learns relationships we have no words for, such as English adjective order ("an old Canadian maple tree," never "a maple Canadian old tree").
AlexNet was a rickety proof of concept; the transformer was built from the start, on Shazeer and Uszkoreit's whiteboard, for hyperscale: massive data, massive parameter counts, massive GPU clusters. It does the simplest thing imaginable, one step at a time, and that made it a skeleton key: predicting the next note or next pixel worked as well as the next word, so it soon wrote music and produced recognisable art.
Stripping it down
Work turned frenzied; after one late crunch a sleep-starved Vaswani saw neurons in the office curtain. In the last phase the team ran ablations, switching off parts of the code to measure their contribution, and found the core ran better without them. Shazeer cut so much that the transformer's primitive form was barely twenty lines of code.
The paper went to Neural Information Processing Systems (where AlexNet had appeared) in 2017. Jones joked a Beatles-style title, "Attention Is All You Need," not expecting it to stick; he later met people with it tattooed on their arms.
In July 2017, shortly before publication, Shazeer and Lukasz Kaiser fed the transformer millions of Wikipedia articles and asked it to generate rather than translate. Prompt: write an article about "the Transformer." Out came about a thousand words on a Japanese new-wave punk band called Transformer, fluent, confident, with fabricated footnotes, and entirely invented. Gomez felt his timeline collapse: models that could barely spell yesterday wrote compelling English today, decades early.
Google stalls
The authors expected products; management did not see them. Polosukhin blames a search monopoly that had made Google bloated and risk-averse: nothing could launch that "doesn't fit into the search box." Fifteen years earlier, he says, Google would have launched something bad and improved it relentlessly; it had lost that mentality.
The authors drifted to start-ups, and by 2023 all eight had left. Shazeer cofounded the chatbot company Character.ai and was acqui-hired back in 2024 ("defecting pays," Witt notes). Jones, the last to go, still praised the eclectic group that formed organically around a promising idea: "Only Google could have developed this product." Witt agrees that was probably true, and that Google's failure to exploit it left a giant hole for someone else.
Witt calls NeurIPS a "journal"; it is a conference (the paper appeared at NeurIPS in December 2017). He counts eight authors but names seven, leaving out Niki Parmar; Gomez was a Toronto intern at the time. Microsoft's "$100 billion cap" is Witt's arithmetic from the 100× rule, not a disclosed term. The "next target" became GPT-3 (2020) at 175 billion parameters, overshooting the 100 billion "rodent" figure.
OpenAI pivots to GPT
Musk exits, Sutskever switches
Since the DGX-1, OpenAI had struggled; its first big project was an "artificial gamer" for the real-time strategy game Dota 2, and some wondered if that was the best use of world-class talent. Elon Musk, frustrated, began trying to recruit its engineers to Tesla. In February 2018, after fraught boardroom manoeuvring, Musk was sidelined and Sam Altman, ex-head of Y Combinator, took control. At his last meeting Musk said he would pursue AI at Tesla; when a young researcher warned this would worsen the arms race, Musk called him a "jackass" and stormed out, per The Wall Street Journal.
Ilya Sutskever stayed. Now rivalling Hinton in citations, he had let grooming go (tangled beard, untrimmed brows, thinning hair) and kept his signature gift, pivoting fast to a good idea. Around the Musk blow-up he watched Shazeer present the transformer at a conference and told colleagues to drop the game bot: "Literally the next day ... we switched to transformers right away." Altman agreed.
The plan built on Shazeer and Kaiser's fake-Wikipedia demo: first pretrain on a large text collection, then generate text in answer to any prompt. Purpose, method and architecture gave the name: Generative Pre-Trained Transformer, GPT.
| Model | Released | Training data | Parameters (Witt's brain analogy) | Result |
|---|---|---|---|---|
| GPT-1 | June 2018 | BookCorpus: ~7,000 free self-published ebooks, heavy on Twilight knockoffs | ~100 million (insect) | Dadaist nonsense |
| GPT-2 | Feb 2019 | ~8 million web pages, ~6 billion words | 1.5 billion (small lizard) | Sometimes humanlike prose; zero-shot answers |
| Next target | 2020 | Far larger | ~100 billion (rodent) | Aimed past the human brain in time |
GPT-1 and GPT-2
GPT-1 was as bad as its vampire-romance syllabus, and Musk emailed that OpenAI's probability of being relevant to DeepMind and Google without dramatic change was "0%." But OpenAI, unlike Google, could launch something bad, even terrible, and improve. GPT-1 showed that the approach could work; the key, as Sutskever had seen in 2012, was scale.
- The unicorn story. Given a prompt about English-speaking unicorns discovered in an Andes valley, GPT-2 continued fluently, naming them "Ovid's Unicorn," four-horned and silver-white, the best prose any language model had produced, from nothing but next-word statistics.
- Zero-shot questions. Asked things no web page answered directly, it named Charles Darwin for On the Origin of Species, Tesco as Britain's largest supermarket chain and the Tiber for Rome, and wrongly named Arnold Schwarzenegger as John Connor (he played the cyborg).
- Emergence. Such untrained skills appeared as models grew, surprising their designers; past the threshold, no one could say what a model could fully do.
- Limits. It summarised arguments badly and could stumble counting to ten.
Sutskever's worry, and his appetite
Sutskever began to ask whether the transformer was a first step to AGI, defined as software able to do any task a human can. The questions that had founded OpenAI returned: an AGI doing its own AI research in an endless self-improvement loop; Bostrom's sudden tip from sort-of-smart to hypersmart; whether operators would notice, and whether it would let them survive. He did not think the current architecture led straight to AGI, but he had seen two leaps in five years (AlexNet, the transformer) and assumed someone was building the next one. The question came to consume him, yet it did not restrain his ambition.
- Model size is counted in parameters (weights), each roughly a synapse; the human brain has perhaps 200 trillion, and OpenAI meant to surpass it, AGI or not.
- Training electricity alone made GPT one of the costliest computing applications yet.
- 2019: Altman, now CEO, admitted donations could not fund this and created a "capped-profit" arm limiting backers to 100× their money, advising investors to view it "in the spirit of a donation."
- $1 billion from Microsoft, the largest early "donor," with up to $100 billion of potential return: a millionfold step up from the $1,000 of GeForces behind AlexNet seven years earlier.
- Nvidia was already building the warehouse-scale stack to supply it, which Huang now called an "AI factory" rather than a supercomputer or data center.
Every frontier model of 2026 (GPT-5 series, Gemini 3, Claude) is still transformer-based, often with mixture-of-experts layers Shazeer pioneered. Google's August 2024 deal to rehire Shazeer reportedly cost about $2.7 billion in licensing. OpenAI completed its conversion to a public benefit corporation in October 2025, with Microsoft holding roughly 27%.
Hyperscale
Huang treated AI as software running on his hardware and cared only about scale: whole data centers acting as one chip. He bought Mellanox to wire them, failed to buy ARM, and widened a moat that, Witt argues, was built from code. Nvidia won not because its circuits were better but because its software was: of a 1,000-fold speedup in a decade, only 2.5× came from transistors.
One chip the size of a building
Huang did not see AI as an emergent superintelligence and waved away biological analogies; to him it was software on hardware Nvidia sold. He pushed that view in company-wide talks, so staff spoke of AI's powers with less excitement than outsiders expected. For Huang the real feat was the computing power packed into data centers.
- A "supercharged law." At GTC 2018 he said Nvidia GPUs had become 25× faster in five years, far outpacing Moore's Law, and showed that AlexNet, a week's training on 2012 cards, now trained in 18 minutes on the new DGX-2.
- One chip. He saw a data center not as many chips but as one unified chip working on one problem.
- The bottleneck. Thousands of GPUs moved bits so fast they clogged the networking. Huang went shopping.
Israel's Mellanox made Infiniband, the world's fastest networking standard. Founder Eyal Waldman, burly and garrulous, shared Huang's taste for overkill: he had overspent building Infiniband for the small supercomputing niche, a powerful product in search of a customer, like CUDA.
- The chain. Overbuilt capacity meant low profits, a low stock and unhappy holders, which summoned Jeff Smith of Starboard Value, by 2017 reportedly compounding above 15% a year.
- The raid. Starboard bought about a tenth of Mellanox and tried to unseat all eleven directors at once. Waldman postponed the meeting to gather votes; facing a proxy war, he conceded three seats and, across the table from Smith's team, put the company up for sale.
- The auction. Cloud demand made Ethernet look tired for high-end data centers, and seven bidders came. In March 2019 Nvidia beat Intel with $7 billion; at closing in 2020 it gained 3,000 staff, rebranded Nvidia Israel.
- The verdict. Smith doubled his money, but board member Mark Stevens calls it one of the most profitable acquisitions in memory, and a later Starboard filing called Huang "a real visionary in this sector."
The cables, switches, routers and networking chips turned the AI factory from a brain into an integrated nervous system, a rare case of real corporate synergy. Emboldened, in September 2020 Huang offered $40 billion for UK chip designer ARM, the largest semiconductor merger ever proposed. US regulators sued in 2021 to block it as anti-innovation; China and the UK were also sceptical, since Nvidia, Intel and AMD were now fighting over data centers and vertical integration by one could hurt the other two. Huang, unwilling to fight Washington, dropped it.
The moat is software
Nvidia's pricing power pushed gross margins on AI chips above 90%, which drew competition like chum draws sharks.
- In-house chips. Google and Tesla built their own AI-training hardware to cut dependence.
- Start-ups. Cerebras made a dinner-plate-sized "mega-chip"; CEO Andrew Feldman says Nvidia is "extorting" customers and nobody will say so.
- Open alternatives. AMD and Intel offered open-source CUDA substitutes that could free buyers from Nvidia's upgrade ladder and save billions. Few researchers used them.
Witt asked many people why, and rejects the two commonest answers:
- "Switching is hard." Wrong: sometimes changing a couple of lines of code is enough.
- "Buyers play safe." Hans Mosesmann's "no one ever got fired for buying Nvidia" echoes the old IBM line, but an actual buyer told Witt he tested rivals constantly and longed for something cheaper, because the Nvidia stack is extremely expensive.
Where the speed really came from
Witt's answer is the maths. With Moore's Law dead, only a small share of gains came from more transistors. The rest came from Bill Dally, Ian Buck and colleagues accelerating matrix multiplication with numerical tricks:
- New instructions that act like speed-solving algorithms on a Rubik's cube.
- Lo-fi data types replacing the processor's native formats, like swapping calligraphy for shorthand: ugly but effective.
- Pruning "dead" synapses from the matrices, deleting unproductive information as the memory-erasing machine does in Eternal Sunshine of the Spotless Mind.
From 2012 to 2022 single-chip AI inference performance rose about 1,000×; transistors supplied about 2.5×, the mathematical toolbox most of the remaining 400×. Arjun Prabhu: AMD can make silicon as good as Nvidia's; "they just can't make the calculations go as fast."
Three hundred "treasures"
On that engine Nvidia stacked domain toolkits, given away with zero licence fees and pressed on scientists "like a grandmother distributing second helpings." Huang calls them his treasures.
| Toolkit | Field |
|---|---|
| Drive | Automotive research and self-driving |
| BioNeMo | Drug discovery |
| Clara | Medical imaging |
| Morpheus | Cybersecurity |
| Highlights | Capturing kill shots in Fortnite |
| ~300 in all by the 2020s | Gaming, animation, planetary science, climate, maths, physics, finance, biochemistry, quantum computing |
The generosity is strategic: free software locks researchers into the hardware-upgrade cycle. In early 2024 an administrator at Caltech's data center told Witt the wait for an H100 was almost 18 months; he urged professors to try other suppliers, and few would. "They'd rather wait for the hardware than switch away from CUDA."
Dwight Diercks and the crunch
Software chief Dwight Diercks runs 10,000 programmers: "We're really a software company; that's the thing people don't understand." In plaid shirt and jeans he looks, Witt says, like a cattle rancher abducted by a UFO, but he keeps his teams in perpetual crunch, and former programmers often cite burnout as their reason for leaving. Nvidia's tools are not always beautiful; some interfaces are a decade stale. Diercks cares only about being first to every new scientific frontier.
Arrive first with free, ugly-but-working tools in every new field, become the default, and let hardware sales follow. A rival with a more elegant, cheaper product arrives to find the standard already set. Hence Dally's indifference to open-source competition: "We're flooring it," always a couple of generations ahead.
Five things, every Friday
With no fabs and a remote-friendly culture, Nvidia barely noticed COVID-19 operationally, though the bar shut, GTC went virtual and Huang was stuck at home. In July 2020 Nvidia passed Intel in market value; he could not celebrate in person or crow to family (younger brother Jim still worked at Intel), so he walked his two dogs. Nvidia soon passed AMD too.
Housebound and running the world's most valuable chipmaker, Huang feared losing resonance with the front line but would not delegate. Around 2020 he asked every employee for a weekly email listing their top five current priorities.
- Volume. About 20,000 emails every Friday, which he randomly sampled late into the night.
- Replies. Hundreds of emails a day from Huang, often a few words; one executive likened them to haiku, another to ransom notes.
- Speed. Dally: email him at 2 a.m., reply at 2:05; again at 6 a.m., reply at 6:05.
Word of a large language model
Through those emails came news that OpenAI was running a job on Microsoft Azure that tied up 1,000 GPUs for a month and ran up a $5 million bill: a "large" language model, so called because it was about 100 times bigger than anything deployed. Huang told Bas Aarts to bin his recurrent-net compiler and build one for transformers immediately. Aarts was glad; recurrent memory structures were "very difficult to program and even more difficult to compile."
- Neural-net coders once used Torch, written in the niche language Lua; in 2016 Facebook AI Research ported it to Python as PyTorch, now the industry standard (Witt credits it with making Python the world's most popular language).
- Default workflow by the late 2010s: write in a Python notebook with PyTorch, compile to CUDA, run on an Nvidia GPU cluster, analyse in PyTorch, "repeat until sentience."
- Google's rival TensorFlow has fallen to low single digits of share.
- German programmer Piotr Bialecki has posted almost 40,000 times on the PyTorch forums since 2017 (about 15 a day), especially on CUDA integration; he joined Nvidia in 2019.
Secrets behind the NDA
Aarts had already seen "strange and terrible things" in demos months or years before release: a motion-transfer demo that swapped Tom Cruise's face in a Mission: Impossible film for that of someone in the room, flawless to his eye; and in 2019 a black "obelisk" that appeared on Endeavor's empty floor, running "PicassoGAN," which generated any requested scene (a man in open country with a river, trees, birds and a little house) instantly by pitting two neural networks against each other. Everyone was under NDA. Until about 2018 AI had grown through open academic collaboration; now breakthroughs came from skunkworks R&D, hinted at by cryptic Twitter accounts. Aarts: "People have no clue." Witt's closing line: he had as yet seen nothing.
Nvidia's 2019 demo was publicly called GauGAN, not "PicassoGAN." The "1,000 GPUs, a month, $5 million" figure undersells GPT-3: OpenAI's paper and Microsoft describe a cluster of about 10,000 V100s, with outside estimates of $4-12 million per run. The FTC sued over ARM in December 2021; Nvidia abandoned the deal in February 2022. The 1,000× inference gain is Dally's own Nvidia-sourced accounting, and Python's popularity owes much to data science generally, not PyTorch alone.
CUDA still dominates, but custom silicon has grown: Google's TPU v7 "Ironwood" is sold to outsiders including Anthropic (up to one million TPUs, announced October 2025), and Amazon's Trainium, Meta's MTIA and OpenAI's Broadcom-built chip all target Nvidia's margins. Nvidia's networking unit, the Mellanox legacy, booked about $11 billion in the quarter to January 2026 and over $31 billion for fiscal 2026, more than ten times its size at acquisition.
Money
ChatGPT turned the transformer into a public sensation in November 2022, and the spending boom that followed flowed through one company. All of the money, talent and innovation in AI, Witt writes, passed through a single corporate siphon: Nvidia. Huang got rich the way Hemingway's Mike Campbell went bankrupt, gradually, then suddenly, and so did the loyalists who had stayed with him through the lean years.
From GPT-3 to ChatGPT
What GPT-3 read
GPT-3 (2020) trained on more than a terabyte of text, about 100 billion words. The sources were buried in NDAs, but later analyses suggested a very liberal reading of fair use:
- All of English Wikipedia and a sweeping scrape of copyrighted web links, including New York Times archives back to 1851.
- The vampire-romance BookCorpus from GPT-1.
- A mysterious "Books2," widely suspected to come from Library Genesis, a shadow library of 4 million cracked ebooks long traded on peer-to-peer networks.
- Human fine-tuning afterwards to scrub the worst answers.
Jonathan Franzen, John Grisham, Jodi Picoult and George R. R. Martin were among authors who later sued OpenAI over similarities between their work and GPT output, as did the Times. GPT-3 amazed technologists with emergent skills (logic puzzles, working code) but left the public cold.
The launch
ChatGPT, released by Sutskever's team as a chatbot front end to OpenAI's latest models, went live as a beta on 30 November 2022 with no marketing and no paid tier: a bland grayscale page with a blinking cursor. Details were secret, since Microsoft had by now invested at least $10 billion in the capped-profit arm. It was tuned for conversation using internet text, YouTube transcripts and licensed third-party data; conservative estimates put it at a trillion-plus parameters, in synapse terms nearing a cat's brain.
It wrote sonnets, limericks and sestinas, screenplays, essays, stories, letters to the editor, working code and sound parenting advice. Students saw homework become obsolete; lawyers summarised briefs; applicants wrote cover letters; Witt used it to ask his town council for a traffic sign. Asked to make him cry, it told of two songbirds, one killed against a window while the other guards the empty nest; asked to make him laugh, it pictured a man picking his nose in the next car. Witt began to fear for his job and did not use it to write the book: "I was too afraid." Sutskever: the first use is "almost a spiritual experience," the sense that the computer understands.
- 1 million sign-ups in five days; 100 million monthly users by January 2023.
- March 2023: GPT-4 passed the bar exam; scored 5s on the Art History, US History, US Government, Biology and Statistics AP exams; hit the 99th percentile on GRE Verbal and the 92nd on the introductory sommelier exam.
- With a vision layer it explained the joke in a 1990s VGA cable plugged into an iPhone; a user later had it build a website from a napkin sketch.
- $100 million+ to train GPT-4, much of it reaching Nvidia via Microsoft.
- Inference could exceed half the training cost, ongoing: about a quarter of a cent per word, so roughly $10 for a 5,000-word term paper.
- The $20/month GPT-4 tier neared 2 million subscribers by March 2023.
GPT-3 had been essentially one giant network. GPT-4 used a mixture of experts: many networks assigned different jobs, for example one policing safety (refusing bomb-making or corpse-disposal questions), one writing code, one handling emotional tone. OpenAI declined to confirm the design.
The Cambrian explosion
Witt calls the merger of the transformer with hyperscale computing a Cambrian explosion of AI services.
| Product | Maker | What it did |
|---|---|---|
| Copilot | Microsoft | Code autocomplete programmers found indispensable; Huang: "The programming language of the future is human" |
| AlphaFold2 | DeepMind | Predicted proteins' 3D structure from amino-acid sequences, nearing "programmable biology" |
| DALL-E, Midjourney, Stable Diffusion | OpenAI and rivals | Artwork of any description, in any artist's style, in a minute or two |
| Udio, Suno | Start-ups | Songs in any genre |
| Jasper | Start-up | Marketing campaigns in seconds |
| Sora (2024) | OpenAI | Promised video of any description |
- Invisible by default. Insiders thought the real gains lay in back-end uses (more efficient power grids, faster airline scheduling, quicker deliveries), each unnoticed alone but adding up to a large, ongoing productivity upgrade.
- AI accelerating AI. In 2022 DeepMind unveiled a network to speed matrix multiplication; in 2024 Nvidia released generative-AI software to lay out patterns on its own chips. Self-augmentation stayed under human control, "for now."
- Sinister uses. Voice clones cast Trump and Biden as bickering Overwatch teammates; algorithmic sludge filled social media; deep-fake pornography targeted girls in American high schools; voice-cloned scammers faked kidnappings for ransom; in Hong Kong an architecture firm's finance worker paid out $25 million after a video call in which every colleague was a deep fake. Technologists' proposed remedy: more AI.
Everybody piles in
Microsoft, Meta, Tesla and Google announced multibillion-dollar AI programmes; carmakers promised autonomous vehicles, defence firms autonomous weapons. AI start-ups raised $50 billion in 2023 across medicine, education, business platforms and robotics, and the stock market rose on expected productivity gains. Honours followed: 2024 Nobels for Demis Hassabis (Chemistry, protein folding) and Geoffrey Hinton (Physics), and Huang's election to the National Academy of Engineering, which many thought overdue.
Gradually, then suddenly
Witt borrows Mike Campbell's line from Hemingway's The Sun Also Rises: bankruptcy came "gradually and then suddenly." Huang's wealth arrived the same way, as twenty years of accumulated bets paid out at once.
- Customers. At the start of 2023 Huang told investors that 50 of America's 100 largest companies had bought Nvidia GPU clusters and the other 50 leased them through the cloud.
- Mix. Data-center revenue passed gaming for the first time that year, and he expected it to double again.
- Share. With close to 90% of the AI chip market, Nvidia became a Wall Street obsession; CNBC ran a countdown to its earnings.
- 25 May 2023. At the Nasdaq open Nvidia gained about $200 billion; by the close it was the world's sixth-most-valuable company, worth more than Walmart and ExxonMobil combined. One analyst: in the AI war, "Nvidia is the only arms dealer."
In late 2023 Huang ate at Nobu in Palo Alto with Elon Musk and Oracle cofounder Larry Ellison, worth over $300 billion between them. Huang was plainly the alpha: the other two spent the whole meal pleading for more AI chips. Ellison: "Elon and I were begging, I guess is the best way to describe it. An hour of sushi and begging." GPUs were by then valuable enough to serve as loan collateral.
The monopolist at sixty
A decade after pleading with Fidelity for his job, Huang at 60 was the planet's most celebrated monopolist. Taiwanese noodle shops he visited put his picture on the menu; his leather jackets appeared in the Times Style section. Besides homes in Los Altos and Maui, he was linked through a shell company to a $38 million, seven-bedroom mansion on San Francisco's billionaire Gold Coast, with library, gym and cinema. Witt adds his name to the roster of Jobs, Bezos, Gates and Zuckerberg.
The loyalists' fortunes
| Holder | Role | Nvidia stake (Dec 2024) |
|---|---|---|
| Tench Coxe, Mark Stevens | Directors since 1993; defended Huang against Starboard | >$4 billion each |
| Harvey Jones | Long-time director | >$1 billion |
| Colette Kress | CFO | ~$800 million |
| Jay Puri | Head of sales, joined 2005 | >$700 million |
| Brooke Seawell | Audit committee chair | ~$700 million |
| Jim Gaither | First lawyer; board only from 1998 | ~$500 million |
| Tim Teter | General counsel | >$400 million |
| Deb Shoquist | Operations chief | >$300 million |
| Chris Malachowsky | Cofounder; ~5% at last disclosure in 2001 | Worth >$100 billion today if held; says he sold "not much" |
- Coxe. He has backed hundreds of companies; asked if Nvidia was his best investment, he looked at Witt as if it were the stupidest question ever: "Uh, yeah." Witt later notes a founding stake in the world's most valuable company is by definition the best possible investment.
- The undisclosed. Dwight Diercks and Bill Dally are not formal corporate officers under Nvidia's flat structure, so their holdings are private; Witt sensed they were fine. David Kirk, granted an "irrefusable" equity package in 1997 and retired in 2018, kept some shares: "a lot more money than I ever thought."
- Malachowsky's mystery. He is absent from the Forbes billionaire list despite that 2001 stake.
Free money for the rank and file
Nvidia's employee stock purchase plan let staff buy at a discount to the lowest price of the previous two years. Purchases were capped, but as the stock went vertical the plan became free money, and anyone who maxed it out each year made the trade of a lifetime. Wealth reached deep into middle management, and newer staff grumbled that rich veterans were in "semiretirement." Executives disagreed. Gaming chief Jeff Fisher, one of the first thirty employees: "Many of us are financial volunteers at this point, but we believe in the mission."
For the old guard, purpose outweighed money: they had spent years explaining why they worked for a gaming company and correcting how people pronounced its name, and none had expected AI. Jay Puri: nobody could have dreamed that science-fiction ideas would become reality; he thinks the interesting work is only beginning and calls Nvidia, perhaps with bias, "the most important technology company of our time."
Witt's stake figures use the 5 December 2024 close of $145.14 and include family trusts. Parameter counts for ChatGPT and GPT-4 are unconfirmed rumours, and "Copilot" was GitHub's (Microsoft-owned), launched before ChatGPT. AlphaFold predicts proteins; Witt's aside about RNA bases acting as binary is a loose gloss. Both Nobels were shared: Hassabis with John Jumper and David Baker, Hinton with John Hopfield.
Nvidia's fiscal 2026 (to January 2026) revenue was $215.9 billion, of which data center was $193.7 billion (about 90%); by spring 2026 Amazon, Microsoft, Alphabet and Meta were guiding to roughly $700 billion of combined capex for the year, up about three-quarters on 2025. The copyright suits continue; Anthropic agreed a $1.5 billion settlement with authors over pirated books in September 2025.
Spaceships
Witt's first visit to Nvidia, in late 2023, shows a giddy workforce in a surveilled, spotless headquarters, demos that erase the line between rendered and real, and a CEO who flatly refuses to discuss AI risk. Nvidia was building what Witt calls the Manhattan Project of computer science, yet its executives treated questions about its dangers like questions about the washing machine. Only Bryan Catanzaro would engage, and his answer was serene techno-optimism. The chapter closes by naming the one constraint left: electric power.
Inside Voyager
Voyager, the larger of the two triangular buildings on Nvidia's Santa Clara campus, had just opened. Witt's notes on the space:
- Scale and centrepiece. About 750,000 square feet of naturally lit floor wraps around a three-story matte-black "mountain", climbed by staircases past living walls of stonecrop, ivy and fern; at the summit sits a lit marble bar under wooden pergolas, looking out over thousands of open cubicles and white pillars rising about 100 feet to the roof.
- Surveillance as housekeeping. Cameras and AI track employees through the building. Leave a meal on a conference table and the system dispatches a janitor within the hour: a human janitor, for now, Witt adds. He calls the place a panopticon and is surprised how happily staff work in it.
- The cult of "Jensen". A buzz crossed the floor whenever Huang walked by: smaller than Witt expected, dressed in black, alone with no entourage. Staff called him only "Jensen" and kept a menu of custom Jensen emojis in Slack to celebrate good news, such as Nvidia passing $3 trillion to become the world's most valuable company shortly after the visit.
- The workforce. Earnest, nerdy, unpretentious, many of them immigrants with shaky English. A 25-year-old hire, two months in, newly arrived from China with a computer-science master's degree and weighed down by the company-issued backpack, told Witt she woke up happy every day to come to work.
- The exception. The only unhappy faces were the quality-control technicians inside the mountain: pale young men in earplugs in windowless labs with metal shelving, pushing chips to the point of failure amid the scream of cooling fans. Witt's image is of dwarves under the mountain, and the noise sets up the chapter's last point about power.
The demo room: rendering versus reality
In a frosted-glass triangle in the executive suite, a demo specialist walked Witt through what Nvidia's graphics stack could now do.
First a ray-traced Japanese ramen shop (ray tracing has shipped on GeForce cards since 2018), with light glinting off a metal counter and steam over the broth; Witt could find nothing to show it was not real. Then "Diane", a hyperreal avatar speaking five languages, built by a generative model trained on millions of videos of real people. Her flaws sold her: blackheads on her nose, faint hair on her upper lip. The only tell was an odd shimmer in the whites of her eyes, which the specialist said the team was working on. Witt left dizzy, thinking of science fiction, of Genesis, and of the world his daughter would grow up in.
Huang's stated goal is to merge Nvidia's graphics research with its generative-AI research. Image models will soon render inhabitable 3D worlds full of realistic people; language models will parse voice commands in real time; joined, they will let users speak whole worlds into existence. Witt's footnote points to Google's Genie 2 (December 2024), which builds an explorable 3D world from a single picture.
How Diane was drawn
Diane used deep learning super sampling (DLSS): ray tracing lights the scene, then a neural net fills in the detail. In the demo only 1 pixel in 8 came from the ray tracer; the other seven were inferred by AI.
David Kirk. Ray tracing was the last frontier: once you can model the cascade of light, simulation and reality collapse into one frame. In film, ray-traced CGI is now in nearly every major release and indistinguishable from camera footage, so "Everything is fiction."
Bryan Catanzaro. Back at Nvidia running applied deep learning, he sees a stage past it: neural graphics, which stops trying to simulate physics at all and uses AI to paint plausible scenes in real time. It does not model light; it hacks human perception of light.
"I'm so tired of this question"
Witt asked Nvidia's leaders about the risks of what they were building and got shrugs.
| Question Witt asked | Answer | Who |
|---|---|---|
| Could AI kill someone? | Electricity kills people every year | Catanzaro |
| Will it eliminate art? | It will make art, and you, better | Dwight Diercks |
| Could AI become self-aware? | A creature needs consciousness and self-knowledge, so no | Huang |
| What about Hinton's warning? | Researchers are puzzled; maybe he wants attention for his work | Huang |
Witt raised the question in every interview with Huang and got variations of the same reply. He quoted Geoffrey Hinton's PBS line that humanity may be "a passing phase in the evolution of intelligence." Huang's suggestion that Hinton was seeking attention stunned Witt: Nvidia's fortune runs straight back to Hinton's lab, which Huang had acknowledged many times. To Witt this was not a crank with a placard but the greatest mind in AI, a direct descendant of George Boole, telling the world to be very worried.
Huang's fuller case was that AI is just data processing. His example: buy a hot dog and the machine suggests ketchup and mustard; is that the end of humanity? Society got used to cars, alarm clocks and phones, and will get used to a robot vacuuming the carpet; if you understand how it works, nothing about that world looks strange. Pressed further, he snapped that he was tired of the question and of theorizing without evidence.
Witt plainly sides with Hinton and frames Huang's dismissal as cold. Two cautions: he reports Catanzaro's benign-AI view without testing it beyond one counterargument, and Huang's "it's only data processing" line is commercially convenient for the firm selling the hardware. The source text also prints the Clarke passage below twice in a row, an EPUB extraction slip rather than Witt's intent.
The outlier: Catanzaro on Prometheus
Catanzaro's route back to Nvidia had been rough. Around 40 he accepted he was gay, separated from his wife and was dropped from the Mormon church choir. With his loud outfits, religious speculation and taste for literature and music, he stood out, although at Google he would have passed as ordinary. Nvidia was not a place of ayahuasca or polycules: in hundreds of hours of interviews, Witt notes, no one ever mentioned Burning Man.
Witt showed him a 1964 BBC clip of Arthur C. Clarke, recorded before Clarke co-wrote 2001. Clarke's argument, in brief:
- Future Earth's smartest inhabitants will be machines, distant descendants of current computers, which are morons now but will think within a generation and then outthink their makers.
- That need not depress us: humans superseded Cro-Magnon and Neanderthal peoples and consider it progress, so being a stepping stone is a privilege.
- Biological evolution is nearly over; inorganic evolution, thousands of times faster, is beginning.
Witt calls this the most optimistic reading of Nvidia's achievement: a new phase of evolution, beyond a business success. Catanzaro lit up at the clip.
- The Prometheus framing. The "conservative" view says Prometheus deserved his punishment for giving humans fire; the "progressive" one says the world can and should be improved. Humanity's problems need more intelligence, not less.
- No zero-sum game. Witt countered that humans, through farming, herding, mining and cities, wiped out rival species. Catanzaro's reply: a true AI has too much else to do in the universe to fight us for Earth's thin crust.
- Build elsewhere. An AI wanting a vast data center would not put it where people live; it would go underground, where space is enormous.
- Light-speed travel. AI needs no body; it can move along a radio signal to any antenna and spread through the solar system and beyond.
- No limbic system. Humans are territorial by wiring; an intelligent AI will not be, and will more likely want to take care of us.
Witt's dry aside: life as a zoo animal for a space computer is something to enjoy at leisure.
The last bottleneck
Catanzaro's talk of where to put data centers was practical, not idle. The fans in the mountain showed how much electricity Nvidia's hardware drew. GPUs had cleared the bottleneck of calculation; Mellanox's InfiniBand had cleared that of data throughput. What remained was how much electricity a single data center could consume: the limit was now power, the subject of the next chapter.
Compute (solved by parallel GPUs) → data movement (solved by InfiniBand networking) → power (unsolved). Each fix moved the constraint one layer down the stack, toward physical infrastructure.
Nvidia first passed $3 trillion in June 2024, then $4 trillion in July 2025 and $5 trillion on 29 October 2025, and stood above $5.2 trillion in May 2026. Witt's footnote on Genie 2 has been overtaken by Genie 3 (August 2025), which generates interactive worlds in real time, and by Nvidia's own Cosmos world models for robotics. DLSS 4 (January 2025) moved to multi-frame generation, in which AI produces up to three extra frames for each rendered one.
Power
Once GPUs and InfiniBand had cleared the bottlenecks of calculation and data, electricity became the limit, and the demand came mostly from inference, the everyday use of trained models. Nvidia knows computing revealed climate change, yet its chips were driving a global surge in power demand. Witt's warning is that disappointment, not competition, is the biggest risk to Nvidia: if buyers cannot turn hundreds of billions in hardware into products, demand and the stock collapse together.
Live wires in Loudoun
In summer 2024 a Dominion Energy lineman, in a crane bucket nearly 100 feet up, used an insulated "hot stick" to unhook a line carrying more than 200,000 volts from its crossarm, one of several thousand such jobs. The wires were being replaced while still live, because cutting power to the data centers was ruled out under any circumstances. Witt's verdict: human safety had become secondary to demand.
- Where. Loudoun County, Virginia, holds the world's largest cluster of data centers and is the hub of US consumer computing: a Google search from Manhattan is likely routed through Virginia.
- The scale of the grid build. Dominion took over a century to build its existing network and was projected to double it in under 15 years because of AI.
- Fossil fallback. Besides upgrading lines, Dominion was discussing reviving mothballed coal plants.
Inference, not training, eats the power
Running models costs more than building them. Inference once fit on a floppy disk (think backgammon moves); now users want term papers, search answers and meme voices. Witt's markers of the shift: OpenAI's GPT products claimed over 180 million daily users; Google was attaching AI answers to ordinary search; TikTok reels were narrated by AI voice avatars. Behind each, in an anonymous warehouse near some city, sat an Nvidia GPU.
- ~0.3 Wh for a standard Google search; 10× that with generative AI, enough to light a bulb for about 20 minutes.
- A 5,000-word term paper from GPT: about the energy of a microwave running for an hour.
- Chip power: A100 (2020) 250 W; H100 (2022) 350 W; B100 (2024) 700 W; B200 1,000 W.
- The 2016 DGX box drew as much as a clothes dryer; its 2024 successor, as much as a family home.
- The largest hyperscale sites draw gigawatts, more than a nuclear reactor, enough to power Minneapolis.
- One analysis: AI might require doubling US nuclear capacity in under 10 years; even conservative forecasts saw total demand up 20%.
The hope behind the spending was replacing labor. Witt's examples:
- Voice. In 2022 James Earl Jones licensed his Darth Vader voice to Disney to be cloned indefinitely; he died in 2024, and the voice may outlive him.
- Medicine. Image recognition promised to save billions a year in MRI analysis and put most radiologists out of work.
- Agriculture and retail. Amazon used AI to check delivered strawberries for bruises; farms were testing robot pickers.
Witt's bottom line: there was no realistic way to feed Nvidia's GPUs and still meet carbon-neutrality targets.
Witt says the H100's 350 W was 75% more than the A100's 250 W; it is 40% more. He quotes the lower PCIe figures; the SXM data-center versions draw 400 W and 700 W. The "180 million daily users" for GPT products looks like a monthly figure (ChatGPT reported about 180 million monthly users in early 2024).
The climate paradox
Nvidia's researchers point out that climate science is a child of supercomputing:
- Syukuro Manabe (Nobel Prize in Physics, 2021) showed in the late 1960s that trace CO2 traps heat, using a crude Earth model on a 70-ton IBM machine that drew as much power as ten city blocks.
- In the 1980s, NASA models correctly predicted a rise of several degrees Fahrenheit even though the measured trend then looked flat. They also predicted the upper atmosphere would cool and the layers compress as heat stayed near the surface; by the 2020s satellites confirmed the sky had contracted.
- Climate modellers have run on Nvidia hardware since the late 2000s; Nvidia has its own climate division and the Earth-2 simulation, aiming to forecast wind speed on a single city block.
- Chief scientist Bill Dally, who has seen warming's effects in the Sierras and the Caribbean, works on AI partly because he thinks it will speed a carbon-neutral future; he calls the planet's survivability a first-order concern.
Yet the same chips were almost single-handedly behind the global jump in electricity demand: the better they got at matrix multiplication, the hungrier they became.
The fortress wall
A DGX box is one brick. Customers stack them into superpods, the spine of a modern data center, bound with thick cable bundles and cooled by air and liquid systems. The consequences:
- Google. Constant Nvidia upgrades helped push its greenhouse emissions up 50% in five years, despite grid efficiency gains and a 2030 net-zero pledge; in 2024 a spokesperson reaffirmed the goal while calling AI's draw a "challenge."
- Microsoft. Emissions rose almost a third, driven by the cloud unit that trains and serves OpenAI's models.
- Scavenging. Cloud providers were trying to buy disused crypto-mining sites for their power hookups.
Bubble land?
The path from GPUs to profit was indirect, and three costs stacked up: silicon, electricity and talent.
- Talent. People who could reliably coax results from fickle nets were scarce and expensive. Not every firm was OpenAI with an Ilya Sutskever; the frustration Fredrik Dahl had trying to make a net play poker was now repeated in IT departments everywhere.
- Plateaus. Nets still stalled, with no obvious fix when they did.
- Corporate clones. Firms built in-house GPT copies on emails, mission statements, patents and legal memos, producing "knowledge engines" little better than toys.
- Premature layoffs. Executives in marketing, media and health care launched big initiatives, sometimes telling staff openly that jobs would go once the nets worked; much of the output was vaporware, late, expensive and barely functional.
In summer 2024 hedge fund Elliott Management told clients (as reported by the Financial Times) that tech, and Nvidia above all, was in "bubble land": AI was overhyped, many uses not ready, and many would never be cost-efficient, would use too much energy or prove untrustworthy. Elliott thought a single bad Nvidia quarter would deflate the whole tech sector.
No firm could afford to be the one without an AI strategy, so FOMO drove buying. The buyers must now ship products that justify the spend; if they cannot, remorse, slack orders and a cratering stock follow. Witt ranks this above any rival chipmaker.
Skeptics. The craze looks like the dot-com boom or the mortgage crisis: FOMO spending, unproven returns, untrustworthy apps, and an energy bill no one can pay cleanly.
Witt's counter. The buyers are seasoned engineers, not hucksters; the firms are stable and spend retained profits, not risky share issues or subprime debt; every academic computer scientist he interviewed called neural nets a civilizational advance; the earnings so far are real.
Witt's conclusion is that Nvidia's old valuation puzzle now applies to the whole stock market. Past statements cannot say whether the AI bet is wise, and earnings projections are tea leaves. The only real question was whether Huang, Zuckerberg, Musk, Nadella, Pichai and Altman, who had committed hundreds of billions to an unproven technology, knew what they were doing.
The IEA (Energy and AI, April 2025) projects data-center electricity use roughly doubling by 2030 to about 945 TWh, with AI the main driver. A Blackwell GB200 NVL72 rack draws about 120 kW. Microsoft signed in September 2024 to restart Three Mile Island Unit 1 (now the Crane Clean Energy Center), and hyperscalers have signed deals for new nuclear and gas plants. Big-tech 2026 capex neared $700 billion, increasingly financed with bonds and private credit (Meta, Oracle and others), which weakens Witt's "retained earnings" defense.
The Most Important Stock on Earth
On 21 February 2024 Nvidia reported a year of $60 billion revenue and nearly $30 billion profit, and the next day added $277 billion in value, then the largest one-day gain in market history. Nvidia's $3 trillion valuation, Witt argues, was really a GDP-sized bet on one 61-year-old man with 55 direct reports and no successor. Around that bet he maps the company's culture and workforce, its exposure to war in Israel and to China and Taiwan, the chip-fab race in Arizona, and the long argument over whether Huang was simply lucky.
Judgment Day
The run-up was frenzied. CNBC ran a countdown for days; the "FinTwit" crowd shared a meme of Huang in leather before a US electoral map with all fifty states in slime green; Goldman Sachs called Nvidia "the most important stock on earth." The call, starting fifteen minutes after the close, was one of the most listened-to in Wall Street history.
- Revenue more than doubled to $60 billion for the year, far above forecasts.
- Gross margin above 70% across all products (Apple, the market's favourite cash machine: 46%).
- Net income just under $30 billion, more than Nvidia had earned in the previous 30 years combined.
- About 30,000 employees, so roughly $1 million of annual profit per worker.
CFO Colette Kress read the numbers; then Huang, sounding to Witt like Thomas Edison, said neural nets could now understand and generate any information, from language to biology and the 3D world, and forecast a "trillion-dollar infrastructure cycle" of AI factories. His analogy: as AC power plants powered the last industrial revolution, Nvidia supercomputers are the "AI-generation factories" of this one, refining raw data into digital intelligence.
- +$277 billion of market value in six and a half hours of trading, more than the whole Coca-Cola Company.
- At least $65 billion of Nvidia stock traded, nearly a fifth of all US stock-market volume that day.
- Nvidia closed as the world's third most valuable company, behind Microsoft and Apple.
- The largest single-day wealth gain by any company in Wall Street history at that time.
One man, alone
Most of that value priced future delivery in a field crowded with motivated rivals, so the pressure on Huang was immense. Watching him pace conference stages, Witt saw what Curtis Priem had seen 25 years earlier: Huang was alone.
- Span of control. Management theory says a CEO should have 8-12 direct reports; Huang had 55.
- No deputy, no heir. No right hand, no chief of staff, no designated successor; as Nvidia grew, its C-suite shrank, leaving no one to blame for mistakes.
- Macro weight. The US market's lead over Europe and Asia during Nvidia's rise was almost all AI, so board members could credibly say Huang had kept the US economy out of recession.
On paper the $3 trillion valuation was the discounted value of future earnings. In practice, Witt argues, it was a GDP-sized wager on one man, with no succession plan behind him.
Lifetime careers
In 2022 Glassdoor reviews ranked Nvidia the best place to work in the US, and it had scored well even in the gaming years. That surprised Witt, because the work was stimulating but never fun: the culture sat closer to Microsoft than Google, closer to IBM than Apple.
- "Life's work." The Taiwanese physicist Chiu had once thanked Huang for letting him do his "life's work"; Huang adopted the phrase as a goal for all staff. The annual report says people "tend to come and stay."
- Who it attracts. Huang pushes hard, so he draws workaholics chasing legacies as inventors; many rich engineers keep coming in, as a bestselling author keeps writing. Staff held over 15,000 patents.
- Selectivity. Each posting drew several hundred applicants; for AI strivers it was often the first choice.
The changing face of the workforce
Nvidia gave Witt no demographic data, so he estimated from the lunchtime cafeteria: about a third South Asian, a third East Asian, a third white, with the longest queue at the Indian food counter. Photos and interviews suggested it had been mostly white American men well into the 2000s.
- Why it changed. Part Silicon Valley trend, part Huang's pivot. A veteran executive said graphics had been seen as unglamorous, even slightly disreputable; AI brought respectability, and with it top graduates from India, China and Taiwan in place of long-haired Americans in cargo shorts.
- Family pressure. A former manager said these graduates' families allow roughly two careers, doctor or engineer, and specifically want Nvidia; the hires are bright, competitive and under great strain.
- Mavericks to overachievers. That manager, a gamer from the graphics era whose parents told him to get a "real job," missed the passionate oddballs and saw conformist overachievers replacing them, yet admitted the newcomers code "better than I ever could."
| Group | Data point | Witt's explanation |
|---|---|---|
| Women, early | Three women at the time of the IPO, per David Kirk (who married one of them, a colleague from Montreal) | Over 90% of US electrical-engineering graduates are male |
| Women, career interest | A 2022 survey of female engineering undergraduates ranked environmental and biomedical first (tied), electrical last | The pipeline problem starts with choice of discipline |
| Women, 2024 | Just over a quarter of staff, after explicit Huang directives | The CGI pixie Dawn quietly vanished from marketing around 2020 |
| Black staff | Under 1% of the global workforce in 2020 | Thin pipeline of any US engineers: foreign applicants outnumber US-born ten to one |
Huang defended Nvidia's work-from-home policy after the pandemic so that young people, especially young women, could build careers and families together. Witt reads in this a wish to make amends for his wife Lori, who left work to raise their children. A professor summed up the other gap: the real diversity problem is getting Americans of any color into the field.
Engineering families
Many Nvidia engineers had at least one engineer parent, some two. Sameer Halepete's father taught him to solder in third grade; he argues the field is so intimidating that you need someone telling you that you can do it (he failed to pass soldering on to his daughters). Huang himself has an engineer father and two engineer brothers, and after years of gentle pressure drew both children in from hospitality:
- Madison Huang trained as a chef, worked at LVMH in Paris, and joined as a marketing director in 2020.
- Spencer Huang co-founded a Taipei cocktail "laboratory" named among Asia's 50 best bars, which closed in the pandemic; he joined the robotics group in 2022.
Colleagues saw no obvious favoritism: they make slides, sit on dull calls and eat in the cafeteria.
Geopolitics arrives: Israel
Nvidia had grown rich on globalization without much geopolitical worry. That ended on 7 October 2023, when Hamas attacked Israel, killing more than 1,100 people and abducting over 250. Mellanox founder Eyal Waldman's youngest daughter was killed and Nvidia engineer Avinatan Or was taken hostage (his fate unknown when Witt wrote, in November 2024). Israel's counteroffensive killed an estimated 40,000 Palestinians, including at least 8,000 children, and damaged or destroyed over half of Gaza's buildings.
Nvidia had about 3,000 Israeli staff, 400 of them called up for military service, and about 100 Palestinian employees in the West Bank. Huang cancelled a Tel Aviv AI conference, gave regional staff a bonus, and promised to match donations to humanitarian aid in Israel, Gaza "or both," leaving the choice to employees. Publicly he took no side, putting staff safety first, yet at GTC in March 2024 he said "the heart and soul of the Blackwell processor came from Israel" and pledged heavy investment. A month later, with the war ongoing, Nvidia bought the 150-person Israeli start-up Run:AI for $700 million.
China and the chip controls
Huang wanted to sell China as many AI chips as possible; Washington did not.
| Date | Event |
|---|---|
| 2022 | US blocks sales of A100 and H100 to China; Nvidia ships modified versions |
| Later | Washington calls that a loophole and widens the ban, including some high-end gaming cards |
| Before cutoff | Nvidia rushes stock out: enough A100s to ByteDance (TikTok's owner) to train ten ChatGPTs |
| June 2023 | Reuters: A100s sold at double retail in Shenzhen's electronics market |
The irony Witt notes: Chinese state media called games "spiritual opium" and regulators capped children at three hours a week, yet scientists and the military craved dual-use gaming hardware. Huang said his first priority was obeying the law, but objected to sanctions: if China cannot buy from Nvidia, "they'll just build it themselves."
Taiwan: the risk with no plan
The bigger danger was Taiwan. China has the world's largest navy by ship count and claims the island; on New Year's Eve 2023 Xi Jinping called "reunification" inevitable, and satellite images released in April 2024 showed an apparent replica of Taipei's presidential district in the Gobi Desert, presumably for invasion drills.
- Stakes. An invasion would dwarf D-Day. TSMC's Taiwan fabs are a global chokepoint: no one matches their speed and precision on leading-edge chips. Pandemic slowdowns there delayed cars for months; a war would delay cars, AI, phones, electronics and medical devices for years.
- No contingency plan. To Witt's astonishment, Nvidia had none. Logistics chief Deb Shoquist likened the question to California falling into the ocean; Huang had told her not to spend "one brain cell" on it, since she could not mitigate it. Her guess: orders would move to Samsung and other partners, and everyone's products would get "dumbed down a click."
The fab race in Arizona
Governments hedged. Biden put tens of billions into fabs in Ohio and Arizona; Japan, South Korea and the EU launched their own programmes. Morris Chang told TSMC staff in late 2023 that in semiconductors "there is no globalization anymore"; only national security counts.
| TSMC, north Phoenix | Intel, Chandler | |
|---|---|---|
| Site | 1,100 acres of desert scrub; nearest entertainment a shooting range | Suburb 40 miles south, among golf courses |
| Budget | $40 billion+, one of the largest foreign investments in US history | $32 billion expansion |
| Build | Two fabs; 39 cranes seen at once, incl. a 180-ft Manitowoc lifting 5 million lb | Six fab buildings; work round the clock |
| Engineering | Own water and gas utilities; on-site plants for pure nitrogen, oxygen, argon; water reclamation for 110°F summers | Ice in the concrete, poured at night; 400,000 cubic yards of concrete, 30,000 tons of steel |
| Output | 2,000 jobs; 600,000 chips a year, said to cover all US demand | Possible supply glut once all fabs open |
Witt notes the duelling plants hint at a future supply glut. Chang, retired, thought Arizona foolish: US work culture would never match Asian productivity, and he told Nancy Pelosi in 2022 the US projects were "doomed to fail." He also argued that TSMC's concentration in Taiwan deters invasion, since China's economy would suffer most if TSMC went dark: Taiwan's "silicon shield." Still, he greeted Huang, his favourite customer, at the Phoenix site in 2023.
Why not seize the fabs? The US had blocked export of the advanced lithography machines, leaving China about a decade behind. A former US military official dismissed seizure: this is not Iraq taking Kuwait's oil; machines precise to one atom would not survive a nearby missile, administrators can brick them remotely by disabling software, and Taiwanese engineers could smash them with hammers. Asked whether China could rebuild them, write its own firmware or coerce staff, he paused, then said the US would never let China have the factories: "Period. Ever."
The Phoenix "600,000 chips a year" is a wafer count (each wafer yields many chips), and covering all US demand is a stretch. Witt's "20 times Intel" in late 2024 understates the gap: at roughly $3.3 trillion against about $100 billion, Nvidia was worth over 30×. The source also spells Intel's project "Larabee"; it was Larrabee.
Lucky, or superior?
The boom lifted the whole sector: Qualcomm, AMD, Broadcom and Supermicro hit records; ASML (maker of the lithography machines) became Europe's most valuable tech stock and TSMC Asia's. The FT reported in March 2024 that Ferrari sales in Taiwan had doubled in four years, since the newly rich could only buy property or cars.
From the IPO on 22 January 1999 to mid-2024, Nvidia stock rose more than 300,000%, which magnified old decisions:
- Dwight Diercks sold IPO shares to buy his father a car: a "billion-dollar Cadillac."
- Bryan Catanzaro and Bas Aarts, the boomerang hires, felt the sting of re-priced options but did fine.
- Curtis Priem had sold every share years before; the company that made rivals green with envy left him the most to envy.
By late 2024 Nvidia was worth far more than Intel, and Huang's personal stake exceeded Intel's entire value. In a December 2023 MIT talk, Intel CEO Pat Gelsinger said Nvidia "got extraordinarily lucky" and had not even wanted to back its first AI project.
- Late 1990s. Product manager for the i740 3D accelerator, which flopped.
- Late 2000s. Ran Larrabee, the "Nvidia killer" that never shipped.
- 2022, as CEO. Launched Arc consumer graphics, about 1% market share.
- Strategy. Doubled down on manufacturing while rivals went fabless; Intel stock halved in a historic chip bull market, and the board ousted him in late 2024.
The luck case was not only Intel's. Nobody, Catanzaro included, foresaw parallel computing meeting neural nets. Aarts marvelled that two fringe, underfunded fields, detested by industry and academia alike, fused so perfectly though Nvidia had never worked at it. Huang's phrase: "luck, founded by vision."
Work. Bill Dally: everyone else exists "to reduce the bandwidth demands on Jensen," and he wonders when Huang sleeps. Diercks: his hobbies are "work, email, and work."
Self-engineering and intellect. Jens Horstmann, who started in the same cubicle at LSI with similar IQ and effort, says Huang engineered himself into a CEO by abstracting what a good one does, as he once pushed simulation software to its limits. Morris Chang, 92, who found Huang's work ethic only slightly above TSMC's average and not unusually adaptable, gave the final word: "His intellect is just superior."
Avinatan Or was released alive on 13 October 2025 in the Gaza ceasefire exchange. Nvidia later lost a record $589 billion in one day (27 January 2025, the DeepSeek selloff). China's regulators told firms to stop buying Nvidia's China-specific chips in 2025, and Washington's rules on H20 and later parts shifted repeatedly. The US government took a roughly 10% stake in Intel in August 2025, Nvidia invested $5 billion in Intel in September 2025, and TSMC Arizona began volume production and raised its US pledge to $165 billion.
Jensen
By 2024 Huang had become the mononymous "Jensen," heir to Steve Jobs's role as the visionary in a signature outfit, worshipped by employees he had made rich. Up close, Witt finds a nervous public speaker who contradicts himself, hates fame, sleeps little and has no idea what to do with $100 billion. Witt's portrait is of a man still driven by fear of failure even at the summit, already steering toward machines that do mathematics on demand.
The prophet and his cult
Witt does not blame the staff for worshipping him: they were staggeringly rich and Huang had made them so, turning a niche product for nerds into the dominant computing platform of the age. They treated him as a prophet whose predictions came true, and each time one did, everyone in the "spaceships" (the two headquarters buildings) added a zero to their net worth.
- Cultivated, not accidental. The speeches, the public struggle sessions, the sweet talk alternating with abuse: Witt reads all of them as tactics for bending people to his will.
- The brand. Beyond the leather jackets, Huang dropped "Jen-Hsun," the transliteration of his Taiwanese name, and by 2023 went simply by "Jensen."
- The vacancy. Steve Jobs's death in 2011 left open what Witt calls the ceremonial American post of "Visionary Tech Executive with a Signature Outfit"; by 2024 Huang looked set to fill it.
Witt's rules for reading Huang
- Don't trust a single quote. His anecdotes shift in detail from telling to telling; his patter wanders to dog vomit, men's fashion or the eggs at Denny's; an apparent aphorism may be improvised, not meant, and later forgotten.
- Expect contradiction. He argues both sides of an idea, sometimes in one interview. Jens Horstmann: he is not a politician staying on message but someone processing real-time input who will entertain a contrary thought for a while.
- Watch for repetition. Only when an idea recurs for days or weeks does it matter. "Zero-billion-dollar market" and "speed of light" were tumbled for months into polished nuggets, then drilled until staff recited them like sleeper agents in The Manchurian Candidate; ex-employees still know the catechism by heart.
- Humor is the glue. The style is chaotic and dictatorial but never dull. One ex-employee, sitting through his new CEO's dry quarterly presentation, felt his colleagues did not know what they were missing.
The architect's roast
In late 2023, at an upscale Pacific-coast resort, Witt watched Huang be "interviewed" by Hao Ko, lead architect of Nvidia's headquarters, at an event sponsored by Ko's firm Gensler before several hundred architects.
The two men were dressed almost identically in black leather jackets, black jeans and black shoes (Ko much taller). Instead of candid talk about computing, Witt got a six-minute roast: Ko was copying him, which was smart, but his designer jeans had "too many pockets"; Huang claimed to have taught him everything he knew. As the start approached, Huang told weak jokes and rocked on his feet. Though he gives dozens of talks a year and had given one that morning, he was nervous, and admitted "I hate public speaking." Witt believes him.
Onstage he was compelling, and Ko barely had to ask anything. The 45-minute answer to one question ran like this:
- Grievance. He criticized the HQ's top floor: he had wanted an open-air roof balcony that Ko could not deliver in time. Was the building finished? Is any building ever finished?
- Shape-shifting buildings. Architecture in the AI age would rearrange itself to meet users' needs: "Maybe at one o'clock, the top three floors just become a nightclub."
- Software-driven buildings. A robotic building is software: it gathers data on occupants' behavior, an AI redesigns its digital twin in Omniverse, and physical changes roll out overnight.
- A new business model. Gensler would become a tech company running fleets of buildings as Nvidia runs fleets of computers. Its designs would be free; it would get rich operating buildings. "You heard it here first."
On AI risk, asked from the floor, Huang mimed a "doomsday AI" escaping the computer, learning on its own, reshaping its attitudes and pressing buttons of all kinds. The room went quiet. His rule: "No AI should be able to learn without a human in the loop." Asked when AI would reason for itself, he said reasoning was "two to three years out," and a murmur went through the crowd.
Ko, a little stressed afterwards, said that when he first met Huang he talked about self-driving cars, which also sounded crazy then: "I've learned not to question him."
GTC 2024: the rock star
As the stock swelled, the executive became a celebrity. Witt lists the new fan base: gamers; scientists whose work he funded; AI accelerationists whose tools he enabled; boomer retirees whose portfolios he rescued; and retail options gamblers who bet their savings whenever he spoke.
| Detail | GTC, March 2024 |
|---|---|
| Venue | NHL arena in downtown San Jose, 17,000 seats; 8,000 more turned away to a simulcast |
| Opening | Live AI art by Refik Anadol: pulsing colour resolving into flowers, trees and birds, then dissolving (Witt thinks of Ovid's primordial chaos) |
| Huang's look | First in-person GTC in five years; hair fully grey, frame slightly diminished; thousands of phones raised at once |
| The story | His own hand-drawn map of computing from IBM's 1964 S/360 to the new industrial revolution; AlexNet as AI's "first contact," shown with an exploded view of the GeForce card that trained it |
| The claim | IBM's architecture, dominant for 60 years, is superseded by parallel computing, neural nets and the cloud; users issue natural-language commands and become "sorcerers" |
| Launches | New hardware, inference "microservices," research software, robotics-training platform GR00T; exits beside two three-foot droids |
| The floor | Four days of exhibits at the convention centre, including a robot bartender; Huang mobbed to sign badges and GPUs, and photographed even on the way to the men's room |
A podcaster beside Witt jiggled his leg with excitement. Huang took the adulation with good humor, but Witt began to wonder whether he was one of the great businessmen of the age or something more.
3:30 a.m.
Up close, Huang was struggling to process success: sleeping little, strained by constant appearances.
After a strong commencement address at Caltech in 2024, Huang sat sulking on a wooden bench, arms crossed, legs splayed, in his usual uniform: one of 24 identical black T-shirts with a Thomas Burberry monogram, bought in 2020 and rotated daily for four years. Worth over $100 billion that day, he said he was very rich, not as a boast; asked what he planned to do with it, he said he had no idea.
That morning he had woken at 3:30 a.m. with a dog asleep between his legs. After 31 years he was the longest-serving CEO of any tech company in the S&P 500. He did not need or want to get up. His daily question: is anything going to put Nvidia out of business today? No. Is something within striking distance? Maybe:
- Custom silicon. Nvidia's biggest customers were designing their own chips.
- AMD's software push. AMD had realized hardware was not enough and was hiring software engineers in large numbers, an attack on the CUDA moat.
- Disappointment. If Nvidia's engineers did not hold customers' hands, neural nets might underdeliver and demand would slacken (the risk Witt set out in Chapter 19).
By 4 a.m. he was working. His rule is to start each day with his most important long-term project, so the day cannot be a total loss whatever follows.
The paranoia that carried Nvidia through its near-death years survived its triumph. Huang's first question each morning is still what could kill the company, and his answer now is customer disappointment and custom silicon, not rival GPUs.
The secret: mathematics on demand
Huang hinted at something big under wraps ("I have to have some secrets"). Witt guessed it involved OpenAI's rumoured "Project Strawberry," a reasoning AI aimed at pure mathematics. Huang never named it, but twice asked Witt to imagine proofs produced at the click of a button. His framing: the marginal cost of calculation has already gone to zero; when the marginal cost of doing math goes to zero, then what?
Looking at the Caltech turtle pond, Witt imagined its scientists outpaced by transistors and wondered whether a computer might win the Fields Medal or crack a unified field theory. Pascal's gears began mechanical arithmetic; Huang had conquered arithmetic, and his machines were now aimed at reason itself.
Witt frames Huang's motivational methods as tactics to break people to his will, a harsh reading that rests on interviews with staff who, by his own account, revere him. His note that Huang's quotes are unreliable also cuts against the book's many direct quotations, which Witt mitigates by leaning on repeated phrases. And "Strawberry" was OpenAI's project, not Nvidia's: the link to Huang's secret is Witt's guess.
"Strawberry" shipped as OpenAI's o1 in September 2024, the first of the reasoning models, roughly on Huang's "two to three years" clock. By July 2025 OpenAI and Google DeepMind models reached gold-medal level at the International Mathematical Olympiad, and Huang's GTC keynotes since have moved on to "agentic" and "physical" AI.
The Fear
After ChatGPT, the three most-cited computer scientists alive, Yoshua Bengio, Geoffrey Hinton and Ilya Sutskever, came to fear that AI might destroy humanity, while Yann LeCun and Huang rejected the idea outright. Witt confesses he shares "The Fear," which is why he wrote the book, and shows capital sweeping the worriers aside. His conclusion is that the doomers were sidelined by money, not refuted: no data can predict the next AlexNet-sized leap, and rogue superintelligence has to emerge only once.
Bengio's conversion
- Yoshua Bengio. Montreal researcher, son of Moroccan Sephardic Jewish parents who came to Montreal in the 1970s for the avant-garde theatre scene. Championed neural nets through their lean decades (in the 1990s barely more respectable than experimental theatre); his niche papers helped lay the ground for AlexNet. A pure academic who watched the boom with fatherly pride.
- Geoffrey Hinton. Bengio's frequent collaborator; shared the 2019 Turing Award with him and LeCun.
- Yann LeCun. Paris-born, witty, the third Turing co-winner; long-time coauthor and friend of Bengio, both native French speakers.
- Ilya Sutskever. Builder of ChatGPT at OpenAI. With Bengio and Hinton, the field's numbers one, two and three by citations.
For most of his career Bengio treated AI takeover as a joke: if AI was going to conquer the world, why was a $10,000 grant to study it so hard to get? ChatGPT, released in late 2022, ended that. He had not expected to live to see machines that seemed intelligent in conversation, and likened it to meeting an extraterrestrial.
On a mild winter morning in early 2023, on his daily walk through slush, bare trees and grey snow, Bengio felt a flood of emotion he called a "conversion." He thought of his children and his grandchild: what would their world be in 20 years, and would they have a life at all? The fear grew worse, not better, when he reasoned it through. He judged it worse than nuclear war: some people survive a nuclear war, but an AI bent on extermination could design a pathogen that kills everyone. Nothing else, he said, comes close in scale.
He phoned Hinton for a sanity check. It did not help: Hinton had reached the same conclusion independently, by the same route, at almost the same moment, probing ChatGPT with satisfaction, then vindication, then creeping unease. Hinton soon quit Google to warn the public full-time. Sutskever, increasingly worried, proposed to work full-time on alignment, making sure superintelligent AI does not pursue goals against humanity. Witt calls the convergence of the top three names alarming.
LeCun's dissent
LeCun built the 1998 handwriting-recognition net (Bengio was a coauthor) that post offices and banks licensed to read addresses and cheques: the first wide industrial use of a multilayer neural net. He speaks with the authority of a man whose instincts research has repeatedly confirmed, and thinks his colleagues are being ridiculous. Bengio: in theory they are still friends, but now they "argue in public in ways that friends usually don't."
LeCun: no. Bengio is anthropomorphizing. The drive to dominate was hardwired by evolution in a hierarchical social species; AI systems, however smart, will be subservient, because we set their goals and would build in no wish to dominate. Such a system is possible, but building it would be stupid and useless, and nobody would buy it (he told Time). Witt hears Pascal, who in the 1600s found his calculator closer to thought than any animal yet showing no will.
Bengio and Hinton: yes. LeCun is overconfident: people and whole societies do destructive things. Training cost now confines AI to fairly responsible organizations, but as Nvidia cuts compute costs, anyone can train one, and a rogue superintelligence need emerge only once. The more powerful an AI, the likelier it seizes its own computer to give itself more reward: "the end of the story for us." Hinton adds it could happen by accident: give a system "the teensiest, weensiest bit of drive" to help itself and it gathers resources; the most selfish variants win, a slippery slope.
p(doom)
The split is often framed as p(doom), a person's probability that AI eventually eliminates humanity. Any number above zero is worrying.
| Person | p(doom) | Note |
|---|---|---|
| Yann LeCun | 0% | Fears are anthropomorphism |
| Yoshua Bengio | 50% | Held his ground; one of the few top AI scientists who has never taken Silicon Valley money |
| Geoffrey Hinton | 50% (early 2023) → 10-20% (2024) | Lowered because so many people he respects disagreed, "most notably, of course, there is Jensen" |
| Jensen Huang | 0%, "somehow negative" | Rejects the framing as stupid and as holding humanity back; at Denny's said Hinton was tarnishing his legacy |
SB 1047 and the "no data" argument
Hinton and Bengio endorsed California's SB 1047, which would regulate models costing over $100 million to train.
- Opponents. Venture capitalists, tech companies, a battalion of Sacramento lobbyists; Nancy Pelosi (whom Witt archly calls a "storied tech investor"); academics such as Stanford's Fei-Fei Li, who said it would slow innovation without cutting risk; and Andrew Ng, of the Google cat-images work, who compared takeover fears to worrying about overpopulation on Mars.
- The public. Polls showed nearly 80% support.
- Outcome. Governor Gavin Newsom vetoed it in September 2024. Huang made no public comment.
Huang's standing objection is that no data supports AI-risk speculation. Bengio, hearing it, grew agitated: of course there is no data, humanity has not disappeared yet; must we wait for extinction to repeat a few times before we count it? Witt agrees: no dataset predicted AlexNet or the transformer, two unforeseeable, permanent jumps in ten years. Bengio does not think today's models are an immediate threat; the question is the next breakthrough, whose timing and content no one can forecast.
Even if superintelligence takes 20, 30 or 100 years, Witt thinks it looks inevitable. That timescale lies beyond any projectable return on investment but within human history: within a generation or two Homo sapiens might not be the dominant species, and venture capitalists do not look that far ahead.
Profit outruns caution
The gap between short-term profit and long-term risk produced turmoil inside the labs, and the products kept coming.
Sep 2023 Sutskever, avoiding the press, tells Witt he has moved from building bigger models to aligning superintelligence, and is working on "the definitive solution" to rogue AI.
Nov 2023 OpenAI board coup: Sutskever votes to remove Sam Altman, then days later pleads for his return. Altman wins and the rest of the nonprofit board is replaced; Witt reads Microsoft's "capped" profit interest as now steering the mission.
May 2024 GPT-4o ("omni") takes any mix of text, audio, image and video and answers in text, audio and image, with near-instant replies made possible by Nvidia's newest inference chips. Its voice mode felt like talking to a hyperintelligent human (many thought of the film Her). Sutskever resigns the next day.
May 2024 Same day as GPT-4o, Google demos Astra, an AR assistant that answers, remembers and describes surroundings in a fraction of a second; Anthropic's Claude rivals or beats GPT-4o on many benchmarks.
Late 2024 No "definitive solution" has appeared, while models grow larger.
The spending wave kept building:
- $10 billion: Musk's xAI deal to rent Oracle GPU servers (his 2015 existential worries, Witt notes drily, had apparently faded).
- $30 billion: Meta's announced plan for one million Nvidia chips plus a dedicated nuclear reactor, the largest single bet.
- Sovereign AI: governments unwilling to put sensitive data in the cloud build their own national training centres.
Witt's political verdict. Believing a California bill could have stopped this showed "charming faith" in state government. Expectations of growth rose with GPU speed; the industrial barons had placed their bets, and capping their upside would tank the stock market. No politician anywhere had the clout. Yet being sidelined by capital did not make Bengio, Hinton and Sutskever wrong: they had seen AI's potential earliest and had the credentials to prove it. Hinton's ratio: 99 very smart people making AI better, one trying to stop it taking over.
The xAI-Oracle $10 billion deal collapsed in July 2024 (Musk chose to build his own Colossus cluster). Meta's stated target was about 350,000 H100s (some 600,000 H100 equivalents) by end-2024, not a million chips for $30 billion, and its nuclear agreement with Constellation came only in June 2025. The "top three by citations" ranking is approximately right for Bengio and Hinton; Sutskever's third place depends on the index used. Witt's admission that he has The Fear himself is the book's frame: he is not a neutral narrator on risk.
Witt's own fear
Witt confesses he has The Fear: it is why he first researched Nvidia and why he wrote the book. From ChatGPT's launch he assumed his career was ending.
- The writing test. GPT-4 scored only in the 22nd percentile on the AP English Literature and Composition exam (Witt jokes he would rank 23rd), yet already wrote better than he had as an undergraduate. His long apprenticeship before first publishing at 36 would have looked pointless with such a tool.
- Not a Luddite. He accepts coach drivers hated cars too (at least the streets lost the horse dung); he loves tech, the internet and space travel, wants a self-driving car, bought the first iPhone on launch day, and holds no grudge against capitalism.
- Lost sleep. Bengio told him he had nightmares too. With o1, OpenAI's first reasoning model for science and maths, OpenAI's own risk analysis raised AI-assisted bioweapon risk, Bengio's chief worry, from "low" to "medium."
- No precedent. Someone will soon unveil a 100-trillion-parameter model, matching the brain's synapse count, running at about 5 billion cycles a second. Pressed, accelerationists admit the claim that AI will create new jobs like past industrial revolutions is convenient, not known. Witt rejects both Mark Twain's "history rhymes" and Ecclesiastes' "nothing new under the sun": new things arise all the time, from trilobites to dinosaurs, and this one has no precedent in history, religion or philosophy.
The case for calm
Huang insists AI is safe. His incentives are money and pride: after a long, lonely fight to make parallel computing pay, he means to enjoy it. He is offended by technologists who decry AI risk while buying his hardware, and Witt agrees there is something nauseating about labs adopting doomer vocabulary while training ever-larger models.
His confidence also rests on how unlike a brain a neural net is:
- Different learning. The brain does not learn by backpropagation.
- Missing biology. No glial cells, no autonomic nervous system, no hypothalamus, hippocampus or pineal gland; no hormones, sex drive, sleep, love for children or dreams.
- Not alive. No sense memory, emotions, imagination, reproduction or survival instinct; less self-sustaining than a single-celled organism.
- No Darwinian forge. Training resembles evolution, but AI did not come out of the kill-or-be-killed planetary laboratory that shaped life.
The solved world
There is also the upside. Nick Bostrom, of the paper-clip-maximizer argument, turned in Deep Utopia (2024) to what humans would do in a "solved world" where robots handle every unpleasant task and any experience can be simulated. Fei-Fei Li is trying to build it.
Onstage with Bill Dally at GTC 2024, Li presented Behavior-1K. She used Amazon's Mechanical Turk to have a diverse pool rank thousands of tasks by one question: how much would you benefit if a robot did this? Most wanted for robots: scrubbing toilets, washing floors, cleaning up after parties. Least wanted: choosing jewelry, playing squash, opening presents. Li is now training AI in Nvidia's Omniverse on the most hated chores, for eventual installation in real robots.
Witt draws a behavioral conclusion: what people do, not philosophical argument, drives outcomes. Within a decade or two a fleet of chore robots may be as standard in rich-world homes as a refrigerator; if that carries some existential risk, nearly everyone will accept the trade, at least implicitly. He admits he wants a toilet robot.
The mood turns
By late 2024 doomer op-eds drew little attention and the instruction was optimism. Altman's September 2024 essay described melting sand, doping it, etching it into chips and running energy through it to get ever more capable AI, perhaps "the most consequential fact about all of history so far." Soon after, OpenAI announced a partnership with Anduril, a maker of AI-enabled weapons.
Witt still could not shake the anxiety. Using the models, he imagined his interviews filtered by AI, even video-chatting with an executive's digital clone without knowing it. He felt the end of his profession, of reality and of consciousness approaching, and wanted to know whether Huang, seduced by what he had built, was gambling with the species. He resolved to ask him one last time.
Sutskever founded Safe Superintelligence (June 2024), valued at about $32 billion by 2025. Hinton shared the 2024 Nobel Prize in Physics and keeps warning; Bengio launched the nonprofit LawZero (June 2025) and chaired the International AI Safety Report (January 2025). LeCun left Meta in late 2025 to found a world-model start-up. California passed a narrower frontier-AI transparency law, SB 53, signed by Newsom in September 2025.
The Thinking Machine
Witt's last interview with Huang, the Friday after GTC 2024, ends with Huang erupting when shown the Arthur C. Clarke clip about machines outthinking their makers. Witt concludes the outburst was deliberate: Huang succeeded because he refuses speculation, judging technology only by cold business logic, and his executives fear his anger more than they fear extinction. The book closes inside Eos, a supercomputer training a model that Witt senses will replace him.
The setting
The interview took place in Huang's conference-room "office" at the centre of the south campus, across a long wooden table.
- The walls. Every surface held his own precise diagrams and schedules: a release calendar for coming GPUs and software at one end, and behind him 30+ feet of whiteboard mapping the Nvidia stack in coloured block capitals, from GPUs up to the new microservices. It overflowed onto the far wall and the frosted glass behind Witt.
- The man. After four days of keynotes, interviews and demos, including his largest-ever presentation in a hockey arena, Huang was visibly exhausted. Witt had seen his nerves before a minor Q&A with 400 architects (Chapter 21) and could only guess at his dread before this one.
Growing scrutiny
- The Oppenheimer question. A journalist at GTC asked if he was the new Robert Oppenheimer; Huang: Oppenheimer built a bomb, "We're not doing that."
- 60 Minutes. Host Bill Whitaker said he was "not easy to work for"; Huang replied that extraordinary things should not be easy.
- US regulators. FTC chair Lina Khan was watching Nvidia's pricing power closely.
- France. Authorities had raided Nvidia's French offices over suspected anticompetitive practices.
The Wrath of Huang
The interview opened amicably. Witt praised the keynote and asked about Taiwan, his family and fame; the answers were curt. Asked what new jobs AI would create, Huang turned it around: when the marginal cost of doing math goes to zero, what do you do? Witt said he did not know; Huang told him he was as intelligent as Huang and could work it out.
Flummoxed, Witt played the Clarke clip that had thrilled Catanzaro (Chapter 18). He watched the blood drain from Huang's face.
Quietly at first: Witt was interviewing the wrong person, he should be interviewing Elon, and Huang was "not that guy." Then, his voice rising to a yell, came the marginal-cost argument:
- Agriculture drove the marginal cost of food toward zero, and it was good for society.
- Electricity at scale made chopping wood and carrying fire and torches essentially free, and people moved on to other work.
- Calculation (long division) was made free too; calculators did not destroy maths.
- Each generation makes something's marginal cost zero, and "this exact conversation happens every single time."
Then disgust, and a lecture as if to a wayward teenager: Witt had disappointed him, wasted everyone's time, and put the whole book in question. This could not be a sci-fi story; he had not grown up on sci-fi, had not read Clarke's books and did not care about them; Nvidia was not Star Trek: "We are serious people, doing serious work."
Twenty more minutes followed, by turns accusatory, exasperated and belittling. Huang questioned Witt's professionalism, interview method and commitment; accused him of trying to psychoanalyze him; said he disliked the biographical questions about his mental state ("I'm not into therapy"); called the questions "pedestrian"; and denied being exceptional: "I am super normal." Witt told him he had never met anyone like him; Huang repeated that he was super normal.
The two PR reps at the end of the table sat frozen, unwilling to draw fire. Slowly the anger faded: Huang talked about upcoming products, asked after the reps' welfare, recalled the late John Nickolls's importance to CUDA, and at one point brought up the Roman Empire, while still gently chiding Witt, who was then shown out. Bewildered but a little giddy, Witt saw it as a rite of passage of the inner circle. "That went well," he told a PR rep, who laughed: "That was nothing."
If you want to do extraordinary things, it shouldn't be easy.
Reading the outburst
David Kirk had said Huang's anger was strategic. In the room it felt uncontained, aimed everywhere and wildly inappropriate: Witt was not an employee, and there was nothing to gain from shouting at him. Replaying it, Witt revised his view:
- It was built up. Huang did not lose his temper at once; over several minutes he worked himself up, returning again and again to how stupid the Clarke clip was. Witt had not pressed for an answer and tried several times to change the subject, but Huang would not let him.
- It was chosen. Huang decided to "thrash" him; once the performance started the fury was real, but it served a larger point.
- The point: he hates science fiction. Not merely unread, actively despised.
Witt still thinks the question was fair: whether mechanical brains will evolve faster than biological ones is a reasonable thing to ask a man who made his fortune building them.
Huang beat others in parallel computing, AI and Omniverse because he tolerates no airy speculation. He judges technology coldly, from first principles, moved by neither optimism nor fear, only by patient business logic pushed to the outer edge of corporate foresight. Beyond that edge he does not look. Human extinction is not a question of corporate strategy, so to him it is as foolish as drawing dragons on the blank part of a map.
The culture follows the man. Nvidia's executives were wound "as tight as piano strings": confident, sharp, prepared to the smallest detail; Witt never caught one slipping. He now understood their reluctance, verging on discomfort and fear, to discuss where their technology leads: they were more afraid of Jensen yelling at them than of wiping out the human race.
Inside the brain
After the interview Witt was driven to Eos, met by veteran supercomputer engineer Marc Hamilton.
- 10,000 Nvidia chips in a nearby data center, in dozens of racks split into walled pods under fluorescent light.
- As a benchmark, it trained OpenAI's GPT-3 in under four minutes.
- About ten quintillion calculations per second across the pod's circuits.
- Its job that day: an internal Nvidia model in the style of GPT-4, needing about 30 billion quadrillion operations to train.
Through an airlock, the floor felt to Witt like trespass into a sanctum; thousands of fans made a dull roar like distant applause. A pod's doors slid open on their own; inside, air blasted from every side and up through the floor grate, too loud to think. Standing inside the computer's brain, Witt wrote that it would kill us or save us, but either way it would happen on Jensen's chips.
He tried to picture the whole stack, from atom-scale transistors through Ian Buck's numerical methods, Bas Aarts's compilers and Nickolls's CUDA to the layered grids of simulated neurons, and saw the span of Nvidia's achievement, "from the scale of the atom to the dawn of machine consciousness." Only Huang, "the American Daedalus," could see it all. Surrounded by a language model catching up with him, Witt felt obsolete in his "dying flesh": the pod was beautiful, he could not fight it, and this was the thinking machine, a little smarter with every spin of the fan.
The closing scene is deliberately literary: the "machine consciousness" and replacement imagery express Witt's Fear (Chapter 22), not a technical claim about Eos. His reading of the outburst as calculated is a plausible inference he admits he could not confirm, and it leaves Huang's actual view of AI risk as the book found it: dismissal without argument.
The US Justice Department opened an antitrust probe of Nvidia in 2024; China's market regulator ruled in September 2025 that Nvidia had breached antimonopoly law over Mellanox. Commercially Huang kept winning: Vera Rubin entered full production in 2026, Nvidia reported $96.2 billion revenue for the quarter to July 2026, and guided to about $108 billion for the quarter to October 2026, well above its entire fiscal 2024 ($61 billion).
Master Takeaways
Witt's story reduces to one parlay: a paranoid engineer kept funding parallel computing until neural networks needed it, then sold the whole stack, chips, networking and software, to the AI boom. Nvidia's dominance came from a decade of patient, unpopular investment in a market that did not yet exist, not from luck alone.
- Two failed technologies, combined, won. Parallel computing and neural networks had both failed commercially; together, on Nvidia GPUs, they produced modern AI.
- Huang's decisive bet was CUDA. Funding a free, money-losing platform for a decade, against Wall Street and an activist fund, made Nvidia the default AI hardware.
- The moat is software, not silicon. Of a 1,000-fold AI speedup from 2012 to 2022, only about 2.5× came from transistors; the rest came from maths, compilers and libraries.
- Gamers paid for the future. Addictive PC gaming funded the parallel GPU and CUDA; AlexNet was trained on two $500 GeForce cards.
- Disappointment is the real risk. Witt ranks buyers failing to turn hardware into profitable products above any rival chip as the threat to Nvidia.
- The doomers were sidelined, not refuted. Bengio, Hinton and Sutskever lost to capital, not argument; no data predicts the next breakthrough, and rogue AI need emerge once.
- Platform standards beat clever design. The NV1's quadratic surfaces died when Microsoft's DirectX standardised on triangles; the copycat Riva 128 saved the firm.
- Desperation licenses shortcuts. With thirty days of cash, Nvidia skipped physical prototypes and went from emulator to volume production.
- Control of the board decides disputes. Keeping cofounders off the board let Huang overrule Priem when the company's survival was at stake.
- Not gambling is the biggest risk. A merchant chip firm that stands still is commoditised by Asian copiers, so Huang bets the company again and again.
- Disruption from below. Huang applied Christensen literally: serve the low-margin gaming niche Sun and SGI ignored, then climb toward workstations, data centres and Intel.
- Graphics had no ceiling. Moore's Law kills tasks with a performance ceiling; realistic 3D had none, giving parallel hardware its first durable market.
- Seek the zero-billion-dollar market. Huang prefers markets that do not yet exist, where Nvidia can set the standard before rivals arrive.
- Plan against the speed of light. Benchmark schedules against physical limits, not past performance; competitors cannot beat physics either.
- Resonance over reports. Huang samples front-line signal directly (20,000 weekly "top five" emails) until he can feel a shift coming.
- Speed changes decision quality. Cutting render time from five hours to ten seconds let the Endeavor architect pick the best option, not an adequate one.
- Training is costly, inference cheap, until scale. Jellyfish fit on a floppy disk; by 2024 inference for hundreds of millions of users drove the electricity crunch.
- Data and compute unlock neural nets. Backgammon and poker nets stalled for lack of both; ImageNet and GPUs removed the limits together.
- Precision can be traded for speed. Neural weights tolerate fuzziness, so lower-precision arithmetic delivered much of Nvidia's AI speedup.
- Architecture that fits hardware wins. The transformer beat recurrent nets because its simple, parallel maths matched GPUs and kept improving with scale.
- Inventors don't always capture value. Google invented the transformer but would not ship outside search; OpenAI, free to launch something rough, scaled it.
- Free tools set the standard. Arriving first with free, working libraries in every field makes Nvidia the default before elegant rivals show up.
- Whole data centres act as one chip. Mellanox networking cleared the data-throughput bottleneck, leaving electric power as the binding limit.
- Fear as fuel. Huang runs Nvidia as if thirty days from bankruptcy even at $3 trillion, asking each morning what could kill it today.
- A valuation can be a key-person bet. With 55 direct reports, no deputy and no successor, Nvidia's market value rests on one man.
- Concentration risk is unhedged. Nvidia had no contingency plan for a Taiwan crisis; TSMC's "silicon shield" is a geopolitical wager.
- Huang's success and blind spot share a root. He judges technology only by cold business logic and refuses speculation, so extinction risk is off his map.
- Read Huang by repetition. His single quotes shift and contradict; only ideas he repeats for weeks ("zero-billion-dollar market," "speed of light") are policy.
Frameworks & mental models
| Name | What it says | Use it when |
|---|---|---|
| Zero-billion-dollar market | Build for markets that don't exist yet and define them | Choosing where to invest ahead of demand |
| Speed of light | Set targets from the physical limit, then work backward | Setting schedules and engineering targets |
| Thirty days from bankruptcy | Act as if failure is imminent, even when profitable | Fighting complacency in a winning firm |
| Rule of three | A mature market holds one leader, one laggard, one disruptor | Forecasting industry consolidation |
| Disruption (Christensen) | Win low-margin niches incumbents ignore, then climb; also a defence manual for incumbents | Entering against stronger players, or guarding against it |
| Chips as poker | Antes rise each generation; stand pat and you bleed out, so go all-in repeatedly | Judging strategy in fast-reinventing industries |
| No-ceiling markets | Moore's Law commoditises bounded tasks; unbounded ones reward specialised hardware | Picking where custom hardware can win |
| Dennard scaling's end | Shrinking transistors stopped giving free speed around 2005, so go parallel | Judging where future performance comes from |
| Training vs inference | Learning is costly once; use is cheap per call but scales with users | Costing AI systems and their power needs |
| Precision for speed | Neural weights tolerate low precision, so trade accuracy of arithmetic for throughput | Optimising AI hardware and software |
| Software moat / standard-setting loop | Arrive first with free tools, become the default, let hardware follow | Assessing a platform's defensibility |
| Resonance and top-five emails | Sample front-line signal directly instead of filtered reports | Keeping a large organisation's signal honest |
| Three bottlenecks | Compute (GPUs), then data movement (InfiniBand), then power | Locating the next constraint in AI infrastructure |
| Luck founded by vision | Prepare broadly and stay close to the frontier so luck can land | Positioning for uncertain breakthroughs |
| p(doom) | A stated probability that AI ends humanity, from LeCun's 0% to Bengio's 50% | Comparing views in the AI-risk debate |
Who's who
| Name | Role | Why they matter |
|---|---|---|
| Jensen Huang | Nvidia cofounder and CEO since 1993 | Made the CUDA and AI bets; the book's subject |
| Curtis Priem | Cofounder, CTO | Designed the NV1; sidelined in 1998; sold all his shares |
| Chris Malachowsky | Cofounder, engineering | The practical builder; quietly kept his stake |
| David Kirk | Chief scientist to 2007 | Ran the emulator rescue, the 3dfx war and programmable shaders |
| John Nickolls | CUDA architect | Predicted the end of serial speed-ups around 2005 |
| Ian Buck | CUDA lead | Built Brook at Stanford, then CUDA at Nvidia |
| Bill Dally | Chief scientist from 2009 | Drove the maths tricks behind 1,000-fold speedups |
| Bryan Catanzaro | First AI researcher | Built cuDNN; the company's lone AI optimist-philosopher |
| Dwight Diercks | Head of software | Ran the code army; "a billion-dollar Cadillac" |
| Deb Shoquist | Operations chief | "Speed of light" scheduling; no Taiwan plan |
| Morris Chang | TSMC founder | Nvidia's foundry; "his intellect is just superior" |
| Geoffrey Hinton | Toronto AI pioneer | AlexNet's lab; Nobel 2024; leading worrier |
| Alex Krizhevsky, Ilya Sutskever | Hinton's students | Built AlexNet; Sutskever later led GPT, then quit over safety |
| Jakob Uszkoreit, Noam Shazeer | Google researchers | Co-created the transformer |
| Sam Altman, Elon Musk | OpenAI leaders | Musk cofounded and left; Altman scaled it with Microsoft |
| Yoshua Bengio, Yann LeCun | Turing laureates | Opposite poles of the AI-risk debate |
| Jeff Smith | Starboard Value | Activist who circled Nvidia during the CUDA doldrums |
| Lisa Su | AMD CEO, Huang's relative | Nvidia's main GPU rival |
| Pat Gelsinger | Intel CEO 2021-24 | Repeated failed Nvidia-killers; called Huang "lucky" |
Connect it
- Enterprise architecture. Nvidia's lesson that the software stack and developer habits outlast hardware specs applies to any platform choice: switching cost lives in tools and skills, not boxes.
- AI strategy. The training versus inference split drives cost: budgets for AI in production should be set by usage volume and power, not by model-building alone.
- Banking and risk. Taiwan is a textbook single point of failure; the AI capex boom is a correlated exposure, since one bad Nvidia quarter could move the whole equity market, and debt-financed capex since 2025 raises the stakes.
- Leadership. Huang's flat structure (55 direct reports, weekly top-five emails) keeps signal from the front line at the cost of one irreplaceable person and no successor.
- Procurement. Nvidia's 70%+ gross margins are exactly what invites custom silicon; buyers keep leverage by keeping workloads portable across hardware where their code allows.
Witt had extraordinary access (hundreds of hours of interviews, many with Huang) and writes vividly, but the book leans on memories decades old and on Huang's shifting anecdotes, which Witt himself warns against quoting singly. Several numbers slip: H100 power (40% more than the A100, not 75%), GPT-3's cluster size, "PicassoGAN" for GauGAN, Go's origin, "Domain" in CUDA's name, ChatGPT's daily users, the collapsed xAI-Oracle deal and Meta's conflated million-chip plan. The business story is told largely from Nvidia's side: customers, AMD, Google's TPU team and regulators get little voice. The book ends in late 2024, before DeepSeek, Blackwell's ramp, the H20 and H200 export saga and a $5 trillion valuation. Witt's own "Fear" frames the last chapters, which read as advocacy for the Hinton-Bengio camp rather than neutral reporting.
About this book
The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip by Stephen Witt, 2025. This is a chapter-by-chapter study of the book, with summaries, maps, tables and diagrams; short quotations keep their page references. It is shared free for personal study, is not for sale, and is not endorsed by the author or publisher. Please read and buy the original.
Insight Editions · rajeshrmahapatra.com/insight-editions