The scene
A note on the rail that one station never read
At half past eight on a Friday, the kitchen of Annapoorna, a busy restaurant in Jayanagar, is forty tickets deep. Along the back wall runs the rail: a long whiteboard where every order is clipped and where anyone can scribble a note. Lakshmi, the head chef, writes the orders. Imran works the tandoor, Kiran the dosa griddle, Savitha the curries, and Babu the sweets. Each of them glances at the rail when they need to, takes what concerns them, and adds a note when they have something to say. "Naan for T4 burnt, redoing." "Out of paneer." Nothing is ever wiped until closing.
At 8.10 Lakshmi had written, in red, beside table nine's ticket: "T9: no cashew. Allergy." Imran saw it and skipped the cashew paste in the marinade. Savitha saw it and made the korma without nuts. Kiran had nothing with cashews on his station, so he read it and moved on. Babu, at the far end, was plating payasam for six tables at once. He never looked up. At 8.40 Joseph, who runs the pass where finished plates leave the kitchen, caught the bowl with cashews floating on top, a moment before it went out.
Nothing failed in the rail, since the note was there, in red, available to everyone. Three of four stations read it and acted on it. The fourth did not, because availability is not use. And when Joseph, a minute later, shouted "Table nine, cashew allergy, everyone stop and check!", every station did stop. They checked, and two more dishes were remade. The shout did something the rail could not do. It chose one message out of forty, made it loud enough to drown the others for a moment, and reached everyone at once.
That small scene holds the argument of this paper. The residual stream in a transformer is a rail, not a shout. Every component of the model can read what every earlier component wrote, in the same way every station can see the board. People sometimes take that sharing as evidence of something like a global workspace, the stage on which, in one influential theory, conscious contents are broadcast to the whole brain. I think the sharing earns that reading only when four further things hold. Many readers must actually use the content. It must win against rivals for limited room. It must arrive all at once. And it must be reused for jobs it was not written for. I will measure the first in a real model, and argue that the others are mostly absent.
The kitchen is invented, and so are its people. I will come back to it in every part, changing one thing at a time: the size of the board, how loud a note is, whether Joseph shouts. It is a source of intuition and never of evidence. The evidence comes from published work and from a small audit of GPT-2 that I built and ran for this paper.
One question frames everything that follows. When does residual-stream sharing support a global-access interpretation, rather than ordinary feature accumulation? My answer, argued over six parts, is that it does so less often than the architecture diagrams suggest, and in ways we can test.
Part 1
What the residual stream actually is
The residual stream began as a trick for training deep networks, and it is worth seeing that origin before reading anything grander into it.
In 2015, image-recognition networks had a strange problem. Making them deeper should have made them at least as good, since a deeper network could always learn to pass its input through unchanged. In practice, deeper plain networks trained worse. Srivastava, Greff and Schmidhuber responded with highway networks, which add gated paths that let information skip layers. A few months later, Kaiming He and colleagues at Microsoft Research used the simplest possible version: an ungated shortcut that adds each block's input straight to its output. Each block then only has to learn a change, a residual, on top of what it receives. Their networks reached 152 layers, eight times deeper than the previous favourites, and won first place in the ILSVRC 2015 image classification task.
The transformer kept the shortcut around every block. In Vaswani and colleagues' design (2017), each attention layer and each MLP layer, the small two-layer network that follows attention, adds its output to what came in. Follow one token's vector from the bottom of the model to the top and you see a running sum: the token's embedding, plus the first attention layer's contribution, plus the first MLP's contribution, and so on to the last layer. That running sum is what is now called the residual stream.
The channel reading
The name, and the idea of the stream as a channel, comes from Elhage and colleagues at Anthropic (2021). Their description is plain. Each layer reads its input from the stream by a linear projection and writes its result back by adding a linear projection. The stream does no processing itself. Because every layer communicates through it, they suggested thinking of it as a communication channel, and of a layer's input and output weights as choosing which parts of the channel it listens to and speaks on.
Three properties follow, and each one matters later.
First, reachability is total. Anything written at layer 2 is still in the sum at layer 11, unless something later writes the opposite. Every later block can in principle read it. This is the property that tempts people towards a global-access reading.
Second, the stream is shared and finite. GPT-2 small, the model I audit in Part 4, has a stream of 768 numbers per token. It has 12 layers, each with 12 attention heads and one MLP: 156 components, all writing into the same 768 numbers. They cannot all have private lanes. Many features must share directions, which Elhage and colleagues (2022) called superposition. Sharing directions means interference.
Third, the stream has no favoured coordinates. Rotate the whole stream and rotate every weight matrix that touches it, and the model computes exactly the same thing. Information lives in directions, not in particular slots. A feature is a direction: a vector whose size, along that line, carries something such as "the subject is plural".
A minimal worked case
Here is the smallest version that shows how reading works. Use a stream of two numbers. Lakshmi's allergy note is written as the vector a = (2, 0). Kiran's note that a dosa should be extra crispy is written as c = (0, 1). The stream is their sum, s = (2, 1). Nothing else is stored anywhere.
Now Babu reads. A reader is a direction too: say r = (0.6, 0.8), a unit-length vector tilted towards the second axis. His reading is the dot product r · s = 0.6 × 2 + 0.8 × 1 = 2.0. Split it by source and the allergy note contributes 1.2 while the crispy note contributes 0.8. Babu's reading mixes the two. If he had read along (1, 0), he would have heard the allergy at full strength and the crispy note not at all. If he had read along (0, 1), he would have been deaf to the allergy.
Two lessons sit in that arithmetic. A note reaches a reader only through the reader's own direction, so the same note can be loud for one component and silent for another. And when notes share directions, a reader gets a blend, wanted or not. Both lessons matter for the question of global access. A content written into the stream is not "broadcast" in any active sense. It simply sits there, and each component's weights decide how much of it gets through.
Static view: the stream holds an allergy write (2, 0) and a crispy write (0, 1). A reader at 0 degrees reads allergy 2.0 and crispy 0.0. At 53 degrees it reads allergy 1.2 and crispy 0.8. At 90 degrees it reads only the crispy note.
Part 2
Two ways to read one architecture
Given the same diagram, two very different stories are told about it. I will call them the accumulation reading and the global-access reading.
The accumulation reading
On the accumulation reading, the stream is a scratch pad on which a prediction is gradually refined. Each layer nudges the running guess. Nothing is staged; nothing is selected for special treatment. Features pile up and are used by whoever needs them.
The best evidence for this reading started as a blog post. On 31 August 2020, a pseudonymous writer called nostalgebraist published "interpreting GPT: the logit lens". The idea was simple enough to fit in a sentence. GPT-2 turns its final stream vector into word probabilities by multiplying by one fixed matrix. What happens if you apply the same matrix to the stream in the middle of the network? The answer was that the intermediate states decode into sensible guesses about the next word, which sharpen layer by layer. The post noted that, after the very first layer, the stream already looks more like the final prediction than like the input word. The model seems to think in prediction space and refine.
Later work made the idea more careful. Belrose and colleagues (2023) trained a small translator for each layer, the tuned lens, and found the refinement picture holds more reliably across models. Veit, Wilber and Belongie (2016) had already shown, for image networks, that residual networks behave like ensembles of many shallow paths, and that deleting a single layer barely hurts. Language models are similar. Lad and colleagues (2024) deleted and swapped adjacent layers and found the models surprisingly tolerant. Gromov and colleagues (2024) removed large fractions of the deepest layers with modest loss after brief repair training. A stage would not tolerate that. A shared scratch pad, where many writers each add a small correction, would.
The global-access reading
On the global-access reading, the stream is something like the workspace in global workspace theory. That theory, introduced by Bernard Baars in 1988 and given a neural form by Dehaene, Kerszberg and Changeux (1998), says the mind contains many specialist processors working in parallel and out of awareness. A content becomes conscious when it enters a limited workspace and is broadcast to all of them at once.
The appeal of the mapping is obvious. Attention heads and MLPs look like specialist processors, and the stream connects them all.
Butlin, Long and colleagues (2023), in a widely read report on consciousness in AI, listed workspace properties among their indicators and asked whether transformers have them. They took the residual-stream argument seriously and then rejected it. The stream is as wide as the model's input, so it is doubtful that it forms a bottleneck. The deeper problem is that information never flows back from the stream to the modules that wrote it; it only flows on to later layers. Their verdict was that there is "only a relatively weak case" that transformer language models have any of the workspace properties. Goldstein and Kirk-Giannini (2024) argued that language agents, models wrapped in scaffolding with memory and tools, come much closer.
So the diagram does not decide between the readings. Both describe the same wiring. The question is about use: which contents are read, by how many components, and whether contents compete. That is a question we can measure, once we are clear about what global access would require.
Part 3
What global access would require
To test the global-access reading we need its content, not its mood. What does a workspace theory actually claim, in terms that could fail?
The neural version from Dehaene, Kerszberg and Changeux (1998) is unusually specific. It proposes many specialised processors and a separate population of workspace neurons with long-range connections. At any moment, the authors write, only a subset of the processors' outputs actually gets into the workspace. The workspace neurons can amplify some inputs and suppress others. Activity in the workspace appears suddenly and coherently, and the network is wired so that only one workspace representation is active at a time. The rest of the workspace is held quiet. The active pattern can also be kept going after its trigger has gone.
The indicator list in Butlin and colleagues (2023) breaks the theory into four properties. The first is a set of specialised modules working in parallel. The second is a limited-capacity workspace, with a bottleneck and a mechanism that selects what gets in. The third is global broadcast, meaning the workspace's contents reach all modules, including the input modules. The fourth is attention that depends on the system's state, so the workspace can query modules one after another.
From these I take four tests that can be applied to any shared memory, whether a whiteboard, a residual stream or an agent's scratchpad.
- Readership. Is the content actually used by many consumers, of different kinds, beyond what chance would give? Availability does not count. Use does.
- Competition. Does getting this content in come at the expense of rivals? Is there a limit, and a selection that decides what fills it?
- Ignition. Does the content become available suddenly and all at once, or does its influence grow smoothly with its strength?
- Reuse. Once in, is the content used for jobs it was not written for, such as report, planning or a new task, under the control of the system's current goals?
The kitchen makes the tests concrete. The rail passes readership for some notes and fails it for others: three stations read the allergy note, one did not. It has no competition, because a new note does not push an old one off. It has no ignition: a note is either on the board or not, and its effect depends on who happens to look. Joseph's shout passes all four. He chooses one note out of forty, which is selection under a limit. Everyone hears it at once, which is ignition. And the call is used for whatever each station needs: Imran checks his marinade, Babu checks his bowls, and Savitha tells the waiter. That is reuse.
How each test could be measured
Each test needs an operational version before it can be applied to a network, and the versions matter, because a vague test can be passed by almost anything.
Readership is the simplest. Choose a direction with a known meaning, perturb it slightly at the input of each later component, and count the components whose outputs change more than they would for random directions. The random comparison is essential, since every component responds to some degree to every perturbation, and the interesting question is whether a particular content is read selectively.
Competition needs two contents and an intervention. Write one content more strongly and measure whether the readership, or the downstream effect, of a rival content falls. A workspace predicts a substantial fall, because the rival has lost its place. A shared bus predicts little change apart from ordinary interference, because both contents remain in the sum.
Ignition needs a dose-response curve. Increase the strength of a content gradually and watch its downstream influence. A workspace predicts a threshold, below which influence is small and above which it is large and stable, with little in between. A graded system produces a smooth curve, however steep.
Reuse needs a transfer test. Find a content that was learned for one purpose and check whether components serving unrelated purposes also depend on it, and whether that dependence changes with the task the system is pursuing. This is the hardest of the four, since it requires an independent account of what each component is for.
A counterexample that keeps the vocabulary honest
Here is why reachability cannot be the test. A spreadsheet's total cell at the bottom of a column is reachable by every other cell in the workbook. Any formula anywhere can refer to it. Nobody calls it a global workspace, because it is simply a value that happens to be visible. If reachability were enough, every global variable in every program ever written would be a workspace. A global-access claim needs a test that a spreadsheet would fail. Each of my four tests does: the total cell has no competition, no ignition and, unless some formula reads it, no readers.
The residual stream is more interesting than a spreadsheet, because its readers are learned and its writes interact. But the logical point is the same. Showing that every layer can read every earlier write establishes the architecture. It says nothing yet about the four tests.
Why I do not use "modules" as a fifth test
Workspace theories require specialised modules, and transformers arguably have them in attention heads and MLPs. I leave this out because nearly every large network passes it under some description, so it does little work in separating boards from stages. Butlin and colleagues add a sharper point: in a transformer it is hard to say which parts are modules and which part is the workspace. If modules are tied to particular layers, nothing broadcasts back to them; if they are not, the stream cannot be told apart from the modules. I return to this in the objections.
Part 4
Running a readership audit on GPT-2
Readership is the test most directly about the residual stream, and the one most often assumed rather than measured. So I built an instrument for it and ran it on GPT-2 small, a 124-million-parameter model released by OpenAI in 2019 whose weights are freely available.
The instrument
The readership audit asks one question about one direction in the stream: how many later components actually listen to it? It works like this.
Take a direction u, a unit vector in the 768-number stream, and a layer L where it is written. For every component in every later layer, meaning each of the 12 attention heads and the MLP, compute that component's output twice. Once on the real stream, and once with a small nudge along u added to the stream. The nudge is 5 per cent of each token's stream length. Crucially, the nudge goes only into that component's input. The rest of the model runs as normal, so knock-on effects through other layers do not count. What is measured is direct reading.
The size of the change, relative to the component's usual output, is its gain for that direction. A gain means nothing on its own, since some components react to everything. So the same measurement is made for 100 random directions, and a component counts as a reader of u only if its gain beats 95 of the 100 random ones. By construction a random direction should then find about 5 per cent of components as readers. That is the negative control built into the instrument. The audit's output is the readership: the fraction of later components that count as readers.
The audit needs a direction with known meaning. I used grammatical number. I took ten singular subjects ("The dog", "The key") and their plurals ("The dogs", "The keys"), ran each through the first four layers, and took the difference between the average plural and singular stream vectors at the noun. On the ten training pairs, that direction separated every plural from every singular. The text for the audit was twelve ordinary sentences, and I skipped the first position, which in GPT-2 behaves very differently from the rest.
What came back
The number direction, written by layer 4, had 22 readers among the 104 components in layers 4 to 11. That is a readership of 21 per cent. Two fresh random directions, tested in exactly the same way, found 4 readers and 3 readers, about 3 to 4 per cent. That matches the 5 per cent expected by chance, so the control behaves.
The pattern across layers is the interesting part. The number direction had readers in every one of the eight layers, between one and five per layer. Every one of the 22 readers was an attention head; no MLP crossed the threshold. So the feature is read a little everywhere and a lot nowhere. That is a long way from 22 per cent being "the whole model listening". It is also a long way from one private reader.
Readership measures sensitivity. It does not show that the feature matters for behaviour. So the audit ends with a check of use. If the model needs this direction to keep track of number, changing it should change agreement between subject and verb. I took eight new noun pairs not used to find the direction, such as "The cats" and "The cat". For each, I measured how strongly the model preferred "are" over "is" as the next word.
My first version of this check went wrong in an instructive way. I removed the direction at every position, including the first word, "The". Agreement collapsed, and I nearly reported it. Then I ran the removal on the first position alone, leaving the noun untouched. That also wrecked the plural nouns, which fell from favouring "are" by 5.1 logits to favouring "is" by 2.4. The first position in GPT-2 is special, and disturbing it damages many things at once. My clean-looking result was partly the wrong cause.
So the real test changes the direction at the noun only, at layer 4, and treats it as a dial. At one end the dial sets the noun's value to the singular average; at the other, to the plural average. With the model intact, plural subjects favoured "are" by 5.1 logits and singular subjects favoured "is" by 5.6, a gap of 10.7. With the dial at the singular end, the plural nouns switched and favoured "is" by 5.4. With the dial at the plural end, the singular nouns switched and favoured "are" by 4.7. At the midpoint the gap shrank to 1.8. One direction, read by a minority of components, carried most of the model's sense of grammatical number at that point.
Linguists have used subject-verb agreement to probe neural networks for a decade, so the effect is not surprising in itself. What it adds here is the combination. A content that matters for behaviour, written at one layer, is read by a spread of later components, about a fifth of them, and every one of them an attention head. That is broadcast-like availability for one feature, measured directly. It partly passes test 1. The dial also gives a first look at test 3, which I take up in Part 6.
There is a true story behind why anyone noticed these giant values in the first place. When Tim Dettmers and colleagues (2022) tried to shrink large language models by storing their numbers in 8 bits instead of 16, the compressed models broke. The cause was a small set of "highly systematic emergent features" with outsized values, which, in their words, dominate attention and predictive performance. Squeezing them into 8 bits destroyed them. Their fix was to keep those few dimensions in 16 bits and compress everything else, which let a 175-billion-parameter model run on a single server. A handful of coordinates mattered so much that engineering had to route around them. That they matter, and that they are rarely read as content, are two different facts, and it is easy to confuse them.
The failure boundary: when the audit finds nothing
My first version of this instrument failed, and the failure is instructive. Before GPT-2, I built a small toy: a residual network of four blocks, with four outputs. Three outputs needed one hidden bit, computed from the input, and the fourth never needed it, as a control. I found the bit with a linear probe at every depth, with 99.5 per cent accuracy, and removed that direction from the stream.
Almost nothing happened. Each of the three consumers lost at most 1.2 points of accuracy. The control, which did not need the bit at all, lost the most: 3.4 points. On my readership measure the bit had no readers. That is plainly wrong, since the three consumers could not do their job without it.
The reason is redundancy. The network could recompute the bit from other directions in the stream, so removing one copy left others intact. McGrath and colleagues (2023) found the same thing at scale and named it the Hydra effect: remove one attention layer from a language model and another layer compensates. So readership audits have a failure boundary. When a content is stored in several places, removing or nudging one direction underestimates how widely it is used. The GPT-2 audit avoids the worst of this by nudging component inputs one at a time, not ablating the stream, but it cannot see readers who rely on a different copy of the same information. A low readership score is weak evidence of low use. A high score, with a clean random control, is stronger.
Part 5
Where competition lives, and where it does not
Test 1 is the easy one for the stream to pass partly. Test 2 is where the global-access reading is most exposed. A workspace is defined by scarcity. Contents fight for a small space, and the winner shuts the others out for a while. Is there anything like that among the contents of the residual stream?
Start with what the stream does instead of fighting. Elhage and colleagues (2022) showed in small models that networks store more features than they have dimensions. They do it by giving each feature a direction that is nearly, though not exactly, at right angles to the others. They called this superposition. It works because most features are rarely active at the same time. When several are active together, their directions overlap a little and each reader picks up some noise from the others.
That noise is a real cost of sharing. It looks a little like competition, because more active features mean a worse reading of each. But nothing is selected, and no feature wins and pushes the others out. Every feature still gets written, and every reader still gets a blurred mixture. In kitchen terms, superposition is forty notes written in small handwriting on overlapping parts of the rail. The more notes there are, the harder each is to read. Nobody decides which note gets the space.
Readers that specialise
The published picture of how components read the stream points the same way as my audit, towards specialisation rather than universal attention to everything. Geva and colleagues (2020) found that the MLP layers of a transformer behave like large key-value memories, in which each key responds to a particular textual pattern in the stream and each value pushes the prediction towards particular next words. A follow-up by Geva and colleagues (2022) showed that these updates can often be read as promoting recognisable concepts in the vocabulary. Each MLP is therefore a selective reader with its own interests.
More recently, Marks and colleagues (2024) traced sparse feature circuits, small networks of interpretable features that together explain a behaviour, and they were able to remove features a classifier relied on inappropriately. Their circuits are sparse by construction and by result: a behaviour typically depends on a modest number of features read by a modest number of components. None of this contradicts the possibility of a workspace somewhere, but it fits the board much better than the stage, since a stage would produce a few contents read by nearly everyone.
The one real budget
There is one place where the stream does impose a shared limit. Before each attention block and each MLP reads the stream, layer normalisation rescales the token's vector to a fixed length. So the reading each component sees is the stream divided by its own size. If one content is written very loudly, the vector gets longer, and after rescaling every other content looks smaller. That is a genuine budget. Room is finite after normalisation, and a loud write takes it from the quiet ones.
Static view: four notes of equal size each take 0.50 of a normalised reading. Make one note ten times louder and it takes 0.99, while each of the other three falls to 0.10. Normalisation shares a fixed budget.
The interactive shows the arithmetic with four notes of equal size. Each reads at 0.50 after rescaling. Make one ten times louder and it reads at 0.99, while each of the others drops to 0.10. That is a limited-capacity effect, which test 2 asks for. But look at what it lacks. All the quiet notes shrink together, in proportion. None is chosen and none is removed. A workspace selects; normalisation only divides. I hold this claim, C6, as speculative, because I have shown the arithmetic and not that trained models use it to make contents compete. The massive activations of Part 4 may be one such use. Sun and colleagues suggest they act as biases, and a fixed, huge coordinate would also set the scale for everything else after normalisation.
Competition among positions
Real competition does exist in a transformer, although not in the stream. Inside each attention head, the softmax forces the weights over earlier positions to add up to one. If one position gets more weight, the others must get less. That is competition with a fixed budget and a winner, exactly the kind a workspace needs. When a head attends sharply to one earlier word, it has selected that word for its own use.
The catch is that each head runs its own contest. GPT-2 small has 144 heads, and each picks its own winner from the positions before it. One head attends to the subject, another to the previous word, another to the first token, which Xiao and colleagues (2023) showed is often used as a place to park unused attention. There is no single winner for the whole model at a given moment. Attention gives each reader a private spotlight; it does not give the model a public stage.
That is the kitchen again. Each station can look along the rail and choose which note to read. Imran looks for the tandoor orders, and Babu looks for the desserts, so each one's attention is selective. But a choice by Imran does nothing to what Babu sees. Joseph's shout is different in kind. One choice, made in one place, reaches every station at once and stops them reading anything else for a moment. In the transformer, nothing I know of plays Joseph.
The atlas places these systems on two axes: how widely a content is read, and how hard contents compete. The spreadsheet cell sits low on both. The residual stream sits to the right, widely read, but low on competition. Attention over positions sits higher, because its softmax makes positions compete, though each contest is private. A global workspace, as the theory describes it, sits at the top right. The placements are my judgement, and the point is the empty space. Nothing at the level of a single forward pass reaches that corner.
A third analogy, and its break
A family WhatsApp group is a closer everyday picture of the stream than a stage. Everyone in the group can see every message, which is reachability. Most messages are read by a few members and skimmed by the rest, which is readership below the maximum. No message deletes another, and a long one does not silence the short ones, which is the lack of competition. When the group gets busy, each message is harder to find, which is interference. Where the analogy breaks is in time and in the readers. Family members read whenever they like, remember what they read, and reply to each other. Readers in the stream are fixed matrix products, each run exactly once, in a fixed order, with no memory of their own. The group chat has more life in it than the stream does.
The storehouse and the stage
The Indian traditions I draw on have a vocabulary for exactly this difference. In Yogācāra Buddhism, Vasubandhu's Thirty Verses describe a storehouse consciousness, ālayavijñāna. Every act leaves a seed, bīja, in the storehouse. The seeds stay there, unnoticed, and ripen later when conditions call for them. Separately, a mind consciousness, manovijñāna, brings particular objects forward into awareness, one field at a time.
The residual stream looks much more like the storehouse than like the stage. Every write stays in the sum. Later layers draw on earlier writes when the input makes them relevant, which is a fair description of seeds ripening. What the stream lacks is the second function, the one that brings a single content forward and holds it there. In Sāṃkhya terms, the inner instrument, antaḥkaraṇa, has a part that holds impressions and a part that decides, buddhi. The stream holds; the heads and MLPs decide, each for itself.
The break-point map shows where the analogy holds and where it fails. It holds for storage and for delayed use. It strains at the split between holding and deciding, since in the stream the deciders are many and none speaks for the whole. It breaks completely at the knower. Both traditions assume a knower for whom contents appear, whether that is the luminous self of Sāṃkhya, puruṣa, or the stream of awareness Yogācāra analyses. In the residual stream, readers are matrix products. I use the analogy because it names the missing function precisely, not because it shows the stream has a mind.
What sharing gives on the RTM ladder
RTM, the hypothesis I am testing across this series, treats mind as five layers. L1 is responsiveness, spanda, the bare fact that inputs change states. L2 is manifestation, prakāśa, contents becoming available for use. L3 is reflexivity, vimarśa, a system representing its own states. L4 is recognition, pratyabhijñā, the system knowing those states as its own. L5 is the luminous ground, cit, which RTM says no mechanism produces.
The residual stream clearly gives L1: every write shifts every later read. It gives a wide, partial L2: contents become available across layers, and some, like grammatical number, are used by many readers. It does not select, so the L2 it gives is availability without foregrounding. It gives nothing towards L3. No component of a standard transformer, as far as anyone has shown, represents the state of the stream itself and uses that representation. I state this as claim C7 and mark it speculative, because a component like that could exist and not have been found.
Here is the sharpest RTM point the stream offers. If a global workspace were found, it would strengthen L2. It would make some contents not only available but foregrounded. It would still not supply L3. A spotlight on the board is not the board knowing it has a spotlight. That is where RTM and workspace theory part ways, and Thought Experiment 4 is the place where RTM could lose.
Part 6
The board-or-stage test in use
The practical output of this paper is a four-step test for any claim that a model, or a system built on one, has a global workspace. I call it the board-or-stage test. Each step needs its own measurement, and each failure has its own name.
- Is the content read by many components, beyond what random directions get? If not, it is a private note: ordinary local accumulation.
- If it is widely read, does writing it reduce the readership of rival contents? If not, it is a shared bus, broadcast-like but not a workspace.
- If it competes, does it become available all at once, or does its influence grow smoothly with its strength? If smoothly, call it gating.
- If it ignites, is it reused for jobs it was not written for, under the control of current goals? If all four pass, the workspace reading is earned.
The test is cheap to start. A readership audit of one direction in a small open model takes minutes on an ordinary laptop processor, and the random control adds only a constant factor. The competition test needs a second content and a few hundred forward passes. The ignition test is a dose-response curve, which is what the dial in Part 4 already produced. Only the reuse test needs real thought, since it requires a prior account of what the components are for. None of these demands access to a frontier model, which means anyone making a workspace claim about an open model can be asked to run them.
This order is deliberate. It runs from cheapest to most demanding, and each test is only worth running if the one before passes. It also gives honest names to the intermediate cases. "Shared bus" and "gating" are useful descriptions, and they are what most systems actually are.
A first look at ignition
The dial from Part 4 gives a rough test 3 for grammatical number. I moved the noun's value along the number direction in nine steps, from the singular average to the plural average. Then I watched two things: the model's preference for "are" over "is", and the size of the number direction in the stream at later layers.
The preference did not change evenly. For plural nouns it moved from favouring "is" by 3.6 logits to favouring "are" by 2.6 across the middle quarter of the dial. That is about 60 per cent of the whole swing in a quarter of the range. In the stream, the change was amplified. By layer 10 the difference between the two ends had nearly doubled, and the amplification was steepest near the middle, about five times steeper than at the ends.
So there is a steepening, which is how ignition would begin. But there is no jump. Every step on the dial gave a different answer, and the curve is smooth throughout. The other half of ignition, activity that sustains itself after the trigger is gone, cannot even be tested in one forward pass, since nothing loops back. My verdict on test 3 for this feature is "weak": graded amplification near a decision boundary, which is what any good classifier does.
A realistic worked case: the vendor's slide
Here is an invented but typical case, flagged as such. A vendor is pitching a model-editing product to the risk team of a bank in Bengaluru. Slide 14 says: "Our model has a global workspace in its residual stream. A single steering vector, added at layer 20, is broadcast to the whole model, so one edit changes behaviour everywhere."
The risk team has to decide whether to rely on single edits for a compliance control, such as stopping the model from recommending a product to customers who do not qualify.
The board-or-stage test turns the slide into four questions. For test 1, ask for a readership audit of the steering direction with a random-direction control. Our GPT-2 numbers give a scale: 21 per cent readers against 3 to 4 per cent for random. If the vendor's direction scores near random, it is a private note, and the edit will work only where its few readers matter. For test 2, ask whether adding the vector reduces the readership of rival directions, for example the direction that carries the customer's eligibility. If not, the edit and the eligibility signal coexist in the sum, and later readers will mix them.
For test 3, ask for a dose-response curve. A smooth curve means the edit's effect depends on how hard you push, which matters for a control, since some inputs will push back. For test 4, ask whether the same edit changes behaviour on tasks outside the ones it was tuned on. Without that, "everywhere" means "on the evaluation set".
The likely result, based on everything above, is a pass on test 1, no evidence on test 2, a graded curve on test 3 and partial results on test 4. The honest description is then "a widely read steering direction on a shared bus". The decision follows from the description. A control that relies on one edit needs a monitor at the output as well, because nothing in the architecture stops other writes in the stream from diluting it.
Static view: for the grammatical-number direction in GPT-2 small, test 1 (readership) passes, test 2 has no evidence, test 3 is weak (a smooth, steep swing with no jump), and test 4 is untested. Verdict: shared bus, not workspace.
The interactive applies the test to four presets. The GPT-2 number feature passes test 1 and stops there on current evidence. Dimension 447 fails at the first step. The agent scaffold of Thought Experiment 4 passes most tests. Workspace theory passes all four, by definition. Clicking through them is a quick way to see that "has a global workspace" is four claims, not one.
Objections
The best case that I have this wrong
The strongest objection is that I have looked at the wrong grain, and it deserves full strength. A transformer generating text is recurrent across tokens. Each new token is fed back as input, and attention at every layer reads the stored keys and values of all earlier tokens. So information written at one step does flow back into the processing of the next. Butlin and colleagues' main complaint, that nothing flows back to the modules, is answered once you look across time instead of within one pass. On this view the workspace, if there is one, is the sequence of streams over many tokens, and my single-pass audit could never see it.
I think this gets something right. Generation does create a loop, and the loop is where an agent's scratchpad lives too. But look at what flows back. It is one token, chosen from the output, plus a cache of earlier writes that later steps can read. The token is a bottleneck with a winner, which is interesting. It is also the model's output, which every reader of the next step sees. That makes the sampled token a better candidate for a workspace than anything inside the stream. I accept the objection as a redirection, since it moves the question to the token loop. It does not show the stream itself is a stage.
A second objection says my reading of workspace theory is too strict. Later versions of the theory relax the single winner. They allow a few items in the workspace at once, and they describe capacity limits as soft. Under a relaxed theory, LayerNorm's budget plus superposition's interference might count as limited capacity, and 21 per cent readership might count as broadcast. I find this half persuasive. Relaxing the theory makes the stream pass more easily, but it also makes the theory pass for almost any shared memory with a size limit. At that point the spreadsheet counterexample comes back. A theory that cannot tell a stream from a group chat has given up the content that made it worth applying.
A third objection is about the evidence. My audit is one model, one layer, one feature and twelve sentences. GPT-2 small is tiny by current standards. Features in larger models might be read far more widely, and the stage-like contents might be exactly the ones I did not test. All of this is true. It is why claim C2 is marked "likely" and why prediction P16-2 is stated for models of 7 billion parameters or more. The audit is an instrument with a demonstrated control. It is not yet a survey.
A further objection concerns the modules, and I promised in Part 3 to return to it. Butlin and colleagues point out that it is unclear which parts of a transformer count as the modules. If the modules are the heads and MLPs of particular layers, then nothing broadcasts back to them, since each runs once. If the modules are not tied to layers, the stream and the modules cannot be told apart. My readership audit treats each head and MLP as a separate reader, which quietly adopts the first option. A critic could say that I have built the conclusion into the unit of analysis.
My reply is that the audit does not depend on calling heads modules. It depends only on counting which components respond to a direction. Whatever we call them, the readership number stands. The module question matters for the theory, and it cuts against the workspace reading more than against mine. If the stream cannot be separated from its modules, then there is no distinct workspace to speak of, which is the structural problem Butlin and colleagues raised.
A fourth objection comes from my own side. A Trika reader might say that comparing the stream with the storehouse already flatters it. The storehouse in Yogācāra is a level of consciousness, and the residual stream is a sum of vectors. On this view even the Part 5 analogy smuggles in more than it should. I accept the warning, and the break-point map marks the knower row as broken for that reason. I keep the analogy because the distinction it names, holding versus foregrounding, is one the stream lets us test. The comparison is about function. If it ever reads as a claim about experience, it has gone wrong.
Tests
What would change my mind
Four observations would weaken the thesis. First, an intervention study in a standard pretrained transformer showing that writing one content strongly reduces the readership of a rival content, by half or more, without added recurrence or gating. That would break claim C5 and directly contradict prediction P16-1. It is the observation I most want someone to look for, because it is testable now with the instrument in Part 4.
Second, readership audits of larger models finding that most meaningful directions are read by most later components. That would turn "reachable is not read" from a finding into a quirk of small models, and push C2 down. Third, a single-pass ignition effect: a content whose stream size jumps between two layers once its input crosses a threshold, with nothing in between. My dial found steepening and no jump, but one feature in one small model is not much.
Fourth, for the RTM claim C7: a component that represents the state of the stream itself, for instance how confident or how mixed the current contents are, and uses that to steer later processing. That would give L3 a foothold in the architecture, and I would have to revise the ladder figure.
What I still do not know
I do not know whether any contents in the stream compete, because I have not run the test, and I have found no one else who has run it in this form. I do not know how readership changes with scale. I do not know the best way to measure readership when information is stored in several copies, which is the failure boundary of my instrument. I do not know whether the real workspace question belongs at the level of the agent loop. And I do not know whether global access, even if found, has anything to do with experience. That last question sits outside what any audit can answer.
Ledger
Claim Ledger
Each main claim of this paper, with how sure I am, what kind of evidence carries it, and what would knock it down.
| ID | Claim | Confidence | Evidence type | What would refute it |
|---|---|---|---|---|
| C1 | The residual stream is a shared additive channel: every component reads a projection of the sum of all earlier writes. | established | Architecture; Elhage et al. (2021) | A transformer variant where later blocks cannot read earlier writes, performing equally |
| C2 | Reachability is not readership: a meaningful direction is read by a minority of later components, though by many more than chance. | likely | This paper's audit of GPT-2 small: 22 of 104 against 3 to 4 for random directions | Audits in several models finding most meaningful directions read by most components |
| C3 | The largest-magnitude dimension is not the most broadcast; in the audit it had no detected readers. | likely | This paper's audit; consistent with Sun et al. (2024) on massive activations as biases | A larger audit showing massive dimensions carry input-dependent content to many readers |
| C4 | Removing one direction underestimates use when information is stored redundantly. | established | McGrath et al. (2023); this paper's toy null result | Single-direction ablations matching full causal analyses across models |
| C5 | Stream contents do not compete for exclusive access; superposition causes interference, not selection. | likely | Architecture and Elhage et al. (2022); absence of evidence for exclusivity | An intervention showing one written content suppresses the readership of rivals |
| C6 | Layer normalisation imposes a weak shared budget on the stream, a limited-capacity effect without selection. | speculative | Arithmetic of normalisation; illustrative interactive | Evidence that models exploit normalisation to make contents compete |
| C7 | Residual sharing supplies L2-style availability and nothing towards L3 reflexivity. | speculative | Conceptual analysis under RTM | A component shown to represent and use the state of the stream itself |
| C8 | Claims that a model has a global workspace should report readership, competition, ignition and reuse separately. | likely | Derived from Dehaene et al. (1998) and Butlin et al. (2023) | A workspace theory that does not need one of the four |
Dated predictions
| ID | Prediction | Judged by | How it will be judged |
|---|---|---|---|
| P16-1 | By the end of 2028, no peer-reviewed study will show that one content in the residual stream of a standard pretrained transformer suppresses the readership of a rival content by more than half, without added recurrence or gating. | Peer-reviewed interpretability literature | Literature search on 31 December 2028 for intervention studies of content competition in the residual stream |
| P16-2 | Readership audits of models larger than 7 billion parameters will find that most interpretable features are read by fewer than 40 per cent of later components. | Published audits or my own rerun | Rerun this paper's instrument, with a fixed threshold, on an open model of 7B or more |
Words
Glossary
- Residual stream
- The running sum of the embedding and all block outputs, passed from layer to layer in a transformer.
- Residual connection
- A shortcut that adds a block's input to its output, so the block learns a change rather than a replacement.
- Read and write
- A block reads by projecting the stream through its input weights and writes by adding its output back.
- Direction or feature
- A vector in the stream whose size along it carries some piece of information, such as grammatical number.
- Readership
- This paper's measure: how many later components change their output when one direction is nudged, beyond random.
- Superposition
- Storing more features than there are dimensions by letting them share directions, at the cost of interference.
- Layer normalisation
- Rescaling each token's vector to a standard size before a block reads it.
- Massive activation
- One of a handful of stream values far larger than the rest, nearly constant across inputs.
- Global workspace
- A theory of conscious access: many specialist processes, a limited stage, and one content broadcast to all at a time.
- Ignition
- In workspace theory, the sudden, all-or-none amplification that makes one content globally available.
- ālayavijñāna आलयविज्ञान
- Sanskrit: storehouse consciousness in Yogācāra, holding the seeds left by past actions.
- bīja बीज
- Sanskrit: seed; a latent trace that ripens into experience when conditions arise.
- manovijñāna मनोविज्ञान
- Sanskrit: mind consciousness, which takes up one object at a time.
- antaḥkaraṇa अन्तःकरण
- Sanskrit: the inner instrument in Sāṃkhya and Vedānta, comprising mind, intellect and ego-sense, with citta as its store.
- prakāśa प्रकाश
- Sanskrit: the light that makes things show up; RTM layer L2, manifestation or access.
- vimarśa विमर्श
- Sanskrit: reflective awareness; RTM layer L3.
Conclusion
A board everyone can read is not yet a stage
Back at Annapoorna, the night after the cashew scare, Lakshmi did not buy a bigger whiteboard. She knew the board was already reachable by every station. The trouble was that nothing on it ever won. She made a rule instead: any allergy note goes to Joseph, and Joseph calls it. The board stayed a board. The stage was added on top, as a separate job with a limit of one call at a time.
That is the shape of the answer to this paper's question. Residual-stream sharing supports a global-access reading only when four things are shown for a content. It must be read by many components beyond chance. It must beat rivals for limited room. It must become available suddenly, and it must be reused for new jobs. Ordinary feature accumulation passes the first test at most. In GPT-2 small, grammatical number passes it partly, with 22 readers of 104 against 3 or 4 for random directions, and it changes behaviour when moved. The loudest coordinate fails it entirely. Competition among contents, the mark of a stage, has not been shown anywhere in the stream.
For the series, the RTM result is narrow. Sharing gives a model a wide L2 rung, availability without foregrounding, and nothing above it. If a stage exists in these systems, the best places to look are attention's private contests, the sampled token, and the agent loop. Those are the next papers' business.
There is also a lesson about method that I did not expect when I started. My first behavioural test looked like a clean success and was partly the wrong cause, because removing the direction at every position also disturbed the first token. My first toy instrument returned a null result for a feature that three outputs plainly needed, because the network kept copies. Both mistakes would have produced confident sentences in a paper. Both were caught only because I ran a control that could have embarrassed the result. Interpretability claims about global properties of networks are especially exposed to this kind of error, since the architecture invites a broad conclusion before the measurement supports it.
The kitchen has one more thing to teach. Lakshmi's solution worked because she added a stage that was small on purpose. Joseph can call one thing at a time, and the limit is what makes the call reach everyone. A bigger board would have made the problem worse, since more notes would have competed for the same glance. If engineers ever want a workspace in their systems, the lesson from the pass is that it will come from a deliberate limit, placed somewhere specific, and not from making the shared board wider.
The practical rule is simple. When someone says a model has a global workspace, ask which of the four tests they ran. Most will have run the first. The interesting systems will be the ones that pass the second.
Evidence
Sources
Primary research
- He, K., Zhang, X., Ren, S. and Sun, J. (2015). Deep residual learning for image recognition. arXiv:1512.03385. Verified.
- Srivastava, R. K., Greff, K. and Schmidhuber, J. (2015). Highway networks. arXiv:1505.00387. Verified.
- Veit, A., Wilber, M. and Belongie, S. (2016). Residual networks behave like ensembles of relatively shallow networks. arXiv:1605.06431. Verified.
- Vaswani, A. et al. (2017). Attention is all you need. arXiv:1706.03762. Verified.
- Elhage, N., Nanda, N., Olsson, C. et al. (2021). A mathematical framework for transformer circuits. Transformer Circuits Thread. transformer-circuits.pub. Verified.
- Elhage, N., Hume, T., Olsson, C. et al. (2022). Toy models of superposition. arXiv:2209.10652. Verified.
- Belrose, N. et al. (2023). Eliciting latent predictions from transformers with the tuned lens. arXiv:2303.08112. Verified.
- Geva, M., Schuster, R., Berant, J. and Levy, O. (2020). Transformer feed-forward layers are key-value memories. arXiv:2012.14913. Verified.
- Geva, M., Caciularu, A., Wang, K. R. and Goldberg, Y. (2022). Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. arXiv:2203.14680. Verified.
- Dettmers, T., Lewis, M., Belkada, Y. and Zettlemoyer, L. (2022). LLM.int8(): 8-bit matrix multiplication for transformers at scale. arXiv:2208.07339. Verified.
- Sun, M., Chen, X., Kolter, J. Z. and Liu, Z. (2024). Massive activations in large language models. arXiv:2402.17762. Verified.
- Xiao, G. et al. (2023). Efficient streaming language models with attention sinks. arXiv:2309.17453. Verified.
- McGrath, T., Rahtz, M., Kramár, J., Mikulik, V. and Legg, S. (2023). The hydra effect: emergent self-repair in language model computations. arXiv:2307.15771. Verified.
- Lad, V., Lee, J. H., Gurnee, W. and Tegmark, M. (2024). The remarkable robustness of LLMs: stages of inference? arXiv:2406.19384. Verified.
- Gromov, A. et al. (2024). The unreasonable ineffectiveness of the deeper layers. arXiv:2403.17887. Verified.
- Marks, S. et al. (2024). Sparse feature circuits. arXiv:2403.19647. Verified.
- Dehaene, S., Kerszberg, M. and Changeux, J.-P. (1998). A neuronal model of a global workspace in effortful cognitive tasks. PNAS 95(24), 14529-14534. doi:10.1073/pnas.95.24.14529. Verified.
- Butlin, P., Long, R. et al. (2023). Consciousness in artificial intelligence: insights from the science of consciousness. arXiv:2308.08708. Verified.
- Goldstein, S. and Kirk-Giannini, C. D. (2024). A case for AI consciousness: language agents and global workspace theory. arXiv:2410.11407. Verified.
Primary texts and translations
- Vasubandhu, Triṃśikā (Thirty Verses), verses 2-5 on ālayavijñāna and its seeds. Standard editions; paraphrased rather than quoted.
- Īśvarakṛṣṇa, Sāṃkhya Kārikā, kārikās 23-35 on buddhi, ahaṃkāra, manas and the inner instrument. Standard editions; paraphrased.
Secondary and leads
- nostalgebraist (2020). Interpreting GPT: the logit lens. LessWrong, 31 August 2020. Verified.
- Baars, B. J. (1988). A Cognitive Theory of Consciousness. Cambridge University Press. Background; not quoted.
- Radford, A. et al. (2019). Language models are unsupervised multitask learners. OpenAI technical report (GPT-2). Model used in this paper's audit, weights via Hugging Face.
Reference
Cite this work
Mahapatra, R. R. (2026). The residual stream is a shared board, not a stage: when sharing earns the name global access. Mind Frontier paper U0564. rajeshrmahapatra.com. https://rajeshrmahapatra.com/papers/mind-frontier/U0564-residual-stream-shared-board.html. DOI: to be assigned.
@misc{mahapatra2026residual,
author = {Mahapatra, Rajesh Ranjan},
title = {The residual stream is a shared board, not a stage: when sharing earns the name global access},
year = {2026},
howpublished = {Mind Frontier paper U0564, rajeshrmahapatra.com},
url = {https://rajeshrmahapatra.com/papers/mind-frontier/U0564-residual-stream-shared-board.html},
doi = {10.xxxx/placeholder}
}