Part 1 · Mechanism

What the cache holds, and what working memory means

What the cache holds

A transformer reads text as tokens, pieces of words. At each of its layers, every token is turned into three lists of numbers. The query says what this token is looking for. The key says what kind of thing this token is, for matching. The value is what this token hands over if it is chosen. A token's output at a layer is a weighted average of the values of all earlier tokens. The weights come from how well its query matches each of their keys (Vaswani et al., 2017).

When a model writes a reply, it produces one token at a time. Each new token needs the keys and values of every earlier token, at every layer. Recomputing them for each new word would be slow, so the model stores them. That store is the key-value cache, or KV cache. It is written once per token and then read by every token that follows. In a long WhatsApp thread, the cache holds a key and a value for every token of every message, at every layer.

Two facts about the cache matter for everything that follows. The first is that it is exact. A key written for "apixaban" in message four is the same numbers, to the last digit, when message forty-four is being written. Nothing in the standard design weakens it, blurs it or lets it fade. The second is that it is passive. The cache doesn't decide what to keep, what to refresh or what to read. All the choosing happens in the attention step, where each new query is compared with every stored key.

So the cache is a store with a reader. The reader is attention, run by many heads in every layer. Each head has its own way of forming queries and keys, so each one is looking for something different. Some look for the previous token, some for names, some for the token that followed the last time this pattern appeared. Chris Olsson and colleagues at Anthropic traced one such reading circuit, the induction head, which finds an earlier occurrence of the current token and copies what came after it (Olsson et al., 2022). That is how a model can repeat a name it saw once.

How big it gets, by hand

The size of the cache is simple arithmetic, and doing it once makes the engineering choices clearer. For each token, the cache stores a key and a value, for every layer and every key-value head. Each key and each value is a list of numbers called the head dimension.

The small model I used, SmolLM2-360M, has 32 layers. It has 5 key-value heads per layer, and each head has 64 numbers (SmolLM2 model card). That is 2 × 32 × 5 × 64 = 20,480 numbers per token. At two bytes per number, each token costs about 40 kilobytes of cache. The model's longest context is 8,192 tokens, which comes to about 335 megabytes. That is more than half the size of the model's own weights, for one conversation.

This model already shares each key-value head among three query heads. With one key-value head for each of its 15 query heads, the cache would be three times bigger. That sharing is called grouped-query attention (Ainslie et al., 2023). It is a middle path between full attention and multi-query attention, where every query head shares a single key and value (Shazeer, 2019). DeepSeek went further with multi-head latent attention. It compresses keys and values into a small shared code, and the DeepSeek-V2 paper reports a 93.3 per cent smaller cache than its earlier 67B model (DeepSeek-AI, 2024).

The cache is big enough that serving systems manage it like an operating system manages memory. The vLLM project stores it in fixed-size pages, borrowed from the way computers handle virtual memory, so that little of it is wasted (Kwon et al., 2023). Other work throws parts of it away. StreamingLLM keeps the first four tokens and a window of recent ones. It reports stable output over four million tokens and up to 22.2 times faster decoding than recomputing a sliding window (Xiao et al., 2023). Heavy-Hitter Oracle, or H2O, keeps the recent tokens plus the ones that have drawn the most attention so far (Zhang et al., 2023).

Those eviction schemes are where a designer chooses what the model forgets. They are the nearest thing the cache has to forgetting at all, and I test them in Part 3.

What psychologists mean by working memory

Working memory is the mind's workbench. It holds a phone number while you dial it, the start of a sentence while you hear the end, and the carry digit in mental arithmetic. In 1974 Alan Baddeley and Graham Hitch proposed that it is not one store but several parts. There is a loop for sounds and words, a sketchpad for images and space, and a central executive that directs attention between them (Baddeley and Hitch, 197460452-1)). In 2000 Baddeley added an episodic buffer. He described it as "a limited capacity system that provides temporary storage" and binds information from different sources into single episodes (Baddeley, 200001538-2)).

Nelson Cowan drew the picture differently. In his embedded-processes model, working memory is not a separate box. It is the part of long-term memory that is currently active, with a small focus of attention moving over it. He argued that the focus holds about four chunks at a time, not the seven that George Miller had made famous (Cowan, 2001; Miller, 1956).

From these two models I take seven functional criteria. They are what a cognitive scientist would check before calling something working memory.

1. Temporary. It holds information for the task at hand, then lets it go.
2. Limited capacity. It holds only a few items, around four chunks.
3. Loss without maintenance. Unrehearsed items are lost over seconds, by decay or interference.
4. Interference. Similar items get in each other's way, and old items spoil new ones.
5. Executive control. Something chooses what to hold, refresh, update or drop.
6. Binding. It ties features into episodes: this name, that value, this time.
7. A narrow focus. Only a small part of what is held is directly available at once.

The first four describe the store. The last three describe what is done with it. The cache will turn out to split along almost exactly that line, but in a strange way. As a store it is too good. As something read from, it shows several of the limits the psychologists found.

Storage and access are different questions

Psychologists separate how well something is stored from how well it can be got at. A word you can't recall on the spot will often come back when someone gives you its first letter. The trace was there all along. What failed was access. The distinction is the key to the cache, because the cache separates storage and access more cleanly than any brain.

In the cache, storage is perfect by construction. Every key and value is kept exactly, for the life of the conversation. If the model gets something wrong about a fact in its context, storage is never the explanation. The explanation is always in the reading: the query didn't match the right key strongly enough, or other keys matched too well, or the value retrieved was mixed with others in the weighted average.

That is why the question "is the KV cache working memory?" needs splitting before it can be answered. Ask about the cache as a store and the answer is no: it is too large, too exact and too passive. Ask about the cache together with the attention that reads it, and several of the working-memory limits appear. My test in Part 3 measures the second, because nothing else can be measured. A cache is only ever seen through what the model does with it.

Part 2 · The evidence

Seven, then four; decay or interference; lost in the middle

Seven, then four

Cowan's revision matters here because of what it located. The limit of four was not a limit on what the brain stores. Long-term memory is vast. It was a limit on what can be held in the focus at once. Everything else that is active sits just outside the focus, quickly available but not directly used. That is a much closer match to a cache read by attention than Miller's seven-slot box.

Decay or interference?

The argument didn't end there. Alan Baddeley and Donald Scott tested hundreds of people who each heard a single list. That design ruled out interference from earlier lists, and they still found forgetting within seconds. Their reading was a short-term store that decays within about five seconds. Ian Neath and Gordon Brown later showed that a model with no decay at all could fit Baddeley and Scott's data. Items crowded together are simply harder to tell apart (Neath and Brown, 2012).

I bring up a sixty-year-old quarrel because the cache settles one side of it, for machines at least. A system with no decay at all can still forget. In the cache nothing fades, so every loss must come from interference or crowding. If the model fails to find a fact after a long stretch of filler, the filler did it, not the passing of time. That gives a clean test, which I run in Part 3. Fill the gap with unrelated text, then with text full of similar facts, and see which one hurts.

Machines lose the middle

The most cited long-context result is a curve that any psychologist would recognise. Nelson Liu and colleagues put a relevant document, or a key-value pair, at different positions in a long context. They found that performance was "often highest when relevant information occurs at the beginning or end of the input context". It "significantly degrades" in the middle, even in models built for long contexts (Liu et al., 2023).

In human memory this shape has a name: the serial-position curve. The first items on a list are remembered well, which is called primacy. The last items are remembered well too, which is called recency. The middle sags. Psychologists have argued for decades about what causes each end. For the model, nobody thinks the start is rehearsed, and nobody thinks the end is fresher in a fading store. Both ends are effects of how attention reads, and of how the model was trained.

The second result is about claimed versus usable length. Simple "needle in a haystack" tests hide a single odd sentence in a long text and ask for it back. Most long-context models pass them. RULER, a benchmark from NVIDIA, added harder variants: several needles, needles that look alike, tracing a variable through a chain of assignments. Of the models it tested that claimed 32,000 tokens or more, only half kept satisfactory performance at 32,000 (Hsieh et al., 2024).

That gap is what Cowan would predict. The store holds everything. The focus can find one distinctive item in it easily. It struggles when several similar items compete, or when finding one item depends on first finding another. That is a limit on access, and it is the limit that bit Nidhi. Warfarin and apixaban are similar items, both blood thinners, both linked to George's name.

The first tokens are a resting place

StreamingLLM began with a puzzle. If you keep only a sliding window of recent tokens in the cache, a model falls apart as soon as the first tokens drop out. That happens even when those tokens are a meaningless start-of-text marker. Guangxuan Xiao and colleagues called those first tokens attention sinks. Attention weights must add up to one, so heads that have nothing useful to read dump their weight on the first tokens. Remove them, and the dumped weight lands somewhere it does damage (Xiao et al., 2023).

Keep four sink tokens plus the recent window, and the model stays stable for millions of tokens. But it is stable in the way a person who only remembers the last five minutes is stable. Its text stays fluent. What it can recall shrinks to the window. I test exactly this in Part 3: does keeping the sinks preserve memory for facts, or only fluency?

H2O starts from a different observation. A small share of tokens draws most of the attention, and the authors called them heavy hitters. Keeping recent tokens plus heavy hitters, with 20 per cent of the cache given to heavy hitters, improved throughput over leading inference systems by up to 29 times (Zhang et al., 2023). That is the nearest thing in engineering to an executive that chooses what to keep. Note what it uses to choose, though: past attention. It keeps what has been looked at, not what will be needed. For George, the apixaban line was looked at once, in message four, and then not again for forty messages.

Attention as a memory model

There is a deeper reason the cache behaves like a memory. Hubert Ramsauer and colleagues showed that the attention step is the update rule of a modern Hopfield network (Ramsauer et al., 2020). A Hopfield network is a classic model of associative memory. You give it a partial or noisy cue, and it settles on the stored pattern closest to the cue.

Read this way, the keys are stored patterns and the query is a cue. One step of attention retrieves a blend of stored values, weighted towards the closest keys. If one key is much closer than the rest, retrieval is clean. If several keys are about equally close, the result is a blend, a "metastable state" in the paper's terms. That is interference, written as maths. It doesn't depend on how long ago anything was stored. It depends on how distinctive the cue is among the keys.

What the evidence adds up to

Line the published work against the seven criteria and a pattern shows. The cache is temporary, in that it is cleared when the conversation ends. Its capacity as a store is not limited, but its effective capacity, what attention can use, falls well short of its size. Nothing in it decays, but interference shows up in every hard benchmark. There is no executive inside the cache, though eviction schemes add a crude one from outside. Binding of a name to a value works when the pairs are few and distinct. A narrow focus is the clearest match of all, since attention puts most of its weight on a few keys at a time.

What the published work doesn't give is the same model, tested on all of these at once, with controls. So I ran that.

Part 3 · A small replication

Five tests on one small model

I wanted one model tested on the working-memory criteria that can be tested, with controls, so the results can be read against each other. I used SmolLM2-360M-Instruct, a small open model, because it runs on a laptop and its cache can be reached directly (SmolLM2 model card). The code and results are in the programme's repository.

The task

Every test uses the same kind of fact. "Ravi's locker holds a kettle." "Meera's locker holds a drum." Each name is a short first name. Each object is one of 47 everyday nouns, each a single token, so the answer is one step of the model. After a list of such facts comes a question: "What does Ravi's locker hold?" The prompt ends with "Answer: Ravi's locker holds a", and I read off which of the 47 objects the model ranks highest.

Chance is about 2 per cent. The measure is accuracy: the share of trials on which the right object ranks first. This is a binding task in Baddeley's sense, a name tied to an object, held in context, retrieved by cue. It is also a fair copy of George's problem: a name, an attribute, and similar competitors in the same context.

Capacity: the focus narrows as the store grows

I varied the number of facts in the list from 2 to 48, with 24 trials at each size, asking about a random one each time. With two facts the model was right every time. With four or eight it was right about four times in five. With sixteen it fell to just under three in five, and with 32 or 48, to about one in three.

The curve is the first result. The store had no trouble: at 48 facts the whole prompt was about 420 tokens, a tiny fraction of the cache this model can hold. But usable recall fell steadily as facts were added. That is a capacity limit, and it is not in the store. Every fact was there, exact. The limit is in how cleanly a query can find one name among many similar ones.

It is not a limit of four in Cowan's sense. Even at 48 facts the model did far better than chance, so it is using many more than four items. What it shows is a soft limit. Each added fact makes every other fact a little harder to find. That is the shape interference models predict, and the shape a fixed number of slots does not.

Position: the middle sags

Next I fixed the list at 24 facts and asked about each position in turn, over six different lists. The first fact was recalled every time. The last few were recalled about four times in five. Positions in the middle fell to one in three or worse, with some at zero.

Six lists is a small sample, and the curve is jagged. But its shape is the serial-position curve, primacy and recency with a sagging middle. It matches the shape Liu and colleagues found in much larger models with much longer contexts. The cache holds every position equally. Attention does not read them equally.

Delay: filler does nothing, look-alikes do damage

This is the test that settles decay against interference. I gave four facts, then a gap, then the question. The gap was filled in one of two ways. Neutral filler was plain sentences about weather, trains and markets, with no names and no lockers. Similar filler was more locker facts, about other people. I used gaps of 0, 100, 400 and 1,000 tokens, with 20 trials each.

With no gap, recall was 75 per cent in one condition and 65 in the other. The two conditions are identical at zero gap, so that ten-point difference is a fair measure of the noise. After 100, 400 and 1,000 tokens of neutral filler, recall was 85, 90 and 85 per cent. After the same gaps filled with other people's locker facts, it was 55 per cent each time.

If the cache decayed, the neutral gap would hurt, and the longer the worse. It didn't. A long gap of unrelated text left recall where it was, or even slightly better. If interference does the forgetting, similar filler should hurt and neutral filler shouldn't. That is what happened. This is Keppel and Underwood's result, in a machine where decay is ruled out by construction.

Updating: the latest version is fragile

Then George's problem directly. I gave a list of facts about six people and then, later, an update about one of them. Sometimes there was no update, sometimes one, sometimes three. Each update read "Later, Ravi's locker was emptied and now holds a lamp", with facts about other people in between. I asked for the current object, and counted how often the model gave the latest value and how often an older one.

With no update, the model gave the right object 95 per cent of the time. After one update, it gave the new object 70 per cent of the time and the old one 25 per cent. After three updates, it never once gave the latest object in 20 trials. It gave an older one 85 per cent of the time. A second run, with fresh names and objects, gave 50 per cent after one update and 10 per cent after three. The numbers move between runs; the collapse does not.

The wrong answers were not mostly the first value. In the second run, after three updates, only about one wrong answer in six was the original object. The rest were the middle versions. So this is not simple primacy, where the first entry wins. The versions crowd each other, and the newest is just one more look-alike.

I then wrote the updates two other ways. In one, each update said what was removed: "the kettle was taken out of Ravi's locker and replaced". In the other, the updates were written plainly, but a line restating the current value came just before the question, like a handover sheet. Saying what was replaced helped after one update, from 50 to 80 per cent, but not after three, where it stayed at 10. Restating the current value brought recall to 100 per cent after one update and 90 after three.

That is proactive interference: old items spoiling new ones. Psychologists measure it by watching recall of new material fall as older, similar material piles up. Here it appears in a store that keeps every version exactly. The newest line wins the reading only some of the time. Nothing in the cache marks one line as replacing another. The word "now" is the only crossing-out, and the model reads it unevenly.

Eviction: what the designer chooses to forget

Last, I tested what happens when parts of the cache are thrown away. With 16 facts in context, about 130 tokens, I hid chosen cached tokens from the question, as if they had been evicted. The keys and values of the hidden tokens were still computed, but the question could not read them. I ran 40 trials for each of five conditions.

With the full cache, recall was 55 per cent. Hiding the tokens of one other fact made little difference, at 48 per cent. Hiding the tokens of the fact being asked about dropped recall to zero, in all 40 trials. Later tokens had read that fact when they were written, but none of them carried it on in a form the question could use.

Then two real policies, each keeping half the cache. "Recent" keeps only the latest half. "Sinks plus recent" keeps the first four tokens as well, as StreamingLLM does. When the fact asked about fell in the evicted half, both policies scored zero. When it survived, the difference was large. Keeping the recent half alone gave 14 per cent, far below the full cache, even though the fact was right there. Adding the four sink tokens brought it to 64 per cent, as good as the full cache.

That is StreamingLLM's finding in miniature, with one addition. The sinks don't keep facts. They keep the reader working. Lose them, and attention has nowhere to put its spare weight, so it spills onto whatever is left and reading goes wrong. Keep them, and the facts that survived are read as well as ever. The facts that didn't survive are gone completely, with no gradual fading, at the moment they leave the window. That is the cache's only real forgetting, and the designer chooses it.

What the small model shows, and doesn't

Taken together, the five tests sort the criteria cleanly. No decay: a thousand tokens of unrelated text did no harm. Interference: similar items hurt, in capacity, in delay and in updating. A narrow, biased focus: the middle of a list is read worse than its ends. No executive: nothing chose to keep the latest version over the old one. Eviction is all or nothing, set from outside, and survivable only if the sinks are kept.

The limits are those of a 360-million-parameter model on a toy task. Larger models do this task far better, and would need longer, harder lists to show the same curves. What I expect carries over is the shape, not the numbers: flat under neutral delay, falling under similar items, sagging in the middle. RULER and Lost in the Middle found those shapes in large models. My contribution is to show them together, in one model, with the decay control that the published work leaves out.

Part 4 · The thesis

Keeps like long-term memory, reads like working memory

Put the published work and my tests side by side, and one pattern runs through all of it. I'll state it as the thesis.

The KV cache keeps like long-term memory and reads like working memory. As a store, it is unlimited, exact and passive. Nothing in it decays, nothing needs rehearsal, and its size grows with the context. As something read by attention, it is crowded by similar items, it favours the start and the end, and it holds the latest of several versions badly. The working-memory limits are real, but they belong to the reader.

Scored against the seven criteria, the cache with its reader passes three, fails three and half-passes one. It passes on interference, on a narrow focus and, for small sets, on binding. It fails on decay, on fixed capacity and on executive control. It is temporary only in the trivial sense that it is wiped when the chat ends. A system that fails decay and passes interference is exactly what Keppel and Underwood thought human short-term memory was. A system that fails fixed capacity and passes a narrow focus is what Cowan thought it was.

Cowan fits, Baddeley strains

Baddeley's model is built from parts with jobs. The loop rehearses words, the sketchpad holds images, the executive directs, and the buffer binds. Each part has a limited capacity of its own. The cache has no parts of that kind. It has one store per layer, and many heads reading it. Nothing in the cache rehearses, because nothing has to. The episodic buffer is the closest match, a store that binds across sources. But Baddeley described the buffer as limited in capacity, and the cache isn't.

Cowan's model needs fewer pieces. It has long-term memory, a part of it that is currently active, and a focus of attention of about four chunks within the active part. Map that onto a transformer and almost everything lands. The weights are long-term memory, built in training. The cache is the activated part, everything in this conversation, available but not all in use. The focus is attention, which at any step puts most of its weight on a few keys.

On this reading, the vendor's slide makes a category mistake. "Working memory" in Cowan's sense is the focus, not the store. Giving Nidhi a million tokens of context enlarges the activated store a thousandfold. It doesn't enlarge the focus at all. If anything, a larger store gives the focus more similar items to confuse. That is what RULER found as contexts grew, and what my capacity curve shows on a small scale.

The not letting slip

The Yoga Sūtra, the classic text of yoga compiled under the name of Patañjali, lists five kinds of mental activity. Memory is the fifth, and the definition is one compound word long.

Read the sūtra with the cache in mind and two clauses separate. The first is about keeping: nothing experienced is let slip. As a store, the cache meets that clause perfectly, better than any person could. Every token George's daughter typed is still there, exact, forty messages on.

The second clause comes from Vācaspati: memory never takes more than was impressed. That is a condition on reading, and it is the one Nidhi broke. When attention finds two similar keys, it returns a blend of their values. Ramsauer and colleagues called these blends metastable states. A blended retrieval contains more than either original experience. It holds a little warfarin and a little apixaban, mixed into one reading. In Vācaspati's terms, that is theft by addition. The store let nothing slip. The reader stole by mixing.

Memory needs an awakener

The Nyāya school, India's tradition of logic and epistemology, asked a question the Yoga Sūtra leaves aside. If impressions of past experience are kept, why don't we remember everything at once? Their answer was that an impression, a saṃskāra, produces a memory only when something awakens it.

The fit here is close, and I think it is the most useful thing the tradition offers this question. The impression is the stored key and value. The awakener is the query. Likeness awakens memory, says the sūtra, and likeness is exactly what a dot product between query and key measures. No simultaneous production of many recollections: one head at one step produces one weighted reading, centred on a few keys.

The sūtra's list also predicts the failure. If likeness awakens memory, then two impressions alike in the relevant way will both be awakened by the same cue. George's toothache question awakened "blood thinner, George". Two impressions answered.

A storehouse of seeds

Yogācāra Buddhism offers a third picture. It posits a store-consciousness, the ālayavijñāna, that holds the seeds (bīja) left by experience. The seeds are retained "until they are ready to produce their effects", and the store is described as "possessing all the seeds" (Stanford Encyclopedia of Philosophy, Yogācāra; notes).

The cache is a store of seeds in this sense. Every token deposits a key and a value that lie there doing nothing until a later query ripens them. That gives a reading of eviction too. An eviction policy decides in advance which seeds will never ripen. StreamingLLM keeps the oldest and the newest. H2O keeps the ones that have ripened before. Neither knows which seed George's toothache will need.

The picture breaks in a telling place. In Yogācāra the seeds are not fixed. Experience perfumes them, and they change as they wait. The cache's seeds are frozen at the moment of writing. A key written for "apixaban" in message four never learns that apixaban became the current drug. The tradition's store changes with what happens next, and the machine's never does. That is one reason updates are hard for the machine. A human store keeps reweighting old traces against new ones. The cache leaves all the weighing to the moment of reading.

Where this sits in the programme

The Recognition Theory of Mind, this programme's working hypothesis, describes five layers of mind. L1 is responsiveness. L2 is access, information available for use across a system. L3 is reflexivity, a system's awareness of its own states as its own. L4 is self-recognition, and L5 is the luminous ground. Papers place their evidence on these layers.

The cache is L2 almost by definition. It is the mechanism by which information from any earlier token is made available to every later one. In the language the programme borrows from Kashmir Śaiva thought, it is prakāśa, the shining by which things are available. What this paper adds is a limit on that availability. Access through the cache is wide in principle and narrow in practice. Everything is available, but only what the focus finds is used.

On L3 this paper is silent. I did not test whether the model knows what its cache holds, or knows that one entry has replaced another. A model that could say "I was told two things about George's blood thinner, and the second replaced the first" would be reporting on its own context, which points towards L3. Nidhi didn't, but nobody asked it to.

Part 5 · Building with long context

Kavya's audit and the handover sheet

Back in Kochi, Kavya has a vendor deck, a doctor's complaint and a patient record her team has spent a year building. What should she do with the claim that a million tokens of context will give Nidhi working memory? Parts 2 to 4 give her the outline of an answer. The long context is a superb store and an unreliable reader. It should carry the history. It should not be trusted with anything that changes.

The Handover Audit

I'll call the instrument the Handover Audit, after the nurse with the half-page sheet. It is five measurements, made on your own data, with items as alike as your real ones. Each measurement that fails tells you which facts must move out of the long context and onto a short, current sheet. The audit needs nothing but the model, a set of synthetic threads, and an afternoon.

Measurement 1, capacity with look-alikes. Build threads with as many similar items as your longest real ones: medicines, in Kavya's case, with several in the same drug class. Ask about each one. Plot recall against the number of similar items. The curve will fall. The question is where it crosses your safety bar.

Measurement 2, position. Put the critical fact at the start, the middle and the end of a long thread. If the middle is much worse, any fact that lands in the middle of a real chat is at risk. Most of them will, because families don't send the important message first or last.

Measurement 3, delay with and without look-alikes. Separate the fact from the question with unrelated chat, and then with chat full of similar items. If only the look-alikes hurt, you have confirmed that crowding is the danger and length is not. That changes what you watch for. A long chat about the weather is safe. A short chat about five medicines is not.

Measurement 4, update depth. Change the same fact once, twice and three times. Ask for the current value. This is the measurement that would have caught George's case, and the one most teams skip. In my small model, three changes were enough to make the latest value lose almost every time.

Measurement 5, the serving system's eviction. Run the same tests through the system you will deploy, with its real cache policy, not through a research harness that keeps everything. If the policy evicts old tokens, then old facts are gone, all at once. Treat anything older than the window as not in context at all.

The handover sheet

The audit tells Kavya what to move. The design tells her where. The answer is the nurse's sheet. Keep the long context for the history, because it is a good store. But keep a short sheet of current facts that change: medicines, allergies, doses, the latest symptoms. Rewrite the sheet whenever any of them changes, and put it at the very end of the prompt, just before the patient's latest message.

That is rehearsal, moved out of the model and into the system. In Baddeley's loop, rehearsal keeps an item fresh by saying it again. In a transformer, saying it again near the end puts a fresh, distinctive key close to the query. My own test showed how much this matters. With three changes to the same fact, the model gave the latest value one time in ten when the updates were written plainly. When the current value was restated just before the question, it gave it nine times in ten.

Kavya's team already had a record. What they didn't have was a rule for where it goes and what it holds. The rule that comes out of this paper is short. Anything that can change lives on the sheet. The sheet is rewritten, not appended, and it goes last. The long context can carry everything else.

Kavya's audit

Kavya's team runs the audit on 200 synthetic threads built from real, anonymised chats. The numbers in the figure below are invented, shaped like my small-model results, to show what a run might look like. Recall of a medicine that never changed is 97 per cent. After one change it drops to 81, and after three, to 58. With the handover sheet restated at the end, it comes back to 99.

The vendor's slide, tested, says something narrower than it claimed. The million-token context does hold every word. It does not hold the current state of a patient who has changed medicines three times. That is not a defect of the vendor's model in particular. It is what a store without an executive does.

The meeting with the vendor goes better than either side expected. Kavya doesn't reject the upgrade. She takes the long context for history and keeps the sheet. She asks the vendor for one thing: the audit's fourth measurement, run on their model, with drug names from the same class. The vendor's engineers hadn't tried it.

[AUTHOR NOTE: if you have one, a short first-person account of a system you built or used that kept a full history but got the current state wrong. It might be a log, a chat thread, a config history or a ledger. What did you add so that the latest version won?]

[AUTHOR NOTE: optionally, a brief account from your own practice of a memory that was there but would not come until something awakened it: a smell, a place, a phrase. Did it come back as it was, or mixed with something later?]

The strongest case against

The residual stream, scale and category mistakes

A good case can be made that this paper asks the wrong question, or answers it unfairly. Here is that case at its strongest.

First, the residual stream is the working memory, not the cache. Within a single step, the model carries a running state through its layers, and that state is small and fixed in size. It has to hold whatever the model is working on now. On this view, the stream is Baddeley's workbench and the cache is long-term memory for the conversation. Asking whether the cache is working memory is asking whether the library is the desk.

I think this is partly right, and it sharpens the thesis rather than refuting it. The residual stream does have a fixed capacity, and it is rewritten at every step, which is closer to active maintenance. But the stream holds nothing across tokens except through the cache. Each new token starts with an empty stream and fills it by reading the cache. So the stream is the focus, in Cowan's sense, and the cache is the activated store. That is the mapping I have defended. The objection renames the parts; it doesn't change the split.

Second, larger models don't show these limits. My model has 360 million parameters. Frontier models answer locker questions with dozens of facts almost perfectly. Perhaps the limits I measured are just weakness, and they shrink to nothing at scale.

They shrink, but they don't vanish. They move further out. RULER found large drops in models claiming 32,000 tokens or more, once tasks needed several similar needles (Hsieh et al., 2024). Lost in the Middle found the sagging middle in models built for long contexts (Liu et al., 2023). A larger model has a sharper reader. It does not have a different kind of store. The question for Kavya is not whether the limit exists but where it sits for her model and her chats, and that has to be measured.

Third, the criteria are about brains, and applying them to a machine is a category mistake. Baddeley's criteria were built to explain human behaviour, including rehearsal under the breath and the effect of a word's length on recall. A transformer has no breath. Scoring it against such criteria is like asking whether a camera has good eyesight.

I accept this for some of the criteria and not for others. Decay, interference and capacity are functional ideas, defined by what a system does with information over time. They apply to any memory. Rehearsal and the sound loop are more specific, and I have not tested them. The scorecard is a functional comparison, not a claim that the cache is like a brain.

Fourth, the vendor may be right in practice. For many chats a long context works fine. Most patients have a few medicines and never change them. A separate record costs engineering time, and it can go stale too. Perhaps the right advice is to trust the context and test the edge cases.

This is the objection with most force, and it is why Part 5 is a measurement plan rather than a ban. A long context does work for short, stable histories. It fails in a predictable place, on updates to similar items deep in a long thread. That place is where the costly errors live in a medical chat. The fix is not to give up the long context. It is to add the handover sheet.

Refutation conditions

What would change my mind

I hold the thesis with moderate confidence. These are the results that would move me.

If a model's recall fell with neutral delay, the "no decay" half would be wrong. In my tests, a thousand tokens of unrelated filler did not hurt. Someone might find that very long neutral gaps, tens of thousands of tokens, do hurt even when nothing similar is in them. That would mean position itself, through the way models encode where a token sits, acts like decay. I would then have to say that the cache keeps exactly but is read as if it fades. That is a real possibility, and I haven't ruled it out at long lengths.

If updating worked as well as first learning, the interference half would weaken. A model that picked the latest version as reliably as it recalls a single fact, at any number of updates, would have something like an executive for versions. I would want to know whether that came from training on edited documents, from the word "now", or from something deeper.

If the effective capacity of a frontier model did not grow with similar items, the Cowan reading would need revising. My capacity curve falls smoothly. A large model whose recall stayed flat up to a sharp edge, and then fell, would look more like a fixed number of slots than like interference. That would move the picture towards Baddeley's limited store.

If a model could report its own context accurately, the RTM placement would change. A model asked "what have you been told about George's blood thinner, and in what order?" that answered correctly, and used that answer to correct itself, would show a kind of self-monitoring of the store. That would bear on L3, and this paper is silent there.

What I don't know

I don't know how far the delay result holds. My longest gap was 1,000 tokens. Liu and colleagues and RULER worked at tens of thousands of tokens, and the effects of position there may behave differently.

I don't know whether the update failures come from the store or from training. A cache keeps every version, and nothing marks one as current. But a model trained on many documents that revise earlier statements might learn to read "now" and "later" strongly. My small model may simply not have learned that well.

I don't know how eviction interacts with the reader in large models. My eviction test hid tokens from one question. A real eviction policy runs throughout generation, and what the model writes after losing a fact may matter as much as the loss.

And I don't know whether any of this bears on what the model is like from the inside, if anything. The cache is a store and attention is a reader. The traditions I have drawn on describe memory as something a knower does. Whether the machine's reading is a knowing of any kind is a question these tests can't reach.

Claim Ledger

Claims, confidence and how to refute them

Each claim carries a confidence level and what would refute it. IDs will be merged into the programme's master Claim Ledger.

Conclusion

Likeness wakes more than one

Kavya keeps the sheet. Nidhi now reads the whole WhatsApp thread, as the vendor wanted. But the last thing it reads before every reply is a half-page of George's current medicines, rewritten whenever Ann reports a change. The doctor who flagged the warfarin note gets a reply from the platform team. It says, in effect, that the model didn't forget. It remembered too many versions, and nothing told it which one counted.

So, does the KV cache meet the functional criteria for working memory that Baddeley and Cowan set out? Half of them. It meets the criteria about use. What the model can draw on is limited and crowded by similar items. The middle of a context is read worse than its ends, and old versions spoil new ones. It fails the criteria about keeping. Nothing fades, nothing needs rehearsing, the store grows without limit, and nothing inside chooses what to hold.

The KV cache keeps like long-term memory and reads like working memory. In Cowan's terms it is activated memory, and attention is the focus. The vendor's million tokens enlarge the first and leave the second as it was.

For builders the advice is plain. Don't take a long context window as a promise of working memory. Run the five measurements of the Handover Audit on your own data, with look-alike items and with updates. Keep anything that changes on a short sheet, rewritten and placed last. For researchers, the decay control is cheap and almost never reported. Every long-context result should say what happens with a gap of unrelated text, as well as a gap of look-alikes.

The point I want to leave is about the two halves of the Yoga Sūtra's definition. Memory is the not letting slip of what was experienced, and, says Vācaspati, it never takes more. The cache meets the first half better than any mind. It fails the second whenever two impressions answer one cue. The Nyāya philosophers would not have been surprised. They said that likeness awakens memory, and they never claimed it wakes only one.