Invented scenario Everything in this story is invented. It is here to give intuition, never evidence.

Priya Nair has two numbers on the whiteboard at Tunga Labs in Mysuru, and the board meeting is in an hour. The first is 0.71. The second is 0.78. They are bits per byte, the average cost of encoding each byte of a test collection of farm advisory bulletins, written in Kannada and English, under two language models. Lower is better. Model Hamsa scored 0.71. Model Garuda scored 0.78.

The bulletins matter. Tunga is building a telephone helpline for farmers in the Cauvery basin, and one of the two models will sit behind it. The model will answer questions about sowing dates, pest outbreaks and which subsidy forms to fill in.

Dr Sudhir Kamath, the board member who funds the helpline, has read that compression is intelligence. He has a slide ready. It says that Hamsa understands farming better, because it compresses farming text better, and it recommends Hamsa.

Priya's colleague Fatima Sheikh is less sure. She has spent the morning checking where the test bulletins came from. Most of them were posted on a state agriculture website in 2024. Hamsa's training data included a large collection of web pages gathered in 2025. Garuda's did not. Fatima ran one more test before lunch. She gave both models a bulletin issued last month about a new leaf pest that no older text mentions, and asked each one five plain questions about it. Garuda answered four correctly. Hamsa answered one.

"Hamsa has seen the answer key," Fatima says. "It is not reading the bulletins. It is remembering them."

"Then why does it compress them so well?" Sudhir asks.

That question is the subject of this paper. Compression and understanding are linked by real mathematics, and the link has been taken further than many people realise. A system that predicts text well can compress it, and a system that compresses well must have found regularities in what it compresses. Some of those regularities are the ones we mean when we say someone understands. Others are not.

Here is the answer in plain words, before the argument. Compression is an excellent measure of one ingredient of understanding, finding the regularities that generate data, and a poor definition of the whole. It is silent on three things that understanding needs. It does not say what the symbols are about, which is the problem of reference. It does not say which regularities matter, which is the problem of relevance. And it cannot tell a rule that was grasped from a text that was stored, unless the size of the stored material is charged as part of the cost.

The consequence for Priya is practical. A compression score on a test set is only meaningful if the test set is new to the model and the model's own size is counted. The consequence for mind science is sharper. Treating understanding as compression gets the economy of thought right and leaves out what thought is about.

[AUTHOR NOTE: a short first-person account of a time a benchmark score looked impressive until you discovered the test data had leaked into training, if you have one.]

The rest of the paper earns those claims. It starts with the exact identity between prediction and compression, moves to the shortest-program view of complexity and its limits, and then builds an instrument that separates grasped rules from stored text. After that it asks where compression goes silent, and what two Indian traditions of language, the grammarians and Bhartṛhari, add to the question.

Start with a game. A friend thinks of an English sentence and you guess it one letter at a time. Each time you guess wrong, she says "no" and you guess again. Claude Shannon played a version of this game in 1951 to estimate how predictable English is (Shannon 1951). People rarely need many guesses, because English is full of patterns. After "q" comes "u". After "the cat sat on the" comes "mat" more often than "theorem".

That game contains the whole idea of this part. If you can predict the next letter well, you can describe the sentence cheaply. You only need to record where your guesses went wrong.

The identity, in three steps#

The link between prediction and compression is not an analogy. It is an exact result, and it takes three steps.

First, assign probabilities. A predictor gives a probability to each possible next symbol. If it says the next letter is "u" with probability 0.9, it is fairly sure. If it gives every letter of a 27-symbol alphabet the same chance, it is not sure at all.

Second, convert probability to length. Shannon showed in 1948 that the best possible code gives a symbol with probability p a code of length −log₂ p bits (Shannon 1948). A symbol with probability one half costs one bit. A symbol with probability one quarter costs two bits. A symbol with probability 0.9 costs about 0.15 bits. Surprise is expensive. Confirmation is cheap.

Third, add up. The length of the whole message is the sum of those costs over every symbol. A predictor that gives high probability to what actually happens produces a short code. A predictor that is often surprised produces a long one.

The step that makes this practical is arithmetic coding, a method developed in the 1970s that turns any sequence of predictions into an actual file within a couple of bits of the ideal total. So the size of a compressed file and the total surprise of a predictor are the same number, measured in the same units. A language model's training loss, the average of −log p over the training text, is a compression rate. When engineers report "bits per byte", they are reporting how small the model could make the text.

Reading a loss curve as a compression report#

This identity changes how you can read the training curves engineers publish. A language model's loss is usually given in a unit called nats per token, which uses natural logarithms and counts tokens, the word pieces a model reads. Convert nats to bits by dividing by about 0.69. Convert tokens to bytes by dividing by the average number of bytes per token, which for English text is often around four.

A model with a loss of 2.0 nats per token on English text therefore spends about 2.9 bits per token, or roughly 0.7 bits per byte. Ordinary zip compression is far weaker: in my run, it spent 2.98 bits per byte on The Jungle Book. The model is three or four times better at predicting the next byte.

That number is a true measure of prediction, and it says nothing yet about where the prediction came from. A model can predict well because it has learned how English works, because it has learned facts about the world, or because it has seen the very text it is being tested on. The loss curve cannot tell these apart. That is the whole difficulty of this paper in one sentence.

Why this matters for the argument#

The identity has a consequence that is easy to miss. Any claim of the form "model A compresses this text better than model B" is the same claim as "model A predicted this text better than model B". Nothing more. The question of understanding therefore becomes a question about prediction. Does predicting well require understanding, and if so, which kind?

The answer depends on what is being predicted and on what the predictor had seen before. The clerk predicts familiar letters well because she has seen thousands like them. Hamsa predicts the bulletins well, according to Fatima, because it has seen the bulletins themselves. Prediction rewards any source of knowledge about what comes next, including a copy of the answer.

The prize's rule about counting the program will matter again in Part 3. It is the difference between a fair test and the trap Hamsa fell into.

Shannon's measure needs a probability model. It tells you how surprising a message is to a given predictor. In the 1960s three people independently asked a deeper question: how complex is a single object, with no predictor given in advance?

Ray Solomonoff in 1964, Andrey Kolmogorov in 1965 and Gregory Chaitin in 1966 arrived at the same answer (Solomonoff 1964; Kolmogorov 1968; Chaitin 1966). The complexity of an object is the length of the shortest program that prints it and then stops. This is now called Kolmogorov complexity, written K(x).

A string of a million zeros has low complexity. A short program prints it: "repeat 0 a million times". A string of a million coin tosses has high complexity. Almost certainly, no program much shorter than the string itself prints it. The string has no pattern to exploit.

The invariance theorem and its constant#

An obvious worry is that "shortest program" depends on the programming language. A string might have a short program in Python and a long one in some obscure language. Kolmogorov and Solomonoff showed that this matters less than you would think. Any two general-purpose languages can simulate each other with a fixed-size translator. So the complexity of a string in one language differs from its complexity in the other by at most a constant, the size of the translator. That constant does not depend on the string.

This is the invariance theorem. It makes K a property of the string, up to an additive constant. The constant can be large, though, and for short strings it can swamp the measurement. Kolmogorov complexity is objective for long objects and arbitrary for short ones. Most of the things we ask about understanding, a sentence, a proof, a face, are short in this sense.

A small example shows how much the constant can hide. Take the string "Ramanujan was born in Erode in 1887". In a general programming language its shortest program is probably the string itself, plus a print command. In a language built for Indian history, with a dictionary of famous people and places built in, it might be a dozen characters. Which is its "true" complexity? The invariance theorem says only that the two answers differ by at most the size of the history dictionary. For a sentence, that bound is useless, because the dictionary is far longer than the sentence.

This is why the compression view works best on large bodies of text and worst on single acts of understanding. The Hutter Prize uses a gigabyte for a reason. At that scale the choice of language is a rounding error. At the scale of one sentence a student understands, it is everything.

Why K cannot be computed#

The second catch is worse. There is no general procedure that takes a string and returns its Kolmogorov complexity. The proof echoes an old puzzle, the Berry paradox: "the smallest positive integer not definable in under sixty letters". That phrase defines a number in under sixty letters, which contradicts what it says.

The same trick defeats any program that claims to compute K. If such a program existed, you could write a short program that searches for the first string whose complexity exceeds some huge number, and prints it. That short program would then be a short description of a string that has no short description. So the program for computing K cannot exist.

Every practical compressor is therefore an approximation from above. Gzip, lzma and the cmix family used in the Hutter Prize all find some patterns and miss others. Their output size is an upper bound on K, never the value itself.

A counterexample from π#

The gap between true complexity and practical compression can be enormous. I ran a direct test. I took the first 100,000 decimal digits of π, generated 100,000 digits from a seeded random number generator, and gave each to three standard compressors.

Published evidence The results were close to indistinguishable. Lzma, the best of the three on these files, needed 3.50 bits per digit for π and 3.49 bits per digit for the random digits. The theoretical floor for ten equally likely digits is log₂ 10, about 3.32 bits. Neither file came close to beating that floor, so to these compressors the digits of π look exactly as random as noise.

Yet π has very low Kolmogorov complexity. A 63-character Python line, using a standard mathematics library, prints those 100,000 digits. The true complexity of the π file is tiny. The random file's is not.

This is the counterexample the compression view of understanding has to face. A system that understands what π is could describe the file in one line. A practical compressor, which has no idea what π is, sees noise. If understanding were simply practical compression, a pocket calculator would understand π better than lzma does. That is true, and it is the point. Practical compression measures only the regularities the compressor already knows how to find.

Solomonoff's bet on the shortest program#

Solomonoff did not invent this measure for its own sake. He wanted a theory of prediction. His idea was to predict the next symbol of a sequence by considering every program that could have produced the sequence so far, and weighting each by 2 raised to minus its length. Short programs get most of the weight. This is a precise version of Ockham's razor: prefer simple explanations, but keep the complex ones in reserve.

Solomonoff later proved that, on any sequence produced by a computable process, the total expected squared error of this predictor is bounded (Solomonoff 1978). It learns any computable pattern, given enough data. The catch is the same as before. It requires running every program, so it cannot be built. The site already carries a fuller account of Solomonoff's induction for readers who want it (Solomonoff's inductive inference), and of the three complexities (how many bits make an object simple).

Marcus Hutter later built on Solomonoff to define an ideal agent, AIXI, that acts in an unknown world by weighting every computable model of it by simplicity. Shane Legg and Hutter turned the same idea into a formal definition of intelligence in 2007: an agent's ability to achieve goals across all computable environments, with simple environments weighted most (Legg and Hutter 2007). This is the strongest form of the claim that intelligence is compression. It is mathematically exact and practically unbuildable.

The trouble with Hamsa is that it got credit for knowledge it was given. The cure has a name. It was worked out by the Finnish information theorist Jorma Rissanen in 1978, and it is called the minimum description length principle (Rissanen 1978).

The two-part code#

Rissanen's idea is that a fair description of data has two parts. The first part describes a model: a rule, a theory, a set of parameters. The second part describes the data given that model: everything the model failed to predict. The total cost is the sum. The best model is the one that makes the total smallest.

The two parts pull against each other. A very simple model is cheap to state but leaves a lot of data unexplained, so the second part is long. A very complex model explains every wiggle in the data, so the second part is short, but the model itself is long to state. The minimum sits in between.

Here is the second analogy. A two-part code is like flying with a strict baggage allowance where you pay for the suitcase and for its contents. A big suitcase full of specialised tools lets you handle any problem on arrival, but you pay to carry it. A tiny bag is cheap but leaves you buying things at the other end. The cheapest trip packs the tools you will actually use.

A minimal worked case#

Invented scenario I ran a small test that shows the principle working. I generated 60 points from a cubic curve and added random noise with a standard deviation of 0.15. Then I fitted polynomials of every degree from 0 to 10 and computed, for each, the two parts of the code. The data part uses the standard formula for the cost of Gaussian errors. The model part uses Rissanen's rule of thumb: about half of log₂ n bits for each fitted number, where n is the number of data points.

The fit to the training points improved with every extra degree, as it always does. The two-part code did not. It fell steeply from degree 0 to degree 3 and then rose gently. The minimum fell at degree 3, the true degree of the curve that generated the data. On fresh points from the same curve, the degree-3 fit had a mean squared error of 0.025. The constant fit had 0.49. Degrees above 3 did no better on fresh points, and some did worse.

That is the good news for the compression view. When you charge for the theory, the shortest description finds the generating rule, and the generating rule generalises. Compression with the model charged is a theory of generalisation as well as of storage.

The memorisation trap, measured#

Now the bad news. Most published compression numbers for language models do not charge for the model. They report the cost of the test text alone, as if the model were free. That is the setting in which Hamsa wins.

To see how much this matters, I built the second instrument of this paper. It measures conditional compression: how many extra bits it costs to encode a piece of text, given that the compressor has already seen some other text. I used Rudyard Kipling's The Jungle Book, the plain text from Project Gutenberg (Kipling, Project Gutenberg). The first half of the book plays the part of "training" material. Lzma, a standard compressor, plays the role of the model.

The instrument has four cases, and I report what it actually returned.

• Seen text. A 10,000-byte passage taken from inside the first half. Given the first half, it cost 0.04 bits per byte. The compressor simply pointed back to where it had seen the passage.
• New text, same author. A 10,000-byte passage from the second half, never seen. Given the first half, it cost 2.61 bits per byte. On its own, with nothing seen first, the same passage cost 3.50.
• Negative control: shuffled words. The same book with its words in random order. Given the first half, it cost 2.99 bits per byte, worse than the new passage, because word order carries structure the compressor had learned to use.
• Unrelated: digits of π. Ten thousand digits of π, given the first half of the book. It cost 3.53 bits per digit, no help at all from Kipling.

The seen passage is sixty-five times cheaper than the unseen one. A compressor that has seen the test text will report a near-perfect score that says nothing about its grasp of the text. That is exactly what Fatima suspected about Hamsa. A difference of 0.71 against 0.78 is small next to the gap between 0.04 and 2.61.

The instrument: a contamination check by conditional compression#

The instrument is simple enough to run on any pair of model and test set where you can measure bits. Here it is in full.

What it does. It compares the cost of encoding a test set under the model with the cost of encoding fresh text of the same kind that the model cannot have seen. A large gap in favour of the test set is a warning of contamination.

Inputs and output. Inputs: the model, the test set, and a "fresh" set written after the model's training cut-off by the same kind of author. Output: bits per byte on each set, and their ratio.

Assumptions. The fresh set must match the test set in topic, length and style. If the fresh set is harder for other reasons, the check will raise false alarms.

Positive case. A model trained on the test set. The ratio of test cost to fresh cost is far below 1. In my lzma run it was 0.04 against 2.61, a ratio of about 0.015.

Negative case. A model that has never seen either set. The ratio is close to 1. In my run, two unseen passages from the second half of the book cost within a few per cent of each other.

Scaled case. A real language model and a real benchmark. Here the ratio will rarely be as extreme as lzma's, because a large model stores text loosely, not verbatim. I have not run this case, and I flag it as the step that needs doing on real models before the instrument is trusted.

Back to Tunga Labs#

Priya runs the check that afternoon. She asks the state agriculture office for twenty bulletins issued in the last two months, after both models' training data were gathered. On those bulletins, Garuda scores 0.80 bits per byte, close to its 0.78 on the test set. Hamsa scores 0.93, much worse than its 0.71.

The board slide changes. Hamsa's advantage on the original test set was memory, and it disappears on new text. On the bulletins neither model could have seen, Garuda compresses better and answers more questions correctly. Both facts point the same way. Sudhir's principle was right; his test was unfair. Compression measures understanding only on data the model could not have stored.

The compression thesis looked like a philosopher's idea until large language models made it concrete. Two results stand out, one in favour and one that complicates the story.

The image result shows that what a large model learns from text reaches beyond text. Some of its predictive skill is general: it has learned to exploit patterns in any byte stream. That is evidence that prediction training produces general regularity-finding, which is the ingredient of understanding I am willing to grant compression.

Compression tracks benchmark scores#

A second result is closer to Sudhir's slide. In 2024 Yuzhen Huang and colleagues measured the compression rate of 31 open language models on fresh text and compared it with the same models' average scores on 12 benchmarks of knowledge, coding and mathematical reasoning (Huang et al. 2024). They reported that the two lined up almost linearly. Models that compressed fresh text better scored better on the benchmarks.

The detail that matters is the word "fresh". Huang's team built their compression test sets from recently collected text, such as web pages gathered in September and October 2023, and argued that such sets can be refreshed to avoid test contamination. That is the same move Priya made. On fresh text, compression is a good summary of capability. The empirical link between compression and capability holds when the test is new to the model, and it was built to hold only then.

Grokking: when compression and generalisation arrive together#

A third line of work shows the link inside a single training run. In 2022 Alethea Power and colleagues at OpenAI trained small networks on arithmetic tables, such as addition modulo a prime number (Power et al. 2022). The networks first memorised the training examples, reaching perfect training accuracy while still failing on held-out examples. Then, long after, and with a penalty on large weights, they suddenly generalised. Power's team called this grokking.

In 2023 Neel Nanda and colleagues reverse-engineered one such network (Nanda et al. 2023). After grokking, it computed modular addition with a compact algorithm based on rotations, using a handful of frequencies. Before grokking, it had relied on a large memorised lookup. Generalisation arrived when the network replaced a long description with a short one.

This is the cleanest evidence for the compression view that I know. A network that found a shorter internal description of its task began to generalise at the same time. It matches the polynomial case from Part 3, at the level of a network's insides.

What the grokking result does not show#

The grokking result is clean because the task is closed. Modular addition has one right answer for every input, and a short algorithm computes it. Nothing about the meaning of "addition" is left over once the algorithm is found. In that setting, finding the shortest description and understanding the task coincide, because there is nothing more to understand.

Language is not like that. A short description of how farm bulletins are written does not settle what a particular bulletin is asking a farmer to do. The meaning of a sentence depends on a world beyond the text. Grokking shows that compression and generalisation can arrive together. It shows this in a world where understanding is exhausted by the rule. The harder question is what happens when the rule is only part of what there is to grasp, and that is where the next part goes.

The objection from skill acquisition#

A sharper challenge came from François Chollet in 2019 (Chollet 2019). Chollet argued that intelligence should be measured by how efficiently a system acquires new skills, given its prior knowledge and experience, not by how well it performs any fixed task. He built the Abstraction and Reasoning Corpus, a set of visual puzzles each solvable from a handful of examples, to test exactly that.

Chollet's view does not contradict the compression thesis. It refines it. A system that acquires skills efficiently finds short descriptions of new tasks from little data. That is compression of the new task, measured relative to what the system already knew. The compressed file in Chollet's sense is the new skill, and the cost includes the prior. Chollet's measure is compression with the prior charged, which is Rissanen's principle applied to learning.

So far compression has done well. With the model charged and the test kept fresh, short descriptions find generating rules, and generating rules generalise. That is a large part of what we mean by understanding. This part asks what is left over.

The reference gap#

Kolmogorov complexity is defined over strings of symbols. It does not know what the symbols stand for. Two strings with the same structure have nearly the same complexity, whatever they are about.

This is the old problem of symbol grounding, named by Stevan Harnad in 1990 (Harnad 1990). Harnad asked how the symbols in a system could get their meaning from the system itself, rather than from the heads of the people who read its output. His example was a person trying to learn Chinese from a Chinese–Chinese dictionary alone. Every definition sends you to more symbols. The circle never touches the world. John Searle's Chinese room, from 1980, makes a related point about a person following rules for symbols they do not understand (Searle 1980).

Emily Bender and Alexander Koller brought the problem to language models in 2020 with a thought experiment about an octopus (Bender and Koller 2020). The octopus taps a telegraph cable between two islands and learns, from the patterns of messages alone, to imitate one of the islanders. It does well at small talk. Then one islander is attacked by a bear and asks for help making a weapon from sticks. The octopus has never seen a bear or a stick, only words about them. Bender and Koller argued that a system trained only on form cannot, in principle, learn meaning, if meaning is the relation between form and something outside it.

Here is the third analogy. A compressed file is like a very good map with no legend. It preserves every relation between places: which roads join, which towns are near. It does not say which mark is a town and which is a temple. If you already know the territory, you can read the map. If you do not, you have a perfect picture of relations among unknown things.

The relevance gap#

The second gap is about which regularities count. Kolmogorov complexity charges every bit equally. The coffee stain on the forger's record and the boundary line cost the same to describe. Understanding is selective. It cares about the regularities that matter for some purpose and ignores the rest.

Murray Gell-Mann and Seth Lloyd tried to repair this in 1996 with a measure they called effective complexity (Gell-Mann and Lloyd 1996). The idea is to split the description of an object into two parts: its regularities and its random details. Effective complexity is the length of the description of the regularities alone. A random string has low effective complexity, because it has no regularities. A string of all zeros has low effective complexity too. Interesting objects, living things and languages and economies, sit in between.

The repair works only if you can say which features are regularities and which are noise. Gell-Mann and Lloyd were frank that this depends on a judgement of what is important, which comes from outside the mathematics. Effective complexity moves the question of relevance into the choice of what counts as a regularity; it does not answer it.

The use gap#

The third gap concerns what understanding does. When we say a mechanic understands an engine, we mean that she can predict what it will do, and also what would happen if she changed something. Which bolt, if loosened, stops the leak? Which part, if replaced, restores the power?

Prediction of passive data does not require this. A compressor of engine sounds learns which sounds follow which. It does not learn what happens when you intervene, because the data contains no interventions, only observations. Questions about intervention and counterfactuals need a model of causes, and a causal model can be harder to learn from passive data than a merely predictive one.

Three gaps, one diagnosis#

The three gaps share a cause. Compression is a property of a string relative to a description language. Understanding is a relation between a mind, a string and a world. It involves knowing what the string is about, which parts matter, and what would follow from acting on it. Compression is necessary for understanding at scale, because no finite mind can store the world uncompressed, and it is not sufficient, because the same compression can serve a mind that understands and one that does not.

What closing the gaps would look like#

It helps to say what evidence would show a system had crossed each gap. For reference, the test is action on a new referent: something the system has only read about, met for the first time in the world, handled correctly. The new-pest bulletin is a small version of this. For relevance, the test is selective attention under a stated purpose: given a goal, does the system ignore regularities that do not serve it? For use, the test is intervention: does the system predict what happens when something is changed, as well as what usually follows?

None of these tests is a compression test, and none could be turned into one without adding something compression lacks: a world to act in, a purpose to serve or a way to intervene. That is the practical form of my thesis. Compression scores belong in every evaluation of understanding. They should never be the only score.

Indian traditions of language thought hard about economy of description long before anyone wrote a compressor. Two of them bear directly on this question, and they pull in different directions.

Pāṇini's economy#

Pāṇini's Aṣṭādhyāyī, composed around the fourth century BCE, describes the whole of Sanskrit word formation in about 4,000 short rules, called sūtras (सूत्र). It is one of the most compressed scientific texts ever written. Its techniques would look familiar to a compression engineer. A word stated once in an early rule is understood to carry over into the following rules until another word replaces it, a device called anuvṛtti (अनुवृत्ति), so it never has to be repeated. Short codes, such as markers attached to lists of sounds, name whole classes in two or three syllables. A set of rules about how to read rules, the metarules or paribhāṣā (परिभाषा), settles conflicts without spelling out every case.

The later grammarians made economy an explicit value. A famous maxim, collected in Nāgeśa's eighteenth-century Paribhāṣenduśekhara, says: ardhamātrā-lāghavena putrotsavaṃ manyante vaiyākaraṇāḥ (Nāgeśa, Paribhāṣenduśekhara). In plain English: "grammarians celebrate the saving of half a mora as they would the birth of a son." A mora, or mātrā (मात्रा), is the length of a short vowel. Saving half of one is about as small a saving as Sanskrit allows.

Textual reading That is a theory of grammar as compression. A good grammar is the shortest set of rules that generates all and only the correct forms. Rissanen's two-part code would recognise it. The rules are the model; the exceptions listed separately are the data the model fails to predict.

Yet the same collection limits the principle. Another maxim, in the GRETIL text a few entries earlier, says that arguments about brevity and length are not entertained between synonymous words: paryāya-śabdānāṃ lāghava-gaurava-carcā nādriyate (Nāgeśa, Paribhāṣenduśekhara). Economy decides between rules that do the same work. It does not decide which work the rules should do. That is decided by the language as it is actually spoken. Pāṇini's tradition prized brevity as the test of a grammar only after usage had fixed what the grammar must generate.

[AUTHOR NOTE: if you studied Sanskrit grammar, or heard the half-mora saying from a teacher, a two-sentence account of where you first met it would help here.]

Bhartṛhari's flash#

Bhartṛhari, the fifth-century philosopher of language, adds something the compression view lacks. In the second book of his Vākyapadīya he asks how we understand a sentence. His answer, at verse 2.143, is that after the meanings of the separate words are grasped, a different cognition arises, and that cognition is what is called the meaning of the sentence (David 2021). The Sanskrit reads: vicchedagrahaṇe 'rthānāṃ pratibhānyaiva jāyate, vākyārtha iti tām āhuḥ padārthair upapāditam. The cognition is called pratibhā (प्रतिभा), often translated "insight" or "flash of understanding".

Textual reading In the next verse, 2.144, Bhartṛhari says that this pratibhā cannot be described to others as "this is that"; it is established in each person's own experience (David 2021). Hugo David, in a close study of every passage on pratibhā, reads it as an immediate, practical cognition, informed by language, that is productive rather than a copy of anything. Sanjit Das, writing in 2022, reads it as the state by which competent speakers understand new sentences without conscious reasoning and act on them (Das 2022).

Both readings bear on this paper. A sentence's meaning, for Bhartṛhari, is not the sum of its word meanings. It is a single grasp that arises from them and points towards action. Pratibhā names exactly the part of understanding that compression leaves out: the grasp of what a whole utterance is for, which shows itself in what the hearer does next.

What this means for the Recognition Theory of Mind#

The Recognition Theory of Mind, the working hypothesis of this programme, names five layers. This paper bears on the third and fourth. The third layer is reflective grasp (vimarśa, विमर्श), awareness taking hold of what it shows. The fourth is recognition (pratyabhijñā, प्रत्यभिज्ञा), knowing something as what it is.

Compression has a natural home near the fourth layer. Every saving a compressor makes is a recognition of recurrence: this stretch of text is that stretch again, this pattern is that pattern. The Lempel–Ziv family of compressors works by literally pointing back and saying "same as there". In that limited sense, compression is mechanised recognition of sameness.

Open hypothesis RTM's recognition is more than that. Recognition in the RTM sense includes knowing what the recognised thing is, and for the higher layers, knowing oneself as the one who knows. The compressor's "same as there" has no "what" and no "who". It recognises a recurrence without recognising anything. So compression supplies a necessary condition for the fourth layer, the detection of sameness, and none of the sufficient ones.

That gives RTM a testable prediction. If the fourth layer requires grasp of what is recognised, then systems with equal compression should differ on tasks that require knowing what a recognised pattern refers to, such as acting correctly on it in a new context. The farmer's question about a new pest is such a task. Hamsa and Garuda, at matched fresh-text compression, should still differ on it if the extra ingredient is real. If they never differ, the extra ingredient is idle, and RTM's fourth layer reduces to compression.

Here is the strongest case against my thesis, put as its best defenders would put it.

The first objection says the gaps I describe are artefacts of compressing the wrong data. Text alone is ungrounded, granted. But a system that compressed all of a robot's sensory stream, including the effects of its own actions, would have to model the world those signals come from. Reference would be learned because the data would include the relation between words and things. Relevance would follow because regularities that predict reward and survival would dominate the data. Use would follow because the data would include interventions. On this view, compression of the right stream is understanding, and my gaps are gaps in the stream, not in the principle.

This is the objection I take most seriously, and it narrows my claim. I accept that compressing a grounded, interactive stream would close much of the reference and use gaps. The relevance gap remains. A compressor of everything still has to choose a description language, and different languages make different regularities short. Which regularities count as the important ones is then decided by that choice. The objection moves the choice of relevance into the design of the system. It does not remove it.

The second objection comes from cognitive science. Nick Chater and Paul Vitányi argued in 2003 that a simplicity principle, close to Kolmogorov complexity, explains a wide range of human perception and learning (Chater and Vitányi 2003). When people see a pattern, they tend to see the simplest one. If human understanding itself works by finding short descriptions, then the gaps I describe belong to human understanding too, and the distinction I draw between compression and understanding is a distinction without a difference.

I think the simplicity principle is largely right about perception, and I take it as support for the half of my thesis that grants compression a central role. It does not touch the reference gap, because human perception is grounded in a body acting in the world. The simplicity principle explains which interpretation a grounded perceiver prefers. It does not explain how the perceiver came to have interpretations of anything at all.

The third objection is technical. It says my memorisation examples are unfair, because the proper measure, a two-part code, already charges for memorised material. Hamsa only looks good because we ignored its size. Count it, and the problem disappears.

That is correct, and it is my point. The proper measure solves the memorisation problem, but almost no one reports it for large models. Delétang's table shows why: on a gigabyte of text, the adjusted rate for Chinchilla 70B is over 14,000 per cent. For practical models, the two-part code says "this model is too big to count as a compressor of this data". That is true and unhelpful. The working measure used in practice is the one-part rate on fresh text, which is why freshness carries all the weight.

The fourth objection is empirical and, to me, the most interesting. It says that systems trained only to predict sequences do build models of what the sequences are about. In 2022 Kenneth Li and colleagues trained a small transformer to predict legal moves in the board game Othello, from move lists alone (Li et al. 2023). The network was never shown a board. Yet probes found an internal representation of the board state, and changing that representation changed the moves the network predicted, as if the board had changed. In 2023 Neel Nanda, Andrew Lee and Martin Wattenberg showed that the board could be read with simple linear probes, once you asked about "my colour" and "the opponent's colour" rather than black and white (Nanda, Lee and Wattenberg 2023).

That is a striking result, and it does narrow the reference gap. A sequence predictor built a model of the hidden state that generated its data. But notice what made it possible. The game is fully determined by the moves. Nothing about Othello lies outside the move list. A predictor can recover the world behind its data when the data determine that world, and ordinary language does not determine the world it talks about. The Othello result shows what compression can do in a closed world. It does not show what it can do when the world is open.

Taken together, the four objections leave my thesis smaller and better defined. The memorisation problem is solved in principle by the two-part code, and in practice by fresh tests. The grounding problem shrinks when the data determine the world, as in Othello, and when the stream includes the system's own actions. The relevance problem survives every objection, because some choice of description language, or of purpose, has to come from outside the data. What remains is a claim I am willing to defend: compression is the best single measure we have of regularity-finding, and understanding is regularity-finding plus reference, relevance and use.

Concrete observations would weaken the thesis, and I list them so they can be checked.

• If systems matched for compression on fresh text were shown to perform the same on tasks requiring action on what text refers to, across many model families, the extra ingredient I claim would be idle.
• If a system trained only to compress text, with no grounded input, were shown to answer questions about the physical consequences of new actions as well as a system with grounded input, the reference and use gaps would be much smaller than I claim.
• If a principled, task-free way of separating regularities from noise were found that matched human judgements of importance, the relevance gap would close and effective complexity would become a usable measure of understanding.
• If the contamination check from Part 3 gave no warning on test sets known to be in a model's training data, the instrument would be useless and my practical advice would fail.
• For RTM, if the fourth layer's recognition could be fully explained by detection of recurrence, with no further role for knowing what is recognised, the layer would reduce to compression and the theory would lose a distinction.

What I still don't know#

I don't know how large the reference gap is for models trained on text, images and video together, rather than text alone. I don't know whether effective complexity can be measured for anything as large as a language. I don't know how far the contamination check transfers from lzma to large neural models, which store text loosely and paraphrase rather than copy. And I don't know whether Bhartṛhari's pratibhā, read as action-guiding grasp, can be turned into a measurement at all, or whether it marks a limit on what measurement can reach.

There is one more gap I should own. My experiments use a single book and a single standard compressor. They demonstrate the mechanisms. They do not estimate how large the effects are in deployed language models, and that estimate is the next piece of work.

Compression and understanding share a mechanism. Predicting well is describing cheaply, and describing cheaply, with the description charged, finds the rules that generate data. When a network replaces a memorised table with a short rule, it starts to generalise. When a grammarian saves half a mora, he has found a more general rule. That much of the compression thesis is right, and it is a lot.

When you score a model by compression, test it only on text it could not have seen, and treat the score as evidence of regularity-finding, not of understanding. Then test what compression cannot see: whether the model can act correctly on what the text is about, in a situation it has not met.

The rest of understanding lies where compression is silent. It needs symbols that reach the world, a sense of which regularities matter, and a grasp of what a whole utterance is for. Bhartṛhari called that grasp pratibhā and said it could not be passed on as "this is that". A compressor deals only in "this is that". That is its power, and it is the edge of what it can do.

Priya's board slide now has two rows. The first gives each model's compression on bulletins issued after its training data were gathered. The second gives the number of new-pest questions each answered correctly. Garuda wins both. Sudhir was right that compression matters. He was wrong about which compression, and about what it could tell him on its own.

There is a quieter lesson in the board meeting too. Nobody at Tunga Labs was wrong about the arithmetic. Hamsa really did compress the bulletins better. The mistake lay in what the number was taken to be about. Most errors with compression scores are of that kind. The measurement is exact, and the reading of it is loose.

The fix is cheap. Before anyone acts on a compression score, someone should write down, in one sentence, what the score is supposed to show and what else could have produced it. If the second list is long, the score needs company.

1. Shannon's 1948 paper, sections 1 to 7, for the identity between probability and code length (Shannon 1948).
2. Li and Vitányi's textbook on Kolmogorov complexity, chapters 1 to 3, for the theory and its limits (Li and Vitányi 2019).
3. Grünwald's book on minimum description length, part 1, for the two-part code done properly (Grünwald 2007).
4. Delétang and colleagues, Language Modeling Is Compression, for the modern evidence and Table 1 on model size (Delétang et al. 2023).
5. Harnad on symbol grounding, and Bender and Koller's octopus, for the reference gap (Harnad 1990; Bender and Koller 2020).
6. Hugo David's study of pratibhā in Bhartṛhari, for the action-guiding grasp of sentence meaning (David 2021).

Each source was opened and checked against its publisher page, DOI record, arXiv record or a primary-text edition before it was cited. Anything I could not confirm is marked “to verify”.

Primary research

1. Li, K., Hopkins, A. K., Bau, D., Viégas, F., Pfister, H., and Wattenberg, M. (2023). Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task. ICLR 2023; arXiv:2210.13382. https://arxiv.org/abs/2210.13382
2. Nanda, N., Lee, A., and Wattenberg, M. (2023). Emergent Linear Representations in World Models of Self-Supervised Sequence Models. arXiv:2309.00941. https://arxiv.org/abs/2309.00941
3. Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal 27(3), 379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x
4. Shannon, C. E. (1951). Prediction and Entropy of Printed English. Bell System Technical Journal 30(1), 50–64. https://doi.org/10.1002/j.1538-7305.1951.tb01366.x
5. Solomonoff, R. J. (1964). A formal theory of inductive inference. Part I. Information and Control 7(1), 1–22. https://doi.org/10.1016/S0019-9958(64)90223-2
6. Solomonoff, R. J. (1978). Complexity-based induction systems: Comparisons and convergence theorems. IEEE Transactions on Information Theory 24(4), 422–432. https://doi.org/10.1109/TIT.1978.1055913
7. Kolmogorov, A. N. (1968). Three approaches to the quantitative definition of information. International Journal of Computer Mathematics 2(1–4), 157–168. English translation of the 1965 Russian paper. https://doi.org/10.1080/00207166808803030
8. Chaitin, G. J. (1966). On the Length of Programs for Computing Finite Binary Sequences. Journal of the ACM 13(4), 547–569. https://doi.org/10.1145/321356.321363
9. Rissanen, J. (1978). Modeling by shortest data description. Automatica 14(5), 465–471. https://doi.org/10.1016/0005-1098(78)90005-5
10. Legg, S., and Hutter, M. (2007). Universal Intelligence: A Definition of Machine Intelligence. Minds and Machines 17, 391–444; arXiv:0712.3329. https://doi.org/10.1007/s11023-007-9079-x
11. Delétang, G., Ruoss, A., Duquenne, P.-A., et al. (2023). Language Modeling Is Compression. arXiv:2309.10668; ICLR 2024. https://arxiv.org/abs/2309.10668
12. Huang, Y., Zhang, J., Shan, Z., and He, J. (2024). Compression Represents Intelligence Linearly. arXiv:2404.09937. https://arxiv.org/abs/2404.09937
13. Power, A., Burda, Y., Edwards, H., Babuschkin, I., and Misra, V. (2022). Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets. arXiv:2201.02177. https://arxiv.org/abs/2201.02177
14. Nanda, N., Chan, L., Lieberum, T., Smith, J., and Steinhardt, J. (2023). Progress measures for grokking via mechanistic interpretability. arXiv:2301.05217; ICLR 2023. https://arxiv.org/abs/2301.05217
15. Chollet, F. (2019). On the Measure of Intelligence. arXiv:1911.01547. https://arxiv.org/abs/1911.01547
16. Harnad, S. (1990). The symbol grounding problem. Physica D 42(1–3), 335–346; arXiv:cs/9906002. https://doi.org/10.1016/0167-2789(90)90087-6
17. Searle, J. R. (1980). Minds, brains, and programs. Behavioral and Brain Sciences 3(3), 417–424. https://doi.org/10.1017/S0140525X00005756
18. Bender, E. M., and Koller, A. (2020). Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. Proceedings of ACL 2020, 5185–5198. https://aclanthology.org/2020.acl-main.463/
19. Gell-Mann, M., and Lloyd, S. (1996). Information measures, effective complexity, and total information. Complexity 2(1), 44–52. https://doi.org/10.1002/(SICI)1099-0526(199609/10)2:1<44::AID-CPLX10>3.0.CO;2-X
20. Chater, N., and Vitányi, P. (2003). Simplicity: a unifying principle in cognitive science? Trends in Cognitive Sciences 7(1), 19–22. https://doi.org/10.1016/S1364-6613(02)00005-0

Primary texts and translations

1. Nāgeśa (Nāgojibhaṭṭa). Paribhāṣenduśekhara. Sanskrit text on GRETIL (not proof-read), where "ardhamātrālāghavena putrotsavaṃ manyante vaiyākaraṇāḥ" is numbered 132 and "paryāyaśabdānāṃ lāghavagauravacarcā nādriyate" 123; other editions number them differently. https://gretil.sub.uni-goettingen.de/gretil/corpustei/transformations/plaintext/sa_nAgeza-paribhASenduzekhara.txt
2. David, H. (2021). Pratibhā as Vākyārtha? Bhartṛhari's Theory of "Insight" as the Object of a Sentence and Its Early Interpretations. Journal of Indian Philosophy 49, 827–869. Gives the Sanskrit of Vākyapadīya 2.143–2.145 with translation. https://doi.org/10.1007/s10781-021-09482-1

Official documents and datasets

1. Hutter, M. 500'000€ Prize for Compressing Human Knowledge. Official prize page with rules and table of records (consulted October 2026). http://prize.hutter1.net/
2. Mahoney, M. Rationale for a Large Text Compression Benchmark. Author's web page. http://mattmahoney.net/dc/rationale.html
3. Kipling, R. (1894). The Jungle Book. Project Gutenberg eBook 236, plain text, body only, used as test data. https://www.gutenberg.org/ebooks/236

Secondary sources and leads

1. Das, N. (2022). Pratibhā, intuition, and practical knowledge. British Journal for the History of Philosophy. https://doi.org/10.1080/09608788.2022.2074365
2. Li, M., and Vitányi, P. (2019). An Introduction to Kolmogorov Complexity and Its Applications. 4th edition. Springer. https://doi.org/10.1007/978-3-030-11298-1
3. Grünwald, P. D. (2007). The Minimum Description Length Principle. MIT Press. https://doi.org/10.7551/mitpress/4643.001.0001

Mahapatra, R. R. (2026). What a compressor knows: compression as a theory of understanding. Mind Frontier, paper 12, version 1.0. Rajesh Ranjan Mahapatra. https://rajeshrmahapatra.com/papers/mind-frontier/U0101-compression-as-a-theory-of-understanding.html (DOI to be assigned.)

@misc{mahapatra2026u0101,
  author       = {Mahapatra, Rajesh Ranjan},
  title        = {What a compressor knows: compression as a theory of understanding},
  year         = {2026},
  howpublished = {Mind Frontier, paper 12},
  note         = {Version 1.0},
  doi          = {10.XXXX/placeholder},
  url          = {https://rajeshrmahapatra.com/papers/mind-frontier/U0101-compression-as-a-theory-of-understanding.html}
}