The word "attention" now does double duty. It names the mechanism at the heart of every large language model I deploy, and it names the faculty I train every morning on the breath. The shared vocabulary is not an accident: the researchers who introduced the transformer chose the word deliberately, gesturing at the cognitive notion of focusing on relevant information. But shared vocabulary invites confusion, and the confusion runs in both directions: people over-read the machinery as a model of mind, and people dismiss the analogy entirely when it fails. Both moves lose information. The interesting work is in mapping exactly where the two notions align and where they part.
What transformer attention actually is
Strip away the branding and transformer attention is a content-based weighted retrieval. Each token emits three projections: a query, a key, and a value. To compute the representation of a given position, the model takes that position's query, scores it against every key in the context via dot product, softmaxes those scores into a distribution that sums to one, and returns the weighted sum of values. That is the whole operation. "Attention" here means: a learned, differentiable lookup where every element decides how much to read from every other element, with the weights determined by learned similarity.
Three properties matter for the comparison. First, it is all-to-all: in standard self-attention every position can attend to every other position in a single step, at quadratic cost. Second, it is soft: the softmax spreads weight across everything; nothing is ever fully excluded, only down-weighted. Third, it is parallel and stateless within a layer: all positions are computed at once. The operation itself carries no persistent internal state across the sequence beyond what is re-derived from the tokens each pass.
The mechanism is compact enough to state directly:
# Q, K and V are learned projections of the same token sequence.
scores = (Q @ K.T) / sqrt(key_dimension)
weights = softmax(scores, axis="context")
context = weights @ V
This code computes routing weights. It does not identify a target, sustain a goal, allocate metabolic effort or establish conscious access.
What biological attention actually is
Biological attention is not one thing, which is the first and most important disanalogy. Cognitive science distinguishes at least: selective attention (enhancing some inputs, suppressing others), sustained attention (maintaining focus over time), divided attention (splitting across tasks). The orienting, alerting, and executive networks that Posner's work separated and that map to partly distinct neural substrates. When we say a person "attends," we are compressing a family of related but dissociable capacities.
Structurally, biological attention is largely the opposite of the transformer's three properties. It is not all-to-all: the entire point is severe bottlenecking: the visual system alone discards the overwhelming majority of what hits the retina, and only a narrow band reaches the capacity-limited stage. It is closer to hard than soft: attention has a spatial and object-based locus, with measurable costs to switch and real suppression of the unattended, not gentle reweighting.
And it is deeply stateful and embodied: attention is coupled to eye movements, to arousal and neuromodulation (norepinephrine, acetylcholine, dopamine gating salience), to goals held in working memory, and to a body that must act. Attention exists in biology because an organism with finite metabolic budget must select what to act on. Its reason for being is scarcity and consequence.
| Dimension | Transformer attention | Human attention | Safe conclusion |
|---|---|---|---|
| Selection | Soft weighting over available tokens | Capacity-limited selection and suppression | Both route by relevance |
| State | Recomputed from the supplied context | Shaped by goals, arousal, history and body | Similar output can hide different mechanisms |
| Cost | Compute and memory | Metabolic, behavioural and opportunity cost | Scarcity is constitutive in biology |
| Consequence | Changes an internal representation | Changes perception, action, learning and report | Functional analogy is partial |
| Experience | No entailment from the operation | Coupled to conscious access in many tasks | The shared word settles nothing phenomenal |
Where the analogy illuminates
Held carefully, the parallel is genuinely useful.
Relevance-weighted routing. Both systems solve a real shared problem: given far more information than can be fully processed at once, compute a relevance score and let it govern what influences the next step. The dot-product-and-normalize structure of transformer attention is not a bad first-order caricature of how a salience map might weight competing inputs. The function (dynamic, content-dependent gating of information flow) is genuinely shared even where the implementation diverges.
Interpretability leverage. Because transformer attention weights are inspectable, they gave researchers a handle on internal information routing. Which tokens a head "looks at." This resonates with the way attention in cognition marks what is being bound and processed. The caveat, now well established, is that attention weights are not a faithful explanation of a transformer's computation. Much of the real work happens in the value transformations and the MLP layers, and high attention weight does not entail causal importance. The lesson generalizes: in both machines and brains, the locus of attention is not the whole story of the processing.
Capacity as an organizing constraint. Multi-head attention lets a model run several relevance computations in parallel and combine them: a rough echo of the brain running multiple feature- and space-based selection processes at once. In both cases, architecture is shaped by the need to attend to different things for different reasons simultaneously.
Where it breaks
The breakdowns are more instructive than the alignments, because they point at what attention is for.
Consequence and scarcity. Transformer attention is soft precisely because it need not choose: it has the compute to touch everything and merely reweight. Biological attention is hard because the organism cannot afford otherwise; selection has metabolic and behavioral cost, and getting it wrong means missing the predator. Machine attention is abundance managing relevance; biological attention is scarcity managing survival. This is not a detail. It shapes everything downstream.
No global workspace, no broadcast. This is where I think the deepest divergence sits, and where Global Workspace Theory earns its place in the comparison. In GWT (Baars, later Dehaene's global neuronal workspace), consciousness is associated with information being broadcast. A selected coalition of content winning access to a capacity-limited workspace and being made available to the whole system: report, memory, decision, and further reasoning. Human attention is intimately tied to this ignition-and-broadcast dynamic; what we attend to is, roughly, what gets globally shared.
A transformer has attention but no global workspace in this sense: no ignition threshold, no non-linear all-or-none broadcast, no single serial bottleneck through which winning content is made available to an integrated agent. It has parallel soft routing at every layer and nothing that plays the role of the workspace. If GWT is even approximately right about the functional signature of conscious access, then transformer attention is missing the very feature that, in us, makes attention feel like attention.
Prediction, not just retrieval. The predictive processing framework recasts biological attention as the precision-weighting of prediction errors. Attention is the gain the brain assigns to sensory signals based on their expected reliability, within a hierarchical generative model constantly predicting its own input. On this view attention is inseparable from a system that is actively modeling the world and acting to confirm its predictions.
Transformer attention, by contrast, is retrieval over a given context; the model is trained to predict tokens, but its attention operation is not itself a precision-weighted error-minimization over a generative world-model coupled to action. The functional roles overlap at the edges (both modulate the influence of information by something like expected relevance) but the transformer has no perception-action loop and no homeostatic stake in getting it right.
Statelessness versus continuity. My attention this morning was continuous with my attention yesterday, shaped by fatigue, mood, a lifetime of trained saṃskāra. A transformer's attention is recomputed from scratch each forward pass over whatever is in the context window. There is no ongoing attentional state, no drift, no wandering to be noticed and corrected: no one home to notice.
A clean experiment therefore separates three questions: does the system route information selectively, does the routing play the same causal role, and does any evidence bear on experience? Collapsing them produces the very confusion the comparison is meant to prevent.
Why the mapping is worth doing
I do not think transformer attention is a model of consciousness, and the analogy will not tell us whether these systems have any inner life: that question needs the tools of the next essay. But the exercise of laying the two side by side is clarifying in both directions. It sharpens what is special about the engineering: content-based differentiable routing was a genuine breakthrough, and it does not need to be brain-like to be powerful.
And it sharpens what is special about biological attention: the bottleneck is the point, the coupling to a workspace and a body and a predictive model is the point. The felt quality of attending (that we are someone to whom things become available) is exactly the part the machinery does not obviously reproduce.
The shared word is a good servant and a bad master. Used as a servant, it invites the comparison that reveals the difference.