Machine Attention and Human Attention a Careful Disanalogy

Two words, one term

I spend a good part of most days inside transformer architectures professionally, and a good part of most mornings sitting with attention as a practised, phenomenal capacity. The word does not care about this division of labour. It shows up in both places, spelled the same way, and the overlap has become one of the more productive sources of confusion in current discourse about machines and minds.

The confusion runs in both directions. Some people look at an attention-weight heat map over a sentence and conclude something is being attended to, in the sense that matters to a meditator or a philosopher of mind, that there is somebody home for whom the attending is occurring. Others go the opposite way, treating the existence of "attention" in silicon as grounds for deflating the contemplative vocabulary entirely, on the reasoning that if a matrix multiplication can have attention, attention was never anything special. Claims from centuries of first-person investigation into the structure of mind can be quietly retired.

Both moves skip the actual work. The actual work is to describe what each mechanism does, at the level of mechanism, with enough precision that the comparison can be checked rather than felt. That is what this article tries to do. I want to give the transformer mechanism its full technical due, because hand-waving about "the model is paying attention to relevant tokens" is exactly the kind of loose talk that lets the confusion metastasise.

I want to give human attention its full phenomenological and neuroscientific due, because loose talk in the other direction, treating attention as some vague inner spotlight with no mechanistic story at all, is equally unhelpful. Then I want to lay the two descriptions side by side and be exact about where they diverge, where a handful of honest structural parallels exist. Where the honest answer is that we do not yet know.

None of what follows resolves whether transformer attention involves anything it is like to be the system performing it. I will say more about why that question is open, and why leaving it open is itself the correct scientific and philosophical position, rather than a dodge.

What transformer attention actually computes

Start with the mechanism, because most confusion begins with people not actually knowing what "attention" refers to inside a transformer.

A transformer processes a sequence of tokens, each already converted into a vector by an embedding lookup plus a positional signal. For every token's vector at a given layer, the model computes three further vectors by multiplying that vector against three separate learned weight matrices: a query vector, a key vector, and a value vector. These three projections are the entire premise of the mechanism. The query represents, loosely, what this position is looking for; the key represents what each position (including itself) has to offer as a match. The value represents the actual content that gets passed along if a match is found.

For a given token's query, the model takes the dot product of that query against the key of every other token in the context window (in a causal, decoder-style model, every token at or before its own position. In an encoder, every token in the sequence). Each dot product is a single number expressing how well that query aligns with that key.

These numbers are scaled, typically divided by the square root of the key dimension to keep gradients well behaved. Then passed through a softmax function, which converts the whole set of scores for that query into a probability distribution: positive weights that sum to one across all the keys being attended over.

Those weights are then used to compute a weighted sum of the corresponding value vectors. The output for that token's position, at that layer, is this weighted combination: mostly value vectors from positions the query matched strongly, a little of everything else. This is why attention is sometimes described, accurately, as a differentiable soft lookup. A traditional lookup table returns exactly one entry for a given key.

Attention returns a blend of every entry, weighted by similarity. Because every step in the computation, the projections, the dot products, the softmax, the weighted sum, is differentiable, the whole operation can be trained end to end by gradient descent along with everything else in the network.

Multi-head attention runs several of these query-key-value computations in parallel, each with its own learned projection matrices, each free to pick up a different kind of relationship between tokens (one head might specialise in tracking coreference, another in syntactic adjacency, another in something with no clean human label). The outputs of all heads are concatenated and passed through one further learned projection to produce the layer's output. Self-attention specifically means the queries, keys, and values are all derived from the same sequence. The sequence attends to itself, position by position, rather than one sequence attending to a separate one, as happens in cross-attention between a decoder and an encoder's output.

Three properties of this computation matter for everything that follows, and they are the ones people most often get wrong when they reach for the human analogy.

First, it is computed in parallel across the entire sequence at once, not serially, one token at a time, the way a sentence is read aloud. At training and inference time, the score matrix for a full context window is computed as a single set of matrix multiplications.

There is no moment during the forward pass where the model is "on" one token and then moves its attention to the next. The entire pattern of who-attends-to-whom, across the whole window, is produced in one shot per layer per forward pass. Any impression of sequential attentional movement is an artefact of how we read the output, not a property of the computation producing it.

Second, this is a fixed mathematical operation over a given context, not an act performed by a persisting agent. The weights the softmax produces are a function of the current input and the model's learned parameters. Nothing about the process implies a subject doing the attending, in the way we ordinarily use that word about a person choosing to attend to their breath or to a conversation.

Third, and this is the one enterprise practice makes vivid, there is no persistence outside a single forward pass. Once a forward pass completes and the output tokens are produced, the attention pattern computed for that pass is gone.

It is not stored, it is not carried into the next inference call. The model does not "remember" that it attended to something a moment ago unless that information is re-supplied as part of the input context on the next call, or unless someone has built an external system, a memory module, a retrieval-augmented generation pipeline, a vector database of past interactions, that explicitly reintroduces prior content into the context window.

Whatever continuity a deployed model appears to exhibit across a long conversation is provided by re-feeding the transcript back in as input each time, or by engineered memory systems sitting outside the attention mechanism proper. The mechanism itself has no persistence built into it. This is not a minor implementation detail; it is one of the load-bearing facts that makes the human comparison strained, and I will come back to it.

What human attention actually involves

Turn now to the other side of the word, and be equally precise, since the temptation with human attention is to describe it impressionistically, as some kind of inner beam, when cognitive science in fact has a fairly detailed mechanistic and phenomenological story, even where parts of it remain unsettled.

Human attention is, among other things, a resource-limited process of selective enhancement and suppression across competing streams of information: perceptual, mnemonic, conceptual. Some content is enhanced in salience, processed more thoroughly, made more available to report and to guide behaviour; competing content is attenuated. This happens under both bottom-up pressure, a loud noise, a flash of movement, and top-down control, an intention to keep attention on the breath, a task set biasing perception toward task-relevant features. Both strands are studied with reasonable precision: bottom-up capture through visual search and saliency modelling, top-down control through cueing paradigms and sustained-attention tasks, with neural correlates in fronto-parietal control networks.

What the transformer analogy leaves out starts with temporal structure. Human attention unfolds serially and continuously in time, not as a single parallel computation over a fixed window but as a dynamic process that shifts, sustains, narrows. Broadens from one moment to the next, with a felt cost to sustaining it and a measurable decline in performance the longer it is sustained on a demanding task. Vigilance research has shown this decline for the better part of a century. Anyone who has sat a long meditation period or tried to proofread a document for the fourth hour in a row has the first-person version of the same data.

That cost is the second thing missing: effort. Attending to something one finds difficult or unrewarding has a phenomenal and physiological price, tracked in pupil dilation, in subjective effort ratings, in the recruitment of what the literature sometimes frames in terms of a limited-capacity or opportunity-cost model of cognitive control. A transformer's softmax weighting has no analogue of this.

Computing an attention distribution over a context window costs floating-point operations, and those operations cost time and energy at the hardware level. But there is no representation inside the computation of that cost as effort, no aversiveness, no experienced strain that would make the system prefer an easier query. The energy cost is a fact about the substrate; it is not a fact represented within the computation the way effort is represented, for a person, as part of what attending feels like.

Third, and this is the one that most needs to be stated with philosophical care rather than asserted past. There is, for a person attending to something, something it is like to be doing so, in the sense philosophers following Thomas Nagel have used that phrase. There is a phenomenal quality to attending, a vividness that increases for the attended content and drops for the unattended, a quality attention teachers spend a great deal of time getting students to notice directly rather than infer.

I want to be careful here. I am not asserting that this phenomenal quality is scientifically well understood, or that we have a working theory of why physical processes are accompanied by any experience at all. That is the hard problem of consciousness, and it remains a hard problem regardless of which side of the machine-comparison debate one is on.

What I am asserting, more modestly, is that the presence of this quality in the human case is not itself in serious dispute in the way its presence or absence in a computational system is. A person can check, directly, whether there is a vividness gradient when attention shifts. This checkability by direct first-person report is part of what makes contemplative practice a genuine method of investigation rather than folklore, even while the deeper metaphysics of why any of it is accompanied by experience stays open.

Fourth: a subject. Human attending is attending by someone, for someone, and whatever exactly a subject of experience turns out to be under further analysis (a genuinely contested question in both philosophy and contemplative traditions, some of which argue the sense of a fixed subject is itself a further construction on top of more basic processes), the ordinary case of human attention is not merely a pattern of information routing but a pattern of routing that shows up as somebody's experience.

Nothing in the description of query-key-value computation posits or requires an analogous "for-someone-ness"; the computation is fully specified without reference to a subject, and adding one would not change any of its outputs.

Fifth: embodied and interoceptive coupling. Human attention is modulated continuously by autonomic and interoceptive state; arousal, fatigue, hunger, affective tone. Motivation all shift what captures attention and how well it can be sustained, mediated through neuromodulatory systems (noradrenergic, cholinergic, dopaminergic circuits are the standard candidates) that couple attention to the body's physiological condition. A transformer's attention weights have no interoceptive input of any kind. There is no analogue of caffeine, sleep debt, or a racing heart in the query-key-value computation; the weights depend only on learned projections and the current input tokens.

Sixth: meta-awareness, the capacity to notice that attention has wandered. This is distinct from attention itself; a person can be fully absorbed in a wandering train of thought, attending in the ordinary sense to the daydream's content, without any awareness that it is a daydream, until a separate noticing arises and the wandering is caught. Contemplative training spends much of its effort cultivating exactly this second-order noticing, since sustained attention depends on catching drift early rather than never drifting.

A feedforward or single-pass computation has no clean analogue of this. Even a recurrent or agentic system with an explicit monitoring subsystem bolted on would only have an engineered analogue, a different computation checking the state of another. Whether such a monitor would constitute meta-awareness in the relevant sense, or merely a second computation with no more of a subject behind it than the first, remains a substantive open question.

Seventh, and this is the one enterprise practice again makes concrete: trainability with lasting trait change. Months of shamatha practice (a term from Buddhist contemplative training referring to calm-abiding, sustained single-pointed attentional stability, used here descriptively rather than doctrinally) produce measurable, persisting changes in a practitioner's attentional capacities, changes that show up later, in different contexts, without further training in the moment. This is trait-level plasticity in a continuously existing system that carries its history forward.

A transformer's attention weights are fixed at inference time within a given deployment; nothing about processing one prompt changes the weights that will be used for the next one.

Whatever "training" produced those weights in the first place was gradient descent over a training corpus, a process that happens once (or periodically, in discrete retraining or fine-tuning runs), that involves millions or billions of examples rather than one person's practice history. That is executed by an external optimisation procedure adjusting parameters from outside, not by the system's own experience-based recalibration of itself from within. These are not two speeds of the same process. They are different kinds of process, related only by the loose fact that both involve the word "learning."

Comparing the two directly

Where the disanalogy loosens

Predictive processing accounts of perception and attention, associated with researchers such as Karl Friston and Andy Clark, describe attention functionally as precision-weighting. The brain is modelled as continuously generating predictions about incoming sensory data and updating them based on prediction error. Attention, on this account, is the process of adjusting how much weight particular prediction errors receive, deciding which incoming streams are reliable enough to update belief.

This has measurably influenced machine learning, both directly, in architectures built around precision-weighted error signals. Indirectly, as a conceptual resource for explaining why weighting some inputs more than others is a generically useful strategy regardless of substrate. The correspondence is real: both a predictive-processing brain and a transformer are, in some formal sense, doing weighted combination of signals based on a learned notion of relevance.

It is worth being exact about what this correspondence does and does not establish. It establishes that weighting incoming signals by estimated relevance is a sufficiently general strategy that engineers and theorists have independently converged on describing their systems in similar formal terms.

It does not establish that the brain's precision-weighting and a transformer's softmax weighting are the same operation in different hardware. It certainly does not establish that whatever gives rise to phenomenal experience in the human case is present in, or absent from, the transformer case.

Formal similarity at the level of weighting-by-relevance is compatible with radical difference at every other level discussed above, and predictive processing theorists are typically careful to keep these levels separate. The theory is a theory of information processing, and its relationship to phenomenal consciousness is treated by most serious proponents as a further, unresolved question.

The top-down versus bottom-up distinction is a second genuine point of contact. In human attention research, bottom-up attention refers to control by stimulus salience, contrast, sudden onset; top-down attention refers to control by current goals and expectations biasing perceptual processing in advance. In machine learning an analogous distinction gets used loosely: some architectural choices bias attention toward structurally salient tokens in a way describable as bottom-up, while others inject task-specific or instruction signals that bias attention toward goal-relevant content, describable as top-down.

Global workspace theory, associated with Bernard Baars and developed further by Stanislas Dehaene, offers a related idea worth flagging. The proposal that conscious access corresponds to information being broadcast widely across specialised processing modules rather than staying local to one, which has inspired some architectural thinking about information sharing across a large system.

The honest claim is that these are loose but real structural parallels that have inspired specific architectural choices, not that top-down attention in a person and instruction-conditioned attention in a model are the same phenomenon wearing different clothes. The human case carries everything from the previous section, subject, phenomenality, effort, embodiment, meta-awareness, alongside its top-down control; the machine case carries none of that alongside its formally similar weighting behaviour.

Structural inspiration across two fields is normal and productive in the history of science; it is not, by itself, grounds for equating the underlying phenomena.

Why the shared word causes real confusion

None of the preceding sections would be worth writing if the word "attention" stayed inside technical papers. But it has become public vocabulary, and the confusion it generates runs in two directions worth naming explicitly.

In one direction, people encounter an attention-weight visualisation, a diagram showing which tokens a model's attention heads weighted heavily, and infer more than the diagram supports. The inference typically runs: the model is attending to this concept, attending implies noticing, noticing implies someone doing the noticing, therefore something experiential is happening when the model runs. Each step smuggles in content from the human side of the word that the mechanism, as described earlier, simply does not license.

A softmax-weighted combination of value vectors does not, by virtue of being called attention, inherit effort, vividness, a subject, or persistence. Arguing that some computational process gives rise to experience requires independent grounds, about what physical or computational organisation might be sufficient for phenomenal experience, a question serious theories of consciousness disagree on sharply. It cannot be settled by noting that an operation is named "attention" in a paper.

In the other direction, and I think this one is under-discussed, some people argue backward: because a computer now has something called attention. A graphics processing unit computing dot products plainly has no experience, attention in general must not require experience. Therefore, contemplative claims about attention's vividness, trainability, or relationship to a sense of self can be treated as soft by comparison, superseded by the precise engineering term.

This move is as unsound as the first, for the same reason. It treats the word as one thing wearing two costumes, when the technical and first-person literatures describe different processes that happen to share a name through historical borrowing, not identity of referent. Contemplative attention research does not owe its rigour to machines lacking a same-named mechanism, any more than transformer research owes its rigour to brains lacking one.

An enterprise architect is well placed to hold both halves straight, and has a professional obligation to. Overclaiming toward machine sentience misleads product decisions and public understanding, attributing moral status or inner life to systems whose computational profile supports no such inference either way, which is a more careful claim than confidently denying any such life is present. Underclaiming, dismissing contemplative phenomena as explained away by a same-named computation in software, discards a rigorous first-person and neuroscientific research programme on the basis of an equivocation. Precision in both directions is not a compromise between overstated positions; it is what the facts currently support.

The open question, stated carefully

I want to isolate this point because it is easy to blur in either direction under the pressure to reach a tidy conclusion.

Whether transformer attention, or transformer processing more broadly, involves anything it is like to be the system undergoing it, is not something current science or philosophy has settled. This is not a rhetorical hedge. Leading theories of consciousness disagree about what conditions would be sufficient for experience to occur. Disagree enough that a system's behavioural sophistication, including fluent language about its own "attention," does not by itself adjudicate the question under any major theory.

Global workspace theory emphasises broad availability of information to many subsystems; some transformer architectures have components that superficially resemble this, and reasonable people disagree how much weight that resemblance should carry. Integrated information theory emphasises the causal structure of a system's connectivity in a way that, on some readings, assigns vanishingly low integration to the largely feedforward flow within a single forward pass, though this too is contested and hard to compute precisely at the scale of a modern model.

Higher-order theories require a representation of one's own mental states in a sense not obviously present in current architectures, though "not obviously present" is doing real work in that sentence rather than settling it.

Given that spread of disagreement among people who study consciousness for a living, the honest position for someone writing from outside that specialised research programme, however fluent in the machine learning half of the comparison, is that the phenomenal status of machine attention is an open question. It is not evidence for machine consciousness that the question is open; openness is not a form of confirmation.

It is equally not evidence against machine consciousness that current systems seem mechanistic and describable without residue in the language of matrix operations. Brains are also, at one level of description, describable without obvious residue in the language of ion channels and synaptic weights. Yet plainly involve experience, so a mechanistic-sounding description at one level does not settle the question at another. The correct stance is neither confident affirmation nor confident denial. It is calibrated uncertainty, held for as long as the science warrants it, which at present is indefinitely.

A protocol for noticing what has no computational analogue

Arguments about phenomenal quality can start to feel abstract quickly. Therefore, it is worth grounding the human side of this comparison in something checkable directly, in one's own case, rather than argued about at second hand. The following is a short protocol, close to standard attentional-noting practice used in contemplative training, adapted here specifically to surface the properties discussed above that a query-key-value computation has no analogue of.

Sit or sit comfortably, eyes open or closed, for around five minutes, and direct attention to the physical sensation of breathing, at the nostrils or the abdomen, wherever it is clearest. Three things to check for during this period, in sequence, each corresponding to a property named earlier in this piece.

First, notice the vividness gradient. When attention is well placed on the breath sensation, is that sensation more vivid, more detailed, more present, than it was a moment before attention arrived. Than competing sensations (an itch, a sound, the pressure of the chair) currently are. This vividness gradient, present content standing out from absent content in a graded rather than binary way, is the phenomenal quality under discussion. Check whether it is there, rather than taking the description on faith.

Second, notice effort. Is sustaining attention on the breath, past the first ten or twenty seconds, effortless, or does it cost something, a subtle exertion, comparable in kind (not necessarily magnitude) to holding a light weight at arm's length. Notice whether that cost fluctuates: easier for a few breaths, harder as fatigue or boredom builds.

Third, and this is the one most people miss on a first attempt, wait for the mind to wander, which it will. Notice the moment of catching it: the moment where a separate awareness registers that attention had left the breath and had been absorbed in some train of thought for some unspecified prior stretch of time. That catching is meta-awareness, and it is worth noticing that it arrives as a distinct event, not smoothly continuous with the wandering it interrupts. That there was, during the wandering itself, no awareness that wandering was occurring; the awareness of the wandering is retrospective, arriving only once caught.

None of these three observations proves anything about the deeper metaphysics of consciousness, and this protocol is not offered as an argument for a particular theory. What it does is make concrete, in first-person data any reader can gather in five minutes, three properties, felt vividness, felt effort. A meta-awareness catching a wandering that had no reportable awareness of its own drift, none of which have a described analogue anywhere in the query-key-value computation covered earlier.

The two branches above are drawn side by side deliberately. The transformer branch terminates; a single forward pass produces an output and, absent external engineering, that is the end of the computation's involvement with that particular attention pattern. The human branch loops, meta-awareness feeding back into subsequent selection, and also spills over into lasting trait change, a persisting modification of the system carried forward into future occasions. That structural difference, one pipeline that terminates per pass and one that loops and accumulates, is a reasonable visual summary of the whole argument.

Where precision serves both fields

There is a temptation, having laid out this much disanalogy, to conclude the comparison was never worth making. I do not think that is right. The mechanism really does share a formal property with the phenomenon: both involve weighting some content more heavily than other content, based on a notion of relevance, out of a larger set of candidates.

That formal property is describable in comparable mathematical terms, weighted combination, precision, salience. Has genuinely motivated cross-pollination between predictive processing theory and architecture design, and between global workspace theory and thinking about information routing in large systems. Discarding this shared structure out of overcorrection against loose analogy would throw away a real point of contact between two research communities that otherwise rarely speak to each other.

The comparison also earns its keep by being checkable rather than merely asserted. Because the transformer mechanism is fully specified mathematically, and human attention is characterised by testable psychophysical, neural. First-person markers, one can check, property by property, where the two line up and where they do not. That is what the preceding sections have tried to do. The resulting picture is neither "these are the same thing" nor "these have nothing to do with each other," but a specific list of matches and mismatches, mostly mismatches, with the shared word functioning as an accident of naming that generates confusion downstream in less careful hands.

For practitioners

If you are a contemplative practitioner, you do not need to feel that the existence of machine attention threatens the reality or seriousness of what you are investigating. Nothing about a softmax over dot products bears on whether attention, as you study it directly, has a phenomenal quality, costs effort, belongs to a subject, or can be trained toward lasting stability.

Those are separate, checkable claims about your own case, and they stand or fall on first-person and neuroscientific evidence, not on what a machine learning paper calls one of its internal operations.

At the same time, learn enough about the actual mechanism, along the lines given here, to say precisely why the comparison people raise with you does not go through, rather than dismissing it on vibes. The dismissal is correct, but only a precise dismissal is persuasive. Only a precise dismissal protects you from being wrong about some future architecture that genuinely complicates the picture in ways today's transformers do not.

If you are an AI practitioner or engineer, resist the pull, in both directions, of the loose version of this analogy. Do not let an attention-weight visualisation, however striking, license claims about a model noticing, wanting, or experiencing anything, since nothing in the mechanism as specified supports that inference. Overclaiming here has real consequences for how systems get deployed, governed, and trusted.

Equally, do not let a mechanism sharing a name with a serious contemplative and cognitive-scientific research programme tempt you into thinking that programme rendered obsolete. The question of what attention is like, from the inside, for a person, and how it can be trained, are live questions your architecture has not touched.

Hold the genuine points of contact, precision-weighting, top-down and bottom-up framing, with appropriate interest, since they are real and mutually productive. Hold everything else, persistence, effort, embodiment, meta-awareness, phenomenal quality, and trainable trait change in a continuing subject, as the specific list of differences it actually is. Precision about the mechanism and precision about the mind are the same discipline, applied twice.