Functions and Compositional Behaviour: Why the Model is the Path, not the Pile

Imagine receiving a sealed crate labelled “neural model”. Inside are twelve million floating-point numbers. The sender tells you their values, precision and total memory footprint. Nothing else arrives. No tensor shapes. No layer order. No activation functions. No routing rule. No statement of which numbers are multiplied, added, normalised, reused or ignored.

Can you run the model?

You cannot even decide what kind of input it accepts. The numbers might describe a language model, an image classifier, a recurrent controller, an ensemble, a lookup table or meaningless noise. A parameter count tells you how much adjustable storage exists. It does not tell you what transformation is executed.

Now imagine a second crate. It contains only a few hundred lines of architecture code and no learned weights. You still cannot obtain useful predictions, but you can trace the computation. You know where an input enters, how its representation changes, which branches can activate and where an output is produced. One crate contains learned content without an executable grammar. The other contains an executable grammar without learned content. A working model needs both.

The load-bearing claim of this paper is that model behaviour is caused by the composition graph through which parameterised transformations act. Parameters matter inside that graph. They do not replace it. This distinction explains why depth matters, why residual connections alter trainability, why sparse models have several incompatible notions of “size”, and why parameter count is a poor mechanistic explanation even when it remains a useful empirical predictor within a fixed model family.

Part IThe bag of weights that cannot answer

A function is a controlled change

A function is not merely an equation written on paper. It is a rule that associates each permitted input with an output. The notation y = f(x) compresses three commitments: what inputs are allowed, what output space is produced, and what rule connects them. A brightness adjustment maps pixels to pixels. A tokenizer maps text to symbols. A matrix maps one coordinate vector to another. An activation maps each scalar or vector to a transformed state.

Parameters make such a rule adjustable. For a linear map, the matrix entries are parameters. For a small neuron, weights and a bias determine the affine part, while the activation fixes what happens next. Writing fθ says that the function depends on a parameter set θ. It does not say that the function is the parameter set.

The intermediate states are crucial. The second map does not act on the original input. It acts on the world made by the first map. If the first map rotates a coordinate system, the second map stretches along the rotated axes. If the first map zeroes a negative activation, later layers cannot recover its precise negative value unless another path preserved it. If attention mixes information from several tokens, the feed-forward sublayer receives a context-dependent representation rather than an isolated token.

Composition creates dependence between stages. A later transformation inherits both the information preserved by earlier maps and the information they destroyed. This is why a layer’s role cannot be inferred from its weights alone. Its input distribution, upstream basis, downstream use and parallel paths all help determine what that layer does in the complete computation.

Why “pile of parameters” feels plausible

Parameter count correlates with memory, training cost and often performance. Scaling studies have found smooth loss trends when model size, data and compute change inside controlled model families.⁹ That empirical success encourages a stronger interpretation: more parameters cause better behaviour because more stored numbers mean more intelligence.

The hidden assumption is additivity. It treats each parameter as if it contributed an independent sliver of capability and the whole model as their sum. Modern networks do not work that way. Parameters interact multiplicatively through matrix products, conditionally through gates, selectively through attention, recursively through repeated blocks and contextually through the states created upstream. The value of one weight can matter only because earlier maps place a feature in its direction and later maps read the result.

A useful analogy is a musical score. Counting notes predicts something about duration and complexity. It cannot identify the melody unless note order, timing, instrumentation and recurrence are known. Yet the model case is stronger than the analogy. In a composition of functions, an early transformation changes the object on which every later transformation acts.

Part IIGeometry made by order

The smallest numerical example

Depth is repeated reuse of changed coordinates

A single hidden layer with enough units can approximate any continuous function on a compact domain under standard conditions.¹ That theorem establishes possibility, not economy. It does not say a shallow network will represent a structured function with a practical number of units, learn it from available data or expose a useful internal decomposition.

Depth changes efficiency because a deep network can reuse intermediate constructions. A first layer can form local boundaries. A second layer can recombine the resulting regions. A third acts on that recombination. For piecewise-linear activations, each stage can fold or partition the input space so that later stages operate on many transformed regions at once. Research on linear regions and depth separation formalises versions of this idea.²³

Depth is not merely “more parameters arranged vertically”. It is the repeated application of maps to representations already altered by previous maps. The same local rule can create rapidly increasing structural complexity when composed with itself.

Residual composition changes the unit of thought

A plain layer asks a representation to become a new representation: hl+1 = fl(hl). A residual layer asks for an update: hl+1 = hl + gl(hl). The identity path preserves the current state while the learned branch proposes a change. Deep residual networks made this organisation practical at large depth and became foundational across vision and later transformer designs.⁴

The distinction is compositional. In the plain case, every layer must carry forward whatever later layers may need. In the residual case, the representation persists unless an update changes it. This supplies a direct gradient path during training and gives the forward computation a trajectory-like interpretation. Neural ordinary differential equations take that intuition further by learning a continuous state derivative rather than a fixed stack of discrete layers.⁷

The reverse also holds. Keep a collection of parameter values fixed but change tensor shapes, operation order, activation placement or connectivity. The realised function changes. Together these facts break a tempting one-to-one picture:

Part IIIThe path through a modern block

A transformer token is repeatedly rewritten

Consider one token entering a transformer block. Its vector is first normalised in many transformer designs. From that representation, learned projections create queries, keys and values. The query interacts with keys from other positions. A softmax converts those scores into routing weights. Weighted values are mixed, projected and added back through a residual path. Another normalisation precedes a feed-forward transformation, whose output is again added to the stream. The original transformer established attention-based sequence modelling; its descendants vary details while retaining this compositional pattern.⁵

No single parameter says “resolve the pronoun using the earlier noun”. The behaviour can arise when upstream features create a useful query, another position creates a matching key, its value contains relevant information, attention routes that value, the residual stream retains it, and later maps convert it into an output preference. The explanation is relational and staged.

A block’s role depends on its neighbours

Suppose two transformer blocks have identical architecture and even identical weights. Place one near the beginning of a network and the other near the end. They need not play the same global role. The early block receives representations close to token and position encodings. The late block receives states that have already been mixed, filtered and rewritten many times. Its output is also closer to the final readout. Equal local maps, inserted at different points in a composition, therefore define different functions of the original input.

The same caution applies when researchers name a layer “the syntax layer”, “the memory layer” or “the planning layer”. A readable feature at one location may be created upstream, transformed locally or merely preserved for a downstream consumer. Transplanting the block into another position changes its input basis and the maps that interpret its output. A mechanistic claim needs an intervention that respects those relations, such as patching the relevant state, bypassing the block or compensating for the changed basis.

A component’s local rule and its compositional role are different descriptions. The local rule tells us how the component transforms the state it receives. The role tells us why that change matters inside this graph, for this input distribution and this downstream use. Parameter inspection can support the first description. The second requires tracing and intervention across boundaries.

Composition can branch, route and reuse

A chain is the simplest composition. Modern models also contain parallel branches, recurrent state, shared weights and input-dependent routing. In a sparse mixture-of-experts layer, a gate selects a small subset of experts for each token. Total parameter count may be enormous, while the active parameters and computation for one token are much smaller.⁸ The question “How large is the model?” now has several answers: total stored parameters, active parameters per input, executed floating-point operations, memory traffic and effective depth.

This suggests a practical construct: the functional itinerary of an input. It is the ordered, possibly branching record of transformations that actually participated, together with the intermediate states handed across their boundaries. For a static dense network, many inputs share the same topological route but differ in activations. For routed systems, both the route and activations can differ. For recurrent or iterative systems, the itinerary also includes how many times a map was applied and what state persisted between applications.

The itinerary gives model comparison a sharper unit than parameter count. Two models can have similar counts but very different active paths. One model can produce different behaviours by sending different inputs through different experts. A weight can be statistically important in aggregate yet absent from a particular decision. Conversely, a small routing gate can control which millions of downstream parameters participate.

Composition need not appear as a visibly finite stack. A deep equilibrium model defines its representation as a fixed point, such as z* = fθ(z*, x), and uses a solver to find a state that no longer changes materially under the shared map.¹² The learned parameters can stay fixed while the number of solver steps changes with tolerance and input difficulty. The functional itinerary must therefore record repeated application and stopping conditions. An implicit model does not escape composition. It makes the composition shared, iterative and partly determined at runtime.

Composition does not guarantee semantic modules

Nested transformations explain how a global function is built from local changes. They do not guarantee that each stage corresponds to a concept a human would name. One layer can participate in many behaviours. One behaviour can depend on features distributed across many layers. Superposition, redundant paths and basis changes can make the same computation look different internally while preserving the output.

Decomposition is also non-unique. The composite map F may be written as g ∘ f, but an invertible change of coordinates gives an alternative factorisation (g ∘ Q⁻¹) ∘ (Q ∘ f). Both compute the same global function while exposing different intermediate coordinates. A convenient decomposition can aid engineering without revealing a uniquely correct ontology of the task.

Where composition fails

Part IVThe function-composition lab

The fastest way to acquire the intuition is to manipulate a sheet yourself. The lab below begins with a regular grid and one tracked point. Choose three maps, then inspect the sheet after each stage. Reverse the order while keeping every selected map unchanged. Try inserting an identity map. Try two bends followed by a fold. The visual difference is the global function becoming visible.

How to use the lab as an experiment

Begin with shear, bend and fold. Record the tracked point and the visible topology of the grid. Reverse the order. A changed output is a positive result for order sensitivity. It permits the narrow conclusion that the selected maps do not commute on that input and region. It does not prove that depth is universally better or that a trained model uses an analogous mechanism.

Next replace the middle map with identity. If the output is unchanged, the identity correctly demonstrates that adding a stage need not add a new function. This is the smallest model of a function-preserving depth increase. If numerical differences appear, inspect implementation precision rather than declaring a mechanism.

Then choose squash early and stretch late. Observe how the grid compresses before the stretch. Reverse the pair. Stretching before saturation and after saturation produce different recoverable ranges. This shows that a later high-gain map cannot necessarily restore distinctions collapsed by an earlier many-to-one or near-saturating map.

const compose = (...maps) => point =>
  maps.reduce((state, map) => map(state), point);

const forward = compose(f1, f2, f3)(input);
const reverse = compose(f3, f2, f1)(input);

const orderSensitive = distance(forward, reverse) > tolerance;

From pretty picture to composition manifest

A real model needs a machine-readable description of more than layer names. At minimum, record each transformation’s input and output type, parameter reference, activation, normalisation, residual parent, routing condition, state read and state write. The “Copy manifest” button exports the current toy route in that spirit.

Part VThe decision this changes

The strongest counterargument: size predicts surprisingly well

Parameter count should not be discarded. Within a stable transformer family, controlled scaling experiments have found that loss can be predicted from model size, data and compute over wide ranges.⁹ Compute-optimal work then showed that parameter growth must be matched by sufficient training data, demonstrating that even a strong size relationship is conditional on the training regime.¹⁰ For planning memory, compute and approximate performance inside a specified family, count is valuable.

The counterargument becomes too strong only when a family-level predictor is treated as a mechanism or transported across unlike architectures. Recent multi-shape scaling work has shown that fitted prescriptions can be sensitive to width, depth, learning rate, schedule and checkpoint selection.¹¹ Sparse routing further separates total parameters from active parameters. A number that predicts loss in one controlled regime cannot, by itself, explain why a particular input produced a particular answer.

Parameter count is a coarse coordinate, not a causal decomposition. Height predicts some properties of buildings, but it does not tell an engineer which beam carries a load. In the same way, model size can organise empirical curves while leaving the transformation path unexplained.

A composition audit before you compare models

When someone says “Model A is better because it has more parameters”, ask for the missing bridge. Are both models dense or sparse? Do they have the same depth, width, activation and normalisation pattern? Do they use the same amount of training data? How many parameters are active per token? Is inference repeated, adaptive or routed? Does one architecture retain state that the other recomputes? Which composed transformation is hypothesised to cause the observed gain?

What to do differently tomorrow

When learning a model, draw the state path before counting the parameters. For every stage, ask what enters, what changes, what is preserved, what is discarded and what receives the result. Then run one intervention: swap two maps, bypass a block, clamp an intermediate state, change a route or repeat an update. An explanatory story that predicts none of these changes is still a description.

When designing a model, choose composition according to the structure of the task. Use depth when intermediate reusable structure is plausible. Use residual paths when state preservation and incremental updates matter. Use routing when different inputs genuinely require different transformations and the gate can be evaluated. Avoid bottlenecks that erase distinctions needed later. Treat normalisation and activation placement as architectural decisions, not punctuation.

When procuring or governing a model, request more than a parameter count. Ask for total and active parameters, depth, recurrence or adaptive compute, routing behaviour, state persistence, context mechanism, precision and the configured inference graph. For consequential outputs, retain an execution trace at the granularity needed to test which transformations and routes participated.

That decision changes how the next papers should be read. Sets and logic will specify what objects and claims a model can relate. Calculus will show how local sensitivity moves backwards through this very composition. Transformers will later appear not as magical monoliths, but as carefully arranged maps whose global behaviour must be traced through intermediate states.

The sealed crate can now be diagnosed precisely. Its numbers are necessary but causally mute until an executable structure gives them roles. Open the crate, recover the maps, recover their order and recover the routes. Only then do you have a model whose behaviour can be explained.