A distant error and a local nudge

A derivative is a controlled perturbation question

Imagine a volume knob connected to a loudspeaker. Turn the knob a hair clockwise and measure the change in sound pressure. The derivative is the limiting ratio between those two small changes. It describes the response near the current setting. It says nothing about whether louder music is desirable, whether the listener likes the song, or what happens after a large turn crosses a hardware limit.

For a scalar function f(x), the derivative at x is the slope approached as the perturbation becomes small. Neural networks use the same idea, but the object of interest is usually a loss L that depends on millions of parameters collected in a vector θ. The partial derivative ∂L/∂θᵢ asks how the loss would respond to a small change in one coordinate while the other coordinates are held fixed.

The gradient packages all coordinate-wise sensitivities into one vector. Under the ordinary Euclidean notion of step size, it points in the direction of steepest local increase. The negative gradient points towards the steepest local decrease for an infinitesimal step. That geometric statement is useful and easy to overextend. A finite update may enter a curved region, interact with other coordinates or overshoot. Reparameterising the same function can also change the gradient vector because “one unit of movement” has changed meaning.

A gradient is a proposal about the next local move, conditional on a chosen coordinate system and objective. It is not a direction towards truth. The distinction becomes decisive once the loss is only a proxy for the result people care about.

The chain rule turns distant effects into local factors

A model is a composition of transformations. Suppose a parameter w₁ changes a pre-activation z, which changes an activation a, which changes a prediction ŷ, which changes a loss L. The chain rule decomposes the distant sensitivity into local terms:

This factorisation is more than a derivation convenience. It exposes where learning can fail. The loss may provide a strong signal, yet a saturated activation contributes an almost zero local derivative. A recurrent transition may multiply the signal by a similar Jacobian hundreds of times. A branch can contribute an additional path whose sensitivities add at a shared node. The topology and local conditioning together determine which parameters can hear the complaint.³

The gradient assigns infinitesimal influence, not moral blame

“Credit” can sound like praise or fault. In backpropagation it has a technical meaning: marginal influence on the selected objective through the selected graph at the current values. A parameter with a large gradient is not necessarily the root cause of an error. It may be the nearest adjustable lever. An upstream parameter may have created the misleading representation, while a downstream weight receives the larger derivative because it directly controls the score.

Causal responsibility also asks a different question. It may compare interventions across alternative system designs, datasets or policies. A gradient holds the current graph fixed and studies local parameter perturbations. It does not identify who chose the label, whether the data omitted a subgroup, or whether a decision threshold created harm after the model output. Those causes live outside the differentiable boundary unless explicitly represented in the objective and graph.

The derivative is exact about a deliberately narrow counterfactual. “What if this number changed slightly?” is not the same as “Why did the system fail?” The first question powers efficient optimisation. The second needs a broader evidence chain.

flowchart LR
  A["The complaint that reaches<br/>every weight"] --> B["A distant error and a local nudge"]
  B --> C["Reverse-mode as sensitivity<br/>transport"]
  C --> D["When correct gradients assign<br/>poor credit"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#087A98,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Reverse-mode as sensitivity transport

Why the graph is traversed backwards

A forward pass computes values from inputs and parameters towards one or more outputs. Training usually reduces those outputs to a scalar loss. The task then has an asymmetric shape: many parameters feed one scalar. Forward-mode automatic differentiation would push one chosen parameter perturbation through the graph at a time. Reverse mode starts from the scalar and computes sensitivities for every upstream value in one reverse traversal. This is why reverse mode suits neural networks with many parameters and a scalar objective.²

The forward pass must retain enough intermediate information to evaluate local derivatives later. A multiplication needs the other operand. A tanh derivative can use the cached activation. A normalisation layer needs statistics from the forward calculation. Frameworks such as PyTorch build or record a dynamic graph of operations and apply registered vector-Jacobian products during the backward pass.¹³

At a branch, one value may influence the loss through several descendants. Each descendant returns a sensitivity, and the contributions add at the shared node. At a merge such as addition, incoming sensitivity is distributed to both parents. At multiplication, each parent receives the incoming sensitivity scaled by the other forward value. The backward rules mirror local dependency, not a narrative about the whole model.

A parameter gradient is a sum over active paths

For a scalar acyclic graph, the derivative with respect to a parameter can be understood as a sum over every directed path from that parameter to the loss. Each path contributes the product of its local derivatives. Backpropagation does not enumerate those paths, which would be wasteful in a large graph. Dynamic programming reuses the accumulated sensitivity at each node, so common suffixes are processed once.

This view clarifies two apparently contradictory observations. A parameter can matter through many weak paths whose contributions add to a useful gradient. It can also matter strongly through separate paths that cancel at the current example. A near-zero total gradient therefore does not always mean “the parameter is irrelevant”. It may mean “the active uses disagree about the local direction”. Interventions and per-path or per-example diagnostics can distinguish those cases.

Shared parameters collect credit from every use

A convolutional kernel is reused across image positions. A recurrent transition is reused across time. A tied embedding may participate in both input encoding and output prediction. The parameter is one stored object, so its gradient is the sum of contributions from every use in the current graph. Reverse mode handles this by accumulating into the same parameter slot.

Suppose a recurrent weight is used at three steps and receives local contributions +0.8, −0.7 and +0.1. The reported gradient is +0.2. That small total hides substantial and conflicting temporal pressure. Changing the sequence length, masking one step or reweighting a loss can alter the cancellation without changing the forward definition of the shared weight. For long-horizon systems, log both the aggregate and the distribution of contributions across time or modules when feasible.

Credit accumulation is lossy evidence. The final parameter gradient is sufficient for a first-order update, but it can erase which examples, positions or uses supported and opposed that update. Auditing consequential training therefore benefits from per-example gradients, slice-level summaries or targeted ablations rather than one global norm alone.

One forward value can have two backward meanings

Automatic differentiation separates values from derivative rules. Two computations can produce the same number in the forward pass yet expose different credit paths. A framework operation such as detach or stop_gradient returns a value that downstream code can use while declaring its derivative with respect to the source to be zero. The prediction can remain numerically identical. Training behaviour changes because one edge has become invisible to the reverse pass.

Checking the reverse rules

A derivative implementation can be wrong even when the forward calculation is correct. The common diagnostic is a finite-difference check. Perturb one parameter by a small positive amount, perturb it by the same negative amount, and compare the observed loss slope with the reverse-mode result. A central difference usually reduces truncation error compared with a one-sided estimate:

Gradient checking isolates a useful claim: the implemented reverse rule matches the local behaviour of the implemented forward function. It cannot certify that the forward function represents the intended objective. The checker later in this article includes a deliberately reversed target. The derivatives pass because they faithfully optimise the wrong target.

When correct gradients assign poor credit

Batch aggregation can hide disagreement before depth matters

Even a short, perfectly conditioned graph can produce misleading aggregate credit when examples disagree. Training commonly averages gradients over a minibatch. Imagine nine routine examples each contributing +1 to one parameter and one rare severe example contributing −9. The batch average is zero. No update occurs, although the severe slice asks for a large change in the opposite direction.

The mathematics is correct for the chosen average loss. The failure lies in treating that aggregate as adequate evidence for every population. Class weighting, stratified sampling, robust objectives or explicit constraints change how examples contribute. Per-example gradient distributions can reveal whether a stable mean reflects agreement or cancellation. This is one reason training dashboards that show only total loss and global gradient norm can miss a material subgroup conflict.

Path products can erase or detonate a signal

In a long chain, the backward signal is repeatedly multiplied by Jacobians. If relevant singular values are mostly below one, sensitivity tends to shrink with depth or time. If they are mostly above one, it can grow rapidly. Recurrent networks reuse a transition many times, which made the vanishing and exploding gradient problem particularly visible. Bengio, Simard and Frasconi analysed the difficulty of learning long-term dependencies, while Pascanu, Mikolov and Bengio later described the geometry and practical role of gradient clipping.⁴⁵

The statement is about usable numerical credit, not the formal existence of a derivative. A parameter may have a non-zero exact gradient so small that floating-point precision, minibatch noise or regularisation overwhelms it. Another parameter may receive an enormous gradient that makes one update destroy previously useful behaviour. Clipping can bound the update generated by an exploding norm. It does not restore information that vanished earlier.

Residual connections create shorter identity routes through which states and gradients can travel. Deep residual networks showed that this architectural change could make much deeper image models train effectively.⁶ Deep supervision adds auxiliary objectives closer to intermediate layers, shortening selected credit paths.⁷ Neither mechanism proves that every early feature receives semantically correct credit. They improve transport conditions for stated losses.

Recent work also changes the route rather than repairing the same path. Kumar and Isola’s 2026 Supervised Memory Training first learns predictive memory targets with a Transformer-based encoder, then trains an RNN on one-step memory transitions without unrolling it.¹⁵ In the reported language, pixel-sequence and synthetic tasks, this creates a constant-length gradient path and can outperform backpropagation through time. The claim is bounded: the learned RNN can drift from the teacher trajectory, teacher expressivity constrains the target, and later BPTT fine-tuning may still be needed. The experiment is valuable because it separates two questions that conventional recurrence merges: what should be remembered, and how should memory update?

Zero local gradient does not prove zero finite influence

A ReLU unit below zero has derivative zero under the usual convention. A hard threshold is constant almost everywhere and discontinuous at its boundary. A saturated sigmoid has a derivative close to zero even though a large input change can move it into a responsive region. In each case, first-order credit at the current point can disappear while a finite intervention still changes behaviour.

This is an explicit boundary of gradient reasoning. The derivative describes the tangent, not the whole function. Second-order information can reveal curvature near a flat direction, but it cannot make a truly discrete operation differentiable. Practitioners then use surrogate gradients, relaxations, straight-through estimators, search or score-function estimators. Each choice changes the evidence carried by the update.

Stochastic and discrete choices change the estimator

When a random action determines later rewards, the realised sample path is not ordinarily differentiable with respect to the probabilities that produced it. The REINFORCE estimator uses the identity ∇θ E[R] = E[R∇θ log pθ(a)] under suitable conditions, turning reward-weighted log-probability gradients into an unbiased estimate.⁸ Its variance can be high, especially when rewards arrive long after many actions.

Stochastic computation graphs generalise the bookkeeping. Deterministic differentiable nodes use pathwise derivatives; stochastic nodes introduce score-function terms or reparameterised paths where available.⁹ Baselines can reduce variance without changing expectation. None of these methods eliminates temporal ambiguity. If a delayed reward follows fifty decisions, the estimator must distinguish which earlier actions changed its probability or value.

At a stochastic boundary, “the gradient” is often an estimate with its own bias, variance and sampling assumptions. Treating it as the same evidence as an exact reverse derivative hides a material change in the credit channel.

The objective can reward the wrong contributor

Suppose a service model predicts which customer messages require urgent review. The training label is “handled within four hours” because that field is easy to extract. The model quickly learns that messages arriving during staffed hours receive faster handling. Its gradients are correct. The learned policy routes daytime messages as urgent and may neglect serious overnight cases. The path from loss to parameters is intact, yet the target encodes operating capacity rather than customer harm.

Adding a fairness penalty, appeal outcome or reviewer-confirmed severity changes the objective and therefore the credit assigned to every feature. That is not a minor reporting adjustment. It changes which perturbations count as improvements. A feature correlated with staffing may receive positive credit under the response-time proxy and negative credit under an independently adjudicated severity loss.

Exact backpropagation is a strong baseline, not a metaphysical requirement

Backpropagation uses the transposed local Jacobians of the forward computation. In biological discussions this creates the weight-transport problem: how would a physical system provide feedback weights that match the forward weights? Feedback alignment research showed that fixed random feedback matrices can, in some networks and tasks, induce useful learning because the forward pathway adapts to the feedback.¹⁰¹¹ Direct feedback alignment sends output error directly to hidden layers rather than recursively through exact transposes.

These results are a serious counterexample to the claim that exact reverse derivatives are the only possible learning signal. They do not show that any approximate signal works, or that alternatives match backpropagation across architectures and scales. Performance remains architecture- and task-dependent. The useful lesson is methodological: compare credit channels by task performance, alignment, stability, communication cost and failure mode rather than treating one implementation as the definition of learning.¹²

The general requirement is informative, controllable credit, not ritual fidelity to one backward mechanism. Exact gradients remain the reference instrument for differentiable programmes because they make a precise optimisation claim. Alternatives inherit an obligation to state what they estimate, under which conditions, and how their errors change learning.

stateDiagram-v2
  [*] --> Claim
  state "A derivative is a controlled<br/>perturbation question" as Claim
  state "The chain rule turns distant<br/>effects into local factors" as Test
  state "The gradient assigns<br/>infinitesimal influence, not<br/>moral blame" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

The credit-path audit

Training trouble often triggers an optimiser search: lower the learning rate, add momentum, switch to Adam, increase the batch size. Those changes can help after a usable gradient reaches the optimiser. They cannot repair a target that rewards the wrong behaviour, a detached encoder, a derivative bug or a stochastic estimator whose variance overwhelms its mean.

Treat the credit path as an architectural object with observable stages. The audit below proceeds from semantic commitments towards numerical transport and finally back to behaviour. Each gate has a distinct test and a distinct repair. Skipping ahead makes one tool answer a question it was not designed to answer.

The optimiser consumes credit; it does not create it

Stochastic gradient descent, momentum and adaptive methods transform the gradient into an update. Momentum mixes current and past directions. Adaptive methods rescale coordinates using running statistics. Weight decay and clipping add further policy. These mechanisms can improve progress on a curved or noisy landscape. They remain downstream of the credit calculation.

If an encoder is detached, its gradient is exactly zero and Adam has nothing to rescale. If a proxy rewards the wrong outcome, momentum can pursue that proxy more efficiently. If per-example contributions cancel in the averaged loss, a sophisticated optimiser sees the cancellation, not the hidden disagreement. This boundary keeps diagnosis ordered: establish that the signal is meaningful and reaches the parameter before debating how to spend it.

Executable artefact: backpropagation checker

The browser instrument below runs the opening minimal network. It compares a hand-coded reverse pass with central finite differences. The positive case uses the correct derivative of tanh. The negative case injects the plausible but wrong rule 1 − tanh(z) instead of 1 − tanh²(z). The boundary case reverses the target and shows that a mathematically valid gradient can serve an inappropriate objective.

The same checker is provided as a dependency-free Python file: Run the embedded programme in this page. It exits successfully only when the outcomes are PASS, FAIL, PASS in that order. The negative result is intentional evidence that the test detects the injected derivative defect.

.

def relative_error(a: float, b: float, floor: float = 1e-12) -> float:
    return abs(a - b) / max(floor, abs(a) + abs(b))

def finite_difference(loss, theta, index, epsilon=1e-6):
    plus = theta.copy()
    minus = theta.copy()
    plus[index] += epsilon
    minus[index] -= epsilon
    return (loss(plus) - loss(minus)) / (2 * epsilon)

# Expected outcomes:
# correct tanh derivative         -> PASS
# injected derivative bug        -> FAIL
# reversed target, valid calculus -> PASS with semantic warning

How to interpret a failed check

A mismatch can come from the derivative rule, tensor broadcasting, in-place mutation, nondeterminism, an unstable finite-difference step or insufficient precision. Reduce the graph until the first disagreeing operator is isolated. Run in double precision. Disable dropout and random augmentation. Compare directional derivatives when checking every coordinate is too expensive. Do not loosen the tolerance until a defect disappears.

A passing check warrants one narrow conclusion: local reverse sensitivities match local changes in the implemented scalar function. Proceed to the next audit gate. Inspect per-layer gradient norms and update-to-weight ratios. Perturb an intermediate feature and verify the predicted downstream change. Replace a proxy label with a counterexample that exposes its semantics. Track whether lower loss improves held-out and operational outcomes.

Test the proposed update, not only the infinitesimal slope

After validating g = ∇L, evaluate several finite steps such as θ − αg with α spanning small orders of magnitude. For a sufficiently small positive step, a smooth loss should usually decrease along the negative gradient. If every practical step increases the loss, the learning rate may be too large, curvature may be severe, or optimiser state may dominate the fresh signal. If only microscopic steps help, the gradient can be correct yet operationally ill-scaled.

This directional test bridges calculus and training. It does not prove generalisation, but it verifies that the proposed update has the local effect the optimiser assumes. Run it on a deterministic minibatch, then repeat across batches to expose noise. Record both absolute loss change and parameter-relative update size, because a numerically small gradient can still be large relative to a tiny parameter.

Separate numerical credit from behavioural credit

A gradient check can pass while the update moves the wrong behaviour. The numerical question is whether the implementation differentiates the declared scalar function. The behavioural question is whether that scalar changes when the decision improves. Test them with two interventions. First, perturb a parameter or intermediate activation along the predicted sensitive direction and confirm the loss change. Second, construct two cases with similar loss but different operational consequences. If the update cannot distinguish them, the missing credit entered before differentiation, in the target, labels or aggregation rule.

This distinction changes debugging order. Do not compensate for a semantically weak loss by tuning momentum or clipping. Preserve per-case contributions long enough to see which examples dominate the batch gradient. Then compare the proposed finite update on held-out counterexamples and on the metric that governs release. A trustworthy credit path needs both receipts: the derivative matches the programme, and the programme rewards the behaviour the system is meant to produce.

sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: The complaint that reaches every weight
  M->>E: A derivative is a controlled perturbation question
  E-->>M: compare rival explanations
  E->>D: Make the credit path an architectural object
  D-->>Q: revise scope or act

Make the credit path an architectural object

The handwriting model's single complaint can now be unpacked. The loss supplies an initial sensitivity. Every operation applies a local reverse rule. Branches contribute and merge. The resulting gradient states how small parameter changes would alter that loss at the current point. Reverse-mode automatic differentiation makes this calculation tractable across a large graph, which is why backpropagation became the computational engine of deep learning.¹²

Yet the chain rule begins after the decisive semantic choice. Someone selected the target, loss and differentiable boundary. The graph may then block, attenuate, amplify or randomise the signal. A finite optimiser step may leave the local regime. A lower loss may fail to improve the decision. A better decision may still fail to produce the intended outcome because authority, workflow or human response sits outside the model.

The architecture decision changes from “Which optimiser should train this model?” to “What evidence makes this credit channel trustworthy?” Record the objective and its proxy risks. Make stop-gradient, routing and auxiliary-loss edges visible. Test custom derivatives against finite differences. Observe gradient transport by layer and time. State where exact derivatives become stochastic estimators. Verify the effect of an update, then verify the resulting behaviour and outcome independently.

The strongest objection also survives. Exact backpropagation is not the only possible credit mechanism. Feedback alignment, local objectives and stochastic estimators demonstrate alternatives under bounded conditions. Their existence strengthens the audit rather than weakening it. Every learning rule must disclose what signal travels, how it is related to the objective, what noise or approximation it introduces, and which failures it cannot detect.

The durable intuition is simple. A distant error becomes a local update only through a path of local sensitivities. The quality of learning is bounded by the weakest claim on that path: the meaning of the objective, the connectivity of the graph, the correctness of the derivative, the conditioning of the transport, the validity of the estimator and the evidence that the resulting action mattered. Backpropagation can calculate credit. Engineering must decide whether that credit is worth following.