The factory alarm and forty million suspects

A visual inspection model sees a manufactured part and assigns a defect probability of 0.91. The verified label says the part is sound. The training loss rises. Somewhere inside the model are forty million adjustable numbers. Which of them caused the mistake?

A literal investigation would be hopeless. Change the first number slightly, run the model again and record the loss. Restore it, change the second and repeat. Forty million controlled perturbations would estimate one gradient for one example. The calculation would also confuse parameters that act through shared intermediate features, branches and reused weights.

Backpropagation makes a different move. It records the computation that produced the 0.91, starts with the sensitivity of the loss to that output, and traverses the graph in reverse. At every operation it asks a small question: if this operation’s output mattered by this amount, how much did each input matter locally? Those local answers compose into the derivative of the global loss with respect to every parameter.

Rumelhart, Hinton and Williams made this procedure central to learning internal representations in multilayer networks, showing that hidden units could be trained by repeatedly adjusting weights to reduce output error.[1] In modern software, the broader mechanism is reverse-mode automatic differentiation. Backpropagation is its specialised use for neural-network objectives.[2]

Part I. Credit is sensitivity along a realised computation

Suppose a model with parameters θ receives input x, produces a prediction and is scored against target y. Training defines a scalar function L(θ; x, y). The gradient is the vector of partial derivatives:

Each component answers a tightly bounded counterfactual. Hold the current example, all other parameters and the executed graph fixed. Nudge parameter θ_i by an infinitesimal amount. How quickly would the loss change? A positive derivative means a positive nudge raises loss locally. A negative derivative means it lowers loss locally. The magnitude is a rate in units of loss per unit of that parameter.

This is causal information about the mathematical computation, not a complete causal explanation of the world. A large derivative does not prove that a parameter encodes the human-recognisable cause of an error. A zero derivative does not prove irrelevance across other inputs, parameter settings or finite interventions. The derivative is indexed to one point and one objective.

Thought experiment: identical predictions, different credit

Build two one-path models and choose their constants so that both output exactly 0.91 for the factory image. In the first, the final value passes through a saturated sigmoid. In the second, it passes through a nearly linear region. Keep the prediction, label and loss identical. Change only the local slope of the internal route.

The output error is the same, yet the early parameter in the saturated route receives almost no gradient because the local derivative is near zero. The parameter in the linear route receives a much larger signal. Backpropagation therefore does not infer credit from the output discrepancy alone. It combines that discrepancy with the mechanism that produced the output.

Why reverse mode fits learning

Automatic differentiation can propagate sensitivities in either direction. Forward mode asks how one chosen input perturbation changes every downstream value. Reverse mode asks how one chosen output quantity depends on every upstream input. For a function from n parameters to m outputs, forward mode naturally computes Jacobian-vector products, while reverse mode computes vector-Jacobian products.

Neural-network training usually compresses many outputs and examples into one scalar loss. The Jacobian from parameters to loss is therefore one row and potentially millions or billions of columns. JAX’s official explanation makes the computational consequence explicit: building that wide Jacobian column by column with forward mode is unsuitable, while reverse mode constructs the needed row efficiently.[3]

The distinction between differentiation and optimisation matters here. Backpropagation computes a gradient; an optimiser decides how to use it. Gradient descent subtracts a scaled gradient. Momentum, Adam, clipping, weight decay and preconditioning transform that information. A correct gradient can still be paired with an unstable or inappropriate update. The adjacent article Optimisation, Approximation and Guarantees examines that downstream problem. This paper stays with the upstream evidence: whether the gradient being supplied is the one the executable loss and graph imply.

flowchart LR
  A["The factory alarm and forty<br/>million suspects"] --> B["Part I. Credit is sensitivity<br/>along a realised computation"]
  B --> C["Part II. The backward pass is<br/>a local message protocol"]
  C --> D["Part III. An exact gradient<br/>can still be the wrong<br/>teaching signal"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#7B4CD8,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Part II. The backward pass is a local message protocol

A computation graph decomposes a complicated function into primitive operations. Addition, multiplication, matrix multiplication, normalisation and nonlinear activation each have a local derivative rule. During the forward pass, the system computes values and retains whatever each backward rule will need. During the reverse pass, it visits operations in reverse topological order.

For an intermediate value v, define its adjoint as v̄ = ∂L/∂v. If an operation computes v = f(u) and receives v̄, it sends to its input:

For vectors, the multiplication becomes a vector-Jacobian product. The operation does not materialise a huge Jacobian if it can apply its transpose to the incoming vector directly. This is the practical heart of reverse mode.

Worked example: one hidden unit, entirely by hand

Take a scalar input x = 2 and target y = 0.4. The network has four parameters: w₁ = 0.5, b₁ = −0.25, w₂ = −1.2 and b₂ = 0.3. It computes:

The forward pass gives z₁ = 0.75, h = 0.635149, ŷ = −0.462179 and L = 0.371676. The prediction is below the target. Now start at the loss and move backwards.

The squared-error rule gives ∂L/∂ŷ = ŷ − y = −0.862179. The output affine operation has local derivatives ∂ŷ/∂w₂ = h, ∂ŷ/∂b₂ = 1 and ∂ŷ/∂h = w₂. Therefore:

The tanh operation contributes the local derivative 1 − h² = 0.596586. Multiplying by the incoming adjoint gives ∂L/∂z₁ = 0.617236. The first affine operation then sends ∂L/∂w₁ = x(∂L/∂z₁) = 1.234473 and ∂L/∂b₁ = 0.617236.

Every factor has a role. The output discrepancy seeds the signal. The downstream weight w₂ says how hidden activation influences the prediction. The tanh slope says whether the hidden unit can still move. The input x scales how strongly w₁ controls the preactivation. The gradient for an early weight is the product of all local sensitivities on the route from that weight to the loss.

Apply one gradient-descent step with learning rate 0.05. The parameters become approximately w₁ = 0.438276, b₁ = −0.280862, w₂ = −1.172619 and b₂ = 0.343109. On the same example, the loss falls to 0.233273. This confirms descent for this local step. It does not prove that every larger step, later batch or downstream metric will improve.

Where paths meet, contributions add

Suppose one hidden representation feeds both a classification head and a calibration head. The total loss is L = L_class + λL_cal, where λ is a declared weight. The shared representation receives two downstream messages. Its adjoint is their sum:

The same addition appears when a parameter is reused at several time steps, when a residual branch rejoins a main path, and when per-example losses are summed or averaged. TensorFlow’s official guide notes that differentiating multiple targets yields the gradient of their sum, equivalently the sum of their gradients.[4]

This explains the word distributed in the title. Each primitive operation performs only a local calculation, yet the incoming message already contains the downstream objective. Local computation does not imply a local learning target. The global loss is compressed into an adjoint and routed backwards through the graph.

Thought experiment: sever learning while preserving the prediction

Run the tiny network twice. In both runs, compute the same hidden value h and therefore the same prediction and loss. In the second run, insert a stop-gradient operation after h. Its forward function is the identity: it returns exactly the same number. Its backward rule returns zero to its input.

The output-layer parameters still receive gradients because they remain connected to the loss. The first-layer parameters receive none. Only backward connectivity changed. TensorFlow documents the general behaviour: variables that are not watched or sources that are not connected to the target return no gradient, and calculations that leave the recorded framework can break the path.[4]

Part III. An exact gradient can still be the wrong teaching signal

Backpropagation is exact relative to its function, arithmetic conventions and derivative rules. That sentence contains its strength and its boundary. The algorithm does not certify that the function represents the desired behaviour, that the local derivative remains informative across depth, or that a finite optimiser step will improve anything outside the current batch.

Boundary one: infinitesimal truth is not a finite-step guarantee

The gradient is the best first-order description of local change. For a smooth loss, moving a sufficiently small distance along the negative gradient decreases the loss unless the gradient is zero. A practical update is finite. Curvature can bend the landscape, stochastic batches can change the direction, and transformed optimisers can move somewhere other than the raw steepest-descent direction.

A directional check therefore has evidential value: on a fixed fixture, evaluate the loss at θ − αg for a sequence of decreasing α. If no small step lowers the loss, suspect a sign error, stale values, an inconsistent objective or numerical failure. If only large steps fail, the derivative may be right while the update scale is wrong.

Boundary two: products can erase or amplify the message

An early gradient contains a product of local Jacobians. If their relevant gains are mostly below one, the signal shrinks exponentially with path length. If they are mostly above one, it can grow explosively. Bengio, Simard and Frasconi analysed why long-term dependencies become difficult for gradient learning as temporal distance grows.[5] Glorot and Bengio later connected deep feed-forward training difficulty to activation saturation and signal propagation under initialisation.[6]

For a scalar illustration, repeat a local gain of 0.5 across twenty steps and a unit signal becomes about 9.5 × 10⁻⁷. Repeat 1.2 and it becomes about 38.3. Real networks use matrices, changing directions and nonlinearities, so singular values and alignment matter more than one scalar. The exponential intuition remains useful.

A small gradient is therefore ambiguous. It can mean true local insensitivity, saturation, a disconnected path, cancellation between objectives, aggressive averaging, underflow or a bug in a custom derivative. A large gradient is equally ambiguous. It can indicate useful sensitivity, unstable amplification, an outlier batch or an incorrectly scaled loss. Norms need context and controlled interventions.

Boundary three: the loss defines what receives credit

Return to the factory model. Keep the data, network and prediction fixed. In one run use ordinary cross-entropy. In another, weight missed defects ten times more heavily. In a third, add a calibration term. The backward mechanism is unchanged, but the gradients differ because the scalar objective changed.

Backpropagation is obedient to the objective, including its omissions. It cannot discover that a false alarm is expensive, that one subgroup needs a separate constraint, or that an action is irreversible unless those consequences enter the training and decision system. An exact gradient can efficiently train a proxy that is poorly aligned with the real decision.

This is the strongest practical objection to treating backpropagation as blame. Blame sounds normative. A derivative is conditional. It says which parameter change would reduce the declared number, not which behaviour an organisation ought to reward.

Boundary four: discrete choices and external effects break the path

Argmax selection, hard routing, sampled tokens, database calls and actions in an external environment may be discontinuous, stochastic or absent from the recorded graph. Ordinary chain-rule derivatives cannot pass through an unobserved real-world effect. Engineers then use score-function estimators such as REINFORCE, continuous relaxations, straight-through estimators, differentiable surrogates or separate credit mechanisms. Williams established a general class of statistical gradient-following reinforcement algorithms for stochastic units.[7] Bengio, Léonard and Courville compared several estimators for stochastic and hard units, including an explicitly heuristic straight-through rule.[8]

These methods change the evidence. An estimator can be unbiased yet high variance, or biased yet useful. A surrogate gradient can provide a training direction without being the derivative of the executed discrete function. The distinction belongs in the experiment record.

Serious objection: exact backpropagation is not uniquely necessary

Feedback alignment weakens any claim that successful multilayer learning requires transporting exact forward weights into the backward path. Lillicrap and colleagues showed that fixed random feedback weights could transmit useful teaching signals on a range of tasks.[9] That result matters because it separates learning from exact gradient computation.

The boundary is equally important. Bartunov and colleagues tested biologically motivated alternatives at larger scale and reported that feedback-alignment and target-propagation variants performed substantially worse than backpropagation on CIFAR and ImageNet settings, especially with locally connected architectures.[10] More recently, NoProp introduced layer-local denoising targets and reported stronger performance than earlier backpropagation-free methods on MNIST, CIFAR-10 and CIFAR-100 under its experimental design.[11]

The cautious conclusion is precise. Exact reverse-mode gradients are a powerful and scalable credit signal, not a logical prerequisite for every form of learning. Alternatives may trade exactness, memory, synchronisation, biological plausibility or objective access against variance, bias and benchmark performance. They do not alter what backpropagation itself computes.

Part IV. Make gradient integrity an engineering control

In a notebook, a wrong derivative often produces a visibly wrong number. In a production-shaped training system, the failure can look like slow convergence, seed sensitivity or a weak model. The graph may include several losses, custom kernels, mixed precision, gradient accumulation, data-parallel replicas and components that intentionally stop gradients. Evidence must be collected at those seams.

Worked scenario: a synthetic multimodal inspection model

Consider a synthetic system that combines an image encoder with vibration-sensor features. The branches merge into a shared representation. One head predicts defect class. Another estimates confidence calibration. The executable objective is the mean classification loss plus 0.2 times the calibration loss and a declared regularisation term.

Eight workers each process a shard of the mini-batch, compute local gradients, and reduce them into one update. This introduces two different senses of distribution. The computation graph distributes derivative calculations across operations. The training platform distributes examples and tensor work across devices. The first is mathematical composition. The second is a systems implementation that must preserve the intended reduction.

Now introduce one plausible bug. The calibration metric is converted to a NumPy value for logging before it is added back to the loss. The visible scalar can still be displayed, but its branch is absent from the autodiff graph. Training appears to use two objectives while only classification supplies gradients. TensorFlow’s guide explicitly warns that calculations outside the recorded framework can disconnect gradients.[4]

A second production bug: correct locally, wrong after reduction

Assume worker k holds n_k examples and reports the mean local gradient g_k. If the global batch contains N = Σ_kn_k examples, the gradient of the global mean loss is:

When all eight shards have equal size, averaging the eight local means gives this result. Summing them instead produces a vector eight times larger. If shard sizes differ, even an unweighted average of the worker means is wrong because a five-example shard receives the same influence as a fifty-example shard. The derivative rules inside every model replica can be flawless while the training system violates the loss definition at the reduction boundary.

This defect can survive superficial monitoring. An eightfold gradient may still lower the loss when the learning rate is small. An engineer may reduce the learning rate until the curve looks stable, accidentally compensating for the reduction error. Later, changing the number of devices changes the effective step again. The run then appears sensitive to scale, hardware or random seed, although the real cause is that the executable update no longer represents the declared global objective.

A matched control exposes the problem. Freeze stochastic augmentation, use the same ordered examples and initialise identical parameters. First compute the raw gradient for the complete batch on one process. Then partition those examples exactly as the distributed job does and collect the reduced gradient before clipping, momentum, adaptive rescaling or weight decay. Compare parameter tensors and also compare each named loss contribution. Small floating-point differences may remain because reduction order changes, so the tolerance must be declared. A consistent factor of eight, a discrepancy confined to short shards, or one missing objective branch identifies a semantic fault rather than ordinary numerical noise.

A distributed optimiser should not be trusted merely because every worker passed a local gradient check. Local checks establish operation-level differentiation. Reduction equivalence establishes that the collection of workers still differentiates the intended dataset-level scalar. Both are necessary, and neither proves that the scalar represents the business consequence that matters.

Reverse mode also has a storage cost. Backward rules often need forward activations. TensorFlow describes gradient tapes retaining intermediate results, and JAX notes that reverse-mode memory grows with computation depth.[4][3] Checkpointing trades recomputation for memory by retaining selected intermediates and recreating others during the backward pass. Chen and colleagues demonstrated systematic sublinear-memory schedules for deep networks.[12] The optimisation objective is unchanged if recomputation is numerically and semantically equivalent, but nondeterministic operations and mutable state make that equivalence an engineering claim that should be tested.

The gradient integrity protocol

The protocol below is a release instrument, not a debugging ritual. Apply it to each new loss composition, custom differentiable operation, precision regime or distributed training change. Preserve the fixture, tolerances and expected results with the training code.

Gradient clipping is not a substitute for this protocol. Clipping can be a legitimate update control, but it can also hide an exploding path, mis-scaled loss or faulty reduction. Measure the unclipped signal, record the transformation and test whether clipping changes which objective contribution dominates.

Executable artefact: manual backpropagation plus a negative control

The following standard-library Python programme implements the tiny network from Part II. It calculates gradients by hand, compares them with central finite differences, deliberately corrupts the tanh derivative, and verifies that one small update lowers the same synthetic loss. Save it as manual_backprop_checker.py and run python3 manual_backprop_checker.py.

"""Manual backpropagation checker for a two-layer scalar network.

Python 3.10+. Standard library only.
All values are synthetic. The positive case uses the correct tanh derivative.
The negative case deliberately uses 1 - h instead of 1 - h**2.
Expected result: PASS for the correct rule, FAIL for the corrupted rule,
and a lower loss after one small gradient step.
"""

from __future__ import annotations

import math
from dataclasses import dataclass


@dataclass(frozen=True)
class Case:
    x: float = 2.0
    target: float = 0.4


PARAMS = {"w1": 0.5, "b1": -0.25, "w2": -1.2, "b2": 0.3}


def forward(params: dict[str, float], case: Case = Case()) -> dict[str, float]:
    z1 = params["w1"] * case.x + params["b1"]
    h = math.tanh(z1)
    prediction = params["w2"] * h + params["b2"]
    loss = 0.5 * (prediction - case.target) ** 2
    return {"z1": z1, "h": h, "prediction": prediction, "loss": loss}


def manual_gradients(
    params: dict[str, float],
    case: Case = Case(),
    *,
    corrupt_tanh_rule: bool = False,
) -> dict[str, float]:
    values = forward(params, case)

    d_prediction = values["prediction"] - case.target
    d_w2 = d_prediction * values["h"]
    d_b2 = d_prediction
    d_h = d_prediction * params["w2"]

    if corrupt_tanh_rule:
        tanh_prime = 1.0 - values["h"]       # Deliberately wrong.
    else:
        tanh_prime = 1.0 - values["h"] ** 2  # Correct.

    d_z1 = d_h * tanh_prime
    return {
        "w1": d_z1 * case.x,
        "b1": d_z1,
        "w2": d_w2,
        "b2": d_b2,
    }


def central_difference(
    params: dict[str, float], name: str, case: Case = Case(), eps: float = 1e-6
) -> float:
    plus = params.copy()
    minus = params.copy()
    plus[name] += eps
    minus[name] -= eps
    return (forward(plus, case)["loss"] - forward(minus, case)["loss"]) / (2 * eps)


def relative_error(a: float, b: float) -> float:
    return abs(a - b) / max(1.0, abs(a), abs(b))


def check(corrupt: bool) -> tuple[float, list[tuple[str, float, float, float]]]:
    manual = manual_gradients(PARAMS, corrupt_tanh_rule=corrupt)
    rows = []
    for name in PARAMS:
        numerical = central_difference(PARAMS, name)
        error = relative_error(manual[name], numerical)
        rows.append((name, manual[name], numerical, error))
    return max(row[3] for row in rows), rows


def one_step(params: dict[str, float], learning_rate: float = 0.05) -> dict[str, float]:
    grads = manual_gradients(params)
    return {name: value - learning_rate * grads[name] for name, value in params.items()}


def main() -> None:
    good_error, good_rows = check(corrupt=False)
    bad_error, bad_rows = check(corrupt=True)

    print("Correct local derivative")
    for row in good_rows:
        print(f"  {row[0]} manual={row[1]: .12f} numerical={row[2]: .12f} rel={row[3]:.3e}")
    print(f"PASS: max relative error = {good_error:.3e}\n")

    print("Corrupted local derivative")
    for row in bad_rows:
        print(f"  {row[0]} manual={row[1]: .12f} numerical={row[2]: .12f} rel={row[3]:.3e}")
    print(f"FAIL AS EXPECTED: max relative error = {bad_error:.3e}\n")

    before = forward(PARAMS)["loss"]
    updated = one_step(PARAMS)
    after = forward(updated)["loss"]
    print(f"One step: loss {before:.12f} -> {after:.12f}")

    assert good_error < 1e-8
    assert bad_error > 1e-2
    assert after < before


if __name__ == "__main__":
    main()

Correct local derivative
  w1 manual= 1.234472645268 numerical= 1.234472645301 rel=2.654e-11
  b1 manual= 0.617236322634 numerical= 0.617236322692 rel=5.801e-11
  w2 manual=-0.547611925301 numerical=-0.547611925250 rel=5.103e-11
  b2 manual=-0.862178742865 numerical=-0.862178742844 rel=2.052e-11
PASS: max relative error = 5.801e-11

Corrupted local derivative
  w1 manual= 0.754960362153 numerical= 1.234472645301 rel=3.884e-01
  b1 manual= 0.377480181076 numerical= 0.617236322692 rel=2.398e-01
  w2 manual=-0.547611925301 numerical=-0.547611925250 rel=5.103e-11
  b2 manual=-0.862178742865 numerical=-0.862178742844 rel=2.052e-11
FAIL AS EXPECTED: max relative error = 3.884e-01

One step: loss 0.371676092324 -> 0.233273151812

The negative case is informative because only one local rule is wrong. Output-layer gradients still pass. First-layer gradients fail. This localisation is what a good checker should provide. PyTorch’s official gradcheck performs the same general comparison between analytical gradients and small finite differences, while JAX supplies direction-based checks to keep numerical cost manageable.[13][14]

stateDiagram-v2
  [*] --> Claim
  state "Thought experiment: identical<br/>predictions, different credit" as Claim
  state "Why reverse mode fits learning" as Test
  state "Worked example: one hidden<br/>unit, entirely by hand" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Compact glossary

Adjoint

The derivative of the final scalar loss with respect to an intermediate value, propagated during reverse mode.

Backpropagation

Reverse-mode differentiation applied to a neural-network computation and loss, usually followed by an optimiser update.

Credit assignment

The problem of determining how adjustable components contributed to an observed objective. Backpropagation supplies local sensitivities for a differentiable recorded graph.

Jacobian

The matrix of partial derivatives of a vector-valued function. Reverse mode usually applies its transpose to a vector without constructing the matrix explicitly.

Gradient check

A comparison between an implemented derivative and an independent numerical approximation on a small controlled fixture.

Stop-gradient

An operation that preserves a value in the forward pass while blocking or replacing its derivative in the backward pass.

Do not tune learning until the credit path is defensible

The factory model’s forty million parameters never become forty million independent suspects. The forward computation organises them into routes. The scalar loss seeds one backward question. Local Jacobian-transpose products distribute the answer. Branches add, reused parameters accumulate, and reverse mode recovers the exact gradient of the recorded differentiable programme.

The durable intuition is simple: backpropagation is a measurement system for local behavioural sensitivity. It is neither an account of biological learning nor a moral theory of blame. It does not select the objective, validate the labels, choose the optimiser, guarantee a finite update, cross an unrecorded external action, or certify that the resulting model helps the real decision.

This changes the engineering sequence. For a scalar differentiable objective with many parameters, reverse mode remains the default because it is exact for the graph and computationally well matched to a wide parameter-to-loss Jacobian. Before experimenting with learning rates or architectures, verify objective semantics, graph reachability, custom local rules, finite-difference agreement, signal scale, small-step descent and distributed reduction.

When the path contains hard choices, stochastic nodes or world effects, name the estimator or surrogate being used and report its bias and variance boundary. When an alternative credit method is chosen for locality, hardware or biological reasons, compare it with exact backpropagation under matched budgets. When the objective omits the consequence that matters, redesign the objective and decision contract rather than demanding more faithful gradients.

The changed decision is to release a training graph only when its backward messages are treated as testable operating evidence. A pleasing loss curve begins the investigation. It does not complete it.