From a pulse to a trainable transformation

Begin with a small machine. It receives measurements x₁, x₂, …, multiplies them by adjustable weights, adds a bias and passes the result through a response curve. For one unit, the computation is:

This form descends from several different historical projects. McCulloch and Pitts used idealised threshold elements to analyse logical relations in nervous activity.[1] Rosenblatt’s perceptron joined a model of adaptive connections to classification.[2] Widrow and Hoff’s adaptive circuits used an error-driven least-mean-squares rule.[3] These are not interchangeable biological claims. Their common engineering move is to make a transformation depend on adjustable coefficients.

A neuron, however, does not learn merely because its coefficients are adjustable. It also needs a criterion. Suppose the unit predicts a probability p and the desired label is y. A loss function turns their mismatch into one scalar. An update rule must then answer a sharper question: which small parameter change would reduce that scalar?

For a single output unit the answer can be local. Under logistic output and binary cross-entropy, the sensitivity of loss to the pre-activation simplifies to p − y. The input-weight sensitivity is that error signal multiplied by the input. A feature with value zero receives no update on that example. A feature with the opposite sign receives the opposite update. The rule already depends on more than activity.

Why hidden units create the real puzzle

A single-layer classifier has a visible target at its output. A hidden unit does not. No label says that hidden unit 7 should emit 0.43. Its usefulness depends on what later layers do with it. This is the multilayer credit-assignment problem.

Rumelhart, Hinton and Williams made the practical neural-network procedure widely visible: repeatedly adjust connection weights to reduce the difference between actual and desired outputs.[5] The deeper mathematical device is reverse accumulation. Linnainmaa had described an algorithm that stores local derivative information during a computation and processes it in reverse to determine how local errors influence a final result.[4] In modern terms, backpropagation is reverse-mode automatic differentiation specialised to a network and its loss.

flowchart LR
  A["The neuron that fired and did<br/>not deserve the blame"] --> B["From a pulse to a trainable<br/>transformation"]
  B --> C["Responsibility travels through<br/>the path"]
  C --> D["When the backward field collapses"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#087A98,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Responsibility travels through the path

Imagine three adjustable dials in a row. The first changes a signal that enters the second. The second changes what reaches the third. The third determines the measured output. Turning the first dial matters only through the transformations after it. The chain rule is the bookkeeping law for this dependence.

For a two-layer scalar chain, let h = f(x, w₁), let ŷ = g(h, w₂), and let L(ŷ, y) measure error. Then:

Each factor is local. Their product is not. It expresses the influence of an early parameter through the whole downstream route. With many branches, contributions from all routes add. With vectors, each local factor becomes a Jacobian or a vector-Jacobian product. The principle stays the same.

For a layer with many outputs, writing the full Jacobian is usually wasteful. The backward sweep already possesses a row vector describing how the loss changes with the layer’s outputs. Multiplying that vector by the local Jacobian produces the sensitivity of the layer’s inputs. This vector-Jacobian product computes exactly the slice needed by the loss. It avoids materialising a matrix whose unused rows would describe other hypothetical output perturbations.

This also explains why branches add rather than compete for a fixed budget. If one hidden state affects the loss through three routes, a tiny change perturbs all three. The total first-order effect is the sum of the route effects. “Credit” is therefore not a conserved quantity divided among neurons. Two parameters may both have large gradients, or several routes may cancel to nearly zero.

Reverse mode is an economical reuse of the graph

A network may contain millions or billions of parameters but usually produces one scalar loss per batch. Finite differences would perturb each parameter, rerun the network and estimate the loss change. That costs roughly one extra forward evaluation per parameter and introduces a step-size approximation. Reverse mode instead computes the loss once, sets its derivative to one, and sweeps backwards. At each operation it combines the incoming sensitivity with the operation’s local derivative.

The sweep is efficient because a downstream sensitivity is reused by every parameter feeding that downstream state. This is why automatic-differentiation systems record a computational graph or retain the values needed to reconstruct local derivatives. The idea is visible in Linnainmaa’s reverse processing of saved operations and partial derivatives.[4]

What “responsibility” means here

The derivative has a counterfactual reading: hold the batch, objective and other parameters fixed, then ask how an infinitesimal change in this parameter would alter the loss. That is useful, but narrower than causal responsibility in a scientific explanation. A zero derivative can occur at a saturated unit or saddle even when a large intervention would help. A large gradient can arise because the chosen loss weights an example heavily, not because the feature caused the real-world outcome.

One parameter can collect credit from many uses

So far, each arrow has looked as though it owns a separate parameter. Real networks reuse parameters. A convolutional kernel is applied at many image locations. A recurrent weight is applied at many time steps. A mini-batch applies the same model to many examples. The computation graph therefore contains several uses of one stored value.

Suppose one scalar weight appears twice in a toy prediction, ŷ = wx₁ + wx₂. With squared error, the backward pass sends one contribution through each use and adds them at the shared parameter:

Set x₁ = 3 and x₂ = −2. The first use argues three units in one direction; the second argues two units in the other. The stored weight receives their sum. If the inputs were 3 and −3, the net gradient would be zero even though both paths carried large, opposing contributions. Cancellation is not the absence of credit. It is disagreement hidden by aggregation.

The same issue appears across examples. For a batch of size B, training usually averages B per-example gradients. A near-zero mean may indicate that every example is locally satisfied. It may instead conceal two subgroups demanding opposite changes. In a convolution, one edge may urge a kernel coefficient upward while another urges it downward. In a recurrent network, an early time step and a late time step may oppose each other through the same transition matrix.

This suggests a practical diagnostic before changing the optimiser: decompose the gradient by example, subgroup, spatial use or time step, then inspect both the sum and its dispersion. Large opposing components point to objective conflict, representational entanglement or a badly mixed batch. Small components everywhere point to weak reach or local saturation. The decomposition remains objective-relative, but it reveals a fact the final gradient norm erases: whether a quiet parameter is irrelevant, already adequate, or caught between incompatible demands.

When the backward field collapses

The chain rule is exact for the differentiable computation it describes. Exactness does not guarantee usefulness. The returned signal may be too small, too large, wrongly directed by a coding error, or correct for an old forward pass. Credit can fail in transport even when the final loss is perfectly well defined.

Four common ways credit transport fails

Saturation flattens the local derivative. A logistic unit driven far from zero emits a value near zero or one, where its derivative is tiny. Glorot and Bengio showed how activation choice and initialisation can push deep networks into regimes with poor gradient transport and long plateaux.[7] Rectifier-aware initialisation was later derived to preserve signal scale more effectively for rectified networks.[8]

Hard gates can sever a path. A ReLU below zero has derivative zero. An argmax, discrete lookup or external tool call may have no ordinary derivative at all. The forward system can still produce a value, but exact backpropagation cannot pass sensitivity through that operation without a surrogate, estimator or redesigned boundary.

Repeated Jacobians can explode. A direction expanded slightly at each layer can become enormous. The gradient may be mathematically correct but numerically unusable. Clipping limits update magnitude, while architectural routes such as residual connections create shorter derivative paths. Residual networks were introduced partly because substantially deeper plain networks were difficult to optimise.[10]

The correct derivative can be paired with the wrong state. If a backward pass consumes activations from another batch, parameter version or stochastic draw, it computes a derivative for a computation that did not produce the declared loss. Frameworks usually manage this tape. Custom asynchronous or distributed systems must make the invariant explicit.

The loss meter decides what counts as error

Credit is also objective-conditioned. Consider a synthetic inspection classifier whose missed defects are assigned eight times the loss weight of normal examples. For a positive case predicted at probability 0.2, the logit sensitivity is 8 × (0.2 − 1) = −6.4. For a negative case at the same probability it is +0.2. The positive example launches a signal 32 times larger in magnitude.

The network did not discover that the hidden units are morally more responsible for defects. The designer declared a cost asymmetry. Backpropagation faithfully transports it. Change the class weight, add an auxiliary loss or optimise calibration instead of accuracy, and the entire responsibility field changes without changing the initial forward activations.

The biological boundary

The artificial neuron is a computational primitive, not a compressed biological neuron. Exact backpropagation normally uses a global objective, stored forward state and backward transformations related to forward weights. That does not show that cortex implements the same algorithm.

Experiments with feedback alignment demonstrate that exact symmetric backward weights are not required for useful learning on every task: fixed random feedback can, under some conditions, become aligned with the forward system.[11] Direct feedback alignment sends random projections of output error directly to hidden layers and also learns on several benchmark settings.[12] More recent work proposes local spatio-temporal approximations in dynamical neuronal networks.[13]

These results weaken one claim, that exact weight transport is the only route to multilayer credit. They do not establish which mechanism a brain uses. The correct comparison is experimental: identify the teaching signal, locality constraints, timing and predicted perturbation response, then test them. Engineering success and biological implementation remain separate evidential ledgers.

stateDiagram-v2
  [*] --> Claim
  state "A complete update you can<br/>calculate by hand" as Claim
  state "Why hidden units create the<br/>real puzzle" as Test
  state "When two correct routes partly<br/>cancel" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Audit the signal before changing the system

When a network fails to learn, teams often jump to more data, more width or a different optimiser. Those changes can hide a broken backward path. A smaller diagnostic question comes first: did the intended credit reach each trainable block, in the right direction, at a usable scale, from the correct forward pass?

The following practitioner synthesis turns that question into a four-part credit integrity profile. It is not a new learning algorithm. It is a decision instrument for deciding whether training evidence is interpretable.

Reach

Record norms of ∂L/∂h_ℓ and ∂L/∂θ_ℓ by layer. A sudden zero can reveal a detached tensor, dead gate or unused branch. Smooth exponential decay suggests transport through derivatives below one. Norms alone do not prove correctness, but they show whether any first-order signal arrived.

Directional agreement

Choose a random unit direction v in parameter space. Compare the analytical directional derivative g · v with a centred finite difference:

Run the check on a small deterministic network in double precision. Large disagreement points to a wrong derivative, indexing error, mutation or unsuitable ε. A gradient check is slower than backpropagation, which is why it is a diagnostic, not the training method.

Scale

Compare the proposed step with the parameter scale, for example η‖g_ℓ‖/(‖θ_ℓ‖ + ε). There is no universal correct band, but orders-of-magnitude differences across layers are actionable. A correct direction with a negligible step produces apparent stasis. An excessive step can increase the loss even though the gradient is right.

Freshness

Bind the loss, tape, batch identity, stochastic seed and parameter version to one pass identifier. In synchronous code this can be implicit. In pipeline parallelism, gradient accumulation, rematerialisation or custom distributed training it should be testable. The invariant is simple: the backward pass must differentiate the computation that produced the declared loss.

"""Credit-assignment lab for Paper 34.

Builds a small multilayer network with NumPy only, implements the forward
pass and backpropagation by hand, checks the analytical gradient against a
finite difference, and compares four credit conditions on synthetic XOR data.

Run:
    python paper34_credit_assignment_lab.py

Requires:
    Python 3.10+
    numpy
"""
from __future__ import annotations

from dataclasses import dataclass
from typing import Dict, Iterable, Literal, Tuple

import numpy as np

Mode = Literal["exact", "severed", "reversed", "saturated"]


def stable_sigmoid(x: np.ndarray) -> np.ndarray:
    """Numerically stable logistic function."""
    positive = x >= 0
    out = np.empty_like(x, dtype=np.float64)
    out[positive] = 1.0 / (1.0 + np.exp(-x[positive]))
    exp_x = np.exp(x[~positive])
    out[~positive] = exp_x / (1.0 + exp_x)
    return out


def binary_cross_entropy_with_logits(logits: np.ndarray, y: np.ndarray) -> float:
    """Mean binary cross-entropy, computed without unstable log(sigmoid(.)) calls."""
    terms = np.maximum(logits, 0.0) - logits * y + np.log1p(np.exp(-np.abs(logits)))
    return float(np.mean(terms))


def make_xor(n: int = 512, seed: int = 7) -> Tuple[np.ndarray, np.ndarray]:
    """Synthetic quadrant-XOR dataset. No external or client data is used."""
    rng = np.random.default_rng(seed)
    x = rng.uniform(-1.0, 1.0, size=(n, 2))
    y = ((x[:, 0] * x[:, 1]) > 0.0).astype(np.float64).reshape(-1, 1)
    return x, y


@dataclass
class Tape:
    """Values saved from one forward pass and consumed by its matching backward pass."""

    pass_id: int
    x: np.ndarray
    z1: np.ndarray
    h1: np.ndarray
    z2: np.ndarray
    h2: np.ndarray
    logits: np.ndarray


class TinyMLP:
    """A 2 -> 8 -> 8 -> 1 network with manually implemented backpropagation."""

    parameter_names = ("W1", "b1", "W2", "b2", "W3", "b3")

    def __init__(self, mode: Mode = "exact", seed: int = 3) -> None:
        self.mode = mode
        self.pass_counter = 0
        rng = np.random.default_rng(seed)

        if mode == "saturated":
            # Deliberately poor choice: large sigmoid pre-activations create flat derivatives.
            self.activation = "sigmoid"
            self.W1 = rng.normal(0.0, 20.0, size=(2, 8))
            self.W2 = rng.normal(0.0, 20.0, size=(8, 8))
            self.W3 = rng.normal(0.0, 1.0, size=(8, 1))
        else:
            self.activation = "tanh"
            self.W1 = rng.normal(0.0, 0.18, size=(2, 8))
            self.W2 = rng.normal(0.0, 0.35, size=(8, 8))
            self.W3 = rng.normal(0.0, 0.35, size=(8, 1))

        self.b1 = np.zeros((1, 8))
        self.b2 = np.zeros((1, 8))
        self.b3 = np.zeros((1, 1))

    def _activate(self, z: np.ndarray) -> np.ndarray:
        return np.tanh(z) if self.activation == "tanh" else stable_sigmoid(z)

    def _activation_derivative_from_output(self, h: np.ndarray) -> np.ndarray:
        return 1.0 - h * h if self.activation == "tanh" else h * (1.0 - h)

    def forward(self, x: np.ndarray) -> Tuple[np.ndarray, Tape]:
        self.pass_counter += 1
        z1 = x @ self.W1 + self.b1
        h1 = self._activate(z1)
        z2 = h1 @ self.W2 + self.b2
        h2 = self._activate(z2)
        logits = h2 @ self.W3 + self.b3
        return logits, Tape(self.pass_counter, x, z1, h1, z2, h2, logits)

    def backward(self, tape: Tape, y: np.ndarray, expected_pass_id: int) -> Dict[str, np.ndarray]:
        if tape.pass_id != expected_pass_id:
            raise RuntimeError(
                f"stale activation tape: loss came from pass {expected_pass_id}, "
                f"but backward received pass {tape.pass_id}"
            )

        n = tape.x.shape[0]
        probability = stable_sigmoid(tape.logits)
        d_logits = (probability - y) / n

        gW3 = tape.h2.T @ d_logits
        gb3 = np.sum(d_logits, axis=0, keepdims=True)

        d_h2 = d_logits @ self.W3.T
        d_z2 = d_h2 * self._activation_derivative_from_output(tape.h2)
        gW2 = tape.h1.T @ d_z2
        gb2 = np.sum(d_z2, axis=0, keepdims=True)

        d_h1 = d_z2 @ self.W2.T
        d_z1 = d_h1 * self._activation_derivative_from_output(tape.h1)
        gW1 = tape.x.T @ d_z1
        gb1 = np.sum(d_z1, axis=0, keepdims=True)

        if self.mode == "severed":
            # The output layer still learns, but no credit reaches hidden transformations.
            gW1 *= 0.0
            gb1 *= 0.0
            gW2 *= 0.0
            gb2 *= 0.0
        elif self.mode == "reversed":
            # The hidden layers receive a signal with the wrong direction.
            gW1 *= -1.0
            gb1 *= -1.0
            gW2 *= -1.0
            gb2 *= -1.0

        return {"W1": gW1, "b1": gb1, "W2": gW2, "b2": gb2, "W3": gW3, "b3": gb3}

    def step(self, gradients: Dict[str, np.ndarray], learning_rate: float) -> None:
        for name in self.parameter_names:
            setattr(self, name, getattr(self, name) - learning_rate * gradients[name])

    @staticmethod
    def accuracy(logits: np.ndarray, y: np.ndarray) -> float:
        return float(np.mean((logits >= 0.0) == (y >= 0.5)))

    def parameter_items(self) -> Iterable[Tuple[str, np.ndarray]]:
        for name in self.parameter_names:
            yield name, getattr(self, name)


def gradient_check(model: TinyMLP, x: np.ndarray, y: np.ndarray, epsilon: float = 1e-5) -> float:
    """Check one W1 element against a centred finite difference."""
    logits, tape = model.forward(x)
    analytical = model.backward(tape, y, expected_pass_id=tape.pass_id)["W1"][0, 0]

    original = float(model.W1[0, 0])
    model.W1[0, 0] = original + epsilon
    plus, _ = model.forward(x)
    loss_plus = binary_cross_entropy_with_logits(plus, y)

    model.W1[0, 0] = original - epsilon
    minus, _ = model.forward(x)
    loss_minus = binary_cross_entropy_with_logits(minus, y)
    model.W1[0, 0] = original

    numerical = (loss_plus - loss_minus) / (2.0 * epsilon)
    denominator = max(1e-12, abs(analytical) + abs(numerical))
    return float(abs(analytical - numerical) / denominator)


def train(mode: Mode, steps: int = 1000) -> Tuple[TinyMLP, list[dict[str, float]]]:
    x, y = make_xor()
    model = TinyMLP(mode=mode)
    learning_rate = 0.4 if mode == "reversed" else 0.8
    history: list[dict[str, float]] = []

    for step in range(steps + 1):
        logits, tape = model.forward(x)
        loss = binary_cross_entropy_with_logits(logits, y)
        gradients = model.backward(tape, y, expected_pass_id=tape.pass_id)

        if step % 50 == 0:
            history.append(
                {
                    "step": float(step),
                    "loss": loss,
                    "accuracy": model.accuracy(logits, y),
                    "gW1": float(np.linalg.norm(gradients["W1"])),
                    "gW2": float(np.linalg.norm(gradients["W2"])),
                    "gW3": float(np.linalg.norm(gradients["W3"])),
                }
            )

        if step < steps:
            model.step(gradients, learning_rate)

    return model, history


def main() -> None:
    x, y = make_xor(n=32)
    check_model = TinyMLP(mode="exact")
    relative_error = gradient_check(check_model, x, y)
    print(f"finite-difference relative error: {relative_error:.3e}")
    print()
    print("condition   final_loss   accuracy   ||gW1||    ||gW2||    ||gW3||")

    for mode in ("exact", "severed", "reversed", "saturated"):
        _, history = train(mode)  # type: ignore[arg-type]
        last = history[-1]
        print(
            f"{mode:<10}  {last['loss']:>9.6f}   {last['accuracy']:>7.3f}   "
            f"{last['gW1']:>8.5f}   {last['gW2']:>8.5f}   {last['gW3']:>8.5f}"
        )

    # Demonstrate the freshness invariant without corrupting a training run.
    model = TinyMLP(mode="exact")
    logits_a, tape_a = model.forward(x)
    _, tape_b = model.forward(x)
    try:
        model.backward(tape_a, y, expected_pass_id=tape_b.pass_id)
    except RuntimeError as exc:
        print()
        print(f"freshness check: PASS ({exc})")


if __name__ == "__main__":
    main()

finite-difference relative error: 2.291e-09

condition   final_loss   accuracy   ||gW1||    ||gW2||    ||gW3||
exact        0.020271     0.998    0.00184    0.00429    0.00390
severed      0.691194     0.529    0.00000    0.00000    0.00015
reversed     0.691646     0.529    0.00263    0.00356    0.00470
saturated    0.255021     0.891    0.00576    0.00465    0.00538

freshness check: PASS (stale activation tape: loss came from pass 2, but backward received pass 1)