The inspector who learns from every forgery

Imagine a printmaker trying to reproduce a banknote. In the first workshop, the inspection rule is fixed: measure the border width, compare three colours and reject any serial number with the wrong checksum. The printmaker may dislike the rule, but the direction of improvement is stable. Make the border closer, mix the colours more accurately, repair the checksum.

In the second workshop, the inspector studies every new batch. When the printmaker fixes the border, the inspector begins noticing fibre texture. When the texture improves, the inspector learns a faint alignment cue. The printmaker is still improving against a measurable signal, yet the meaning of a good move changes after each inspection round. A direction that helped yesterday may be neutral today or actively harmful tomorrow.

This is the causal feature that makes adversarial generation unusual. The generator is not merely optimising a difficult loss. Its loss is supplied by another adaptive model. The critic learns which distinctions separate real examples from generated ones; the generator then changes its distribution to erase those distinctions. Each improvement alters the data on which the critic's next judgement is trained.

The analogy has limits. A GAN critic need not represent truth, legality or human perceptual judgement. It usually produces a differentiable score learned from finite samples, under a chosen architecture and objective. The generator does not understand the critic's reasons. It receives a gradient through the critic and changes parameters accordingly.

The central answer is that a GAN generator learns a sequence of critic-relative corrections. Under ideal assumptions, those corrections can drive its distribution towards the data distribution. Under finite capacity, finite samples and alternating updates, the same mechanism can create disappearing gradients, cycles, escalating oscillation or a narrow generator that exploits what the critic currently misses.

The architectural decision changes immediately. A low generator loss is not evidence that the generator has found a durable notion of quality. It may mean that the critic is weak, stale, overfitted or blind to missing modes. Conversely, a near-perfect critic may provide no useful route from generated samples to real ones. The object that must be evaluated is the coupled training process and the generator distribution it leaves behind.

Part I. The loss is another model

Let a generator G_θ turn a random latent input z into a sample x. Let a discriminator D_φ(x) return a number between zero and one. In the original formulation, the discriminator raises the score of real examples and lowers the score of generated examples, while the generator tries to make generated examples receive higher scores.[1]

p_data is the real-data distribution, p_z is the latent prior, θ are generator parameters and φ are discriminator parameters. Raising discriminator capacity or update frequency changes the function through which the generator receives its gradient.

A familiar supervised model also uses another model during training. A frozen image encoder may supply features; a fixed perceptual network may define a distance. The difference is decisive: those loss-producing components remain fixed. In adversarial training, φ changes in response to θ. The generator update at time t is approximately

but φ_t+1 is then updated using samples from the new generator. The next loss is a different function. This makes the order of updates, the relative learning rates, the critic's regularisation, the minibatches and even stale optimiser state part of what the generator learns from.

What the ideal critic supplies

For a fixed generator and unrestricted discriminator, the original paper derives an optimal discriminator:

Where real density dominates, the score approaches one; where generated density dominates, it approaches zero. The logit of this discriminator is a density-ratio signal. If the discriminator could reach this best response and the generator distribution itself could be updated in function space, the resulting envelope corresponds to minimising the Jensen-Shannon divergence, with equality at p_G = p_data.[1] The later f-GAN framework generalised this view by showing how different adversarial objectives estimate different f-divergences.[2]

That boundary matters because training does not normally solve the inner discriminator problem exactly, then optimise an exact generator objective. It takes a few noisy steps with both parameter sets. The critic is neither fixed nor always optimal. The generator is restricted to distributions reachable through G_θ. The data distribution is represented by a sample. The theorem identifies the desired equilibrium; it does not turn the route there into ordinary optimisation.

Three mathematical objects are easily confused here. The first is the best-response envelope, C(θ) = max_φ V(θ,φ). It asks what value the strongest available critic would assign to each fixed generator. The second is the realised value V(θ_t,φ_t), produced by the critic that training actually has at one moment. The third is the joint update field, which maps the current pair (θ_t,φ_t) to two parameter changes. Only the first resembles a generator-only landscape. The third governs the experiment.

Even if C has a clean minimum, the realised critic can lag behind its best response, overshoot it or fit a different finite-sample separator. The generator then follows ∇_θV at the realised φ, not necessarily ∇C. Reaching a low realised value may therefore say more about critic weakness than generator quality. The distinction between an equilibrium objective and an update trajectory is the conceptual hinge of GAN analysis.

The original minimax generator loss can also saturate when the discriminator confidently rejects poor early samples. The commonly used non-saturating alternative instead maximises log D(G(z)), preserving the same intended equilibrium while delivering stronger early gradients.[1] This is a practical improvement, yet it also makes a deeper point: changing the generator loss changes the vector field even when the named equilibrium stays the same.

A small worked example makes the signal concrete. Suppose real data occupy two shades, 30 and 80, with equal probability. A generator initially emits only shade 25. A trained critic can assign low scores around 25 and high scores near both real shades. Backpropagation may push generated values towards 30 because it is the nearest route that raises the critic score. Nothing in a one-sample update requires half the latent inputs to continue towards 80. The critic supplies local plausibility pressure; distributional coverage must emerge from the game, architecture and sampling process.

flowchart LR
  A["The inspector who learns from<br/>every forgery"] --> B["Part I. The loss is another model"]
  B --> C["Part II. A game can circle its<br/>answer"]
  C --> D["Part III. Stability methods<br/>edit the interaction"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#087A98,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Part II. A game can circle its answer

The strongest simple objection is that neural networks are non-convex anyway. Perhaps GAN instability is merely ordinary gradient descent struggling with a hard landscape. A useful negative control removes the networks and holds the difficulty almost absurdly low.

First optimise the fixed loss ½θ². Its gradient is θ, so gradient descent with learning rate η gives θ_t+1 = (1 − η)θ_t. For 0 < η < 2, repeated steps contract towards zero. There is one learner and one stationary objective.

When a moving judge earns no place

Suppose paired targets make a fixed reconstruction loss available, and a frozen perceptual encoder captures the distinctions that matter downstream. If that baseline matches the adversarial model on blinded human assessment, task utility and coverage while showing lower seed variance, the learned critic has not earned its interaction tax. It may still produce a fashionable sample texture, but the causal reason to accept a second learner is absent.

This is also an experiment that could weaken the paper's preferred explanation in a particular system. Hold generator architecture, optimiser, data order and compute constant; replace the adaptive critic with a fixed evaluator trained beforehand. If oscillation, collapse and seed sensitivity remain unchanged, then ordinary non-convexity or stochastic optimisation may dominate the observed failure. If those behaviours appear or intensify only when the evaluator adapts, the moving-target mechanism gains support. The comparison is more informative than naming the run “unstable”.

Now use the bilinear game L(θ,φ) = θφ. The generator minimises with respect to θ; the critic maximises with respect to φ. The unique equilibrium is still the obvious point θ = φ = 0. Simultaneous updates give

Square and add the two coordinates. The cross terms cancel:

Every step increases the distance from equilibrium by a factor √(1 + η²). Each player follows its correct instantaneous gradient, yet the pair spirals out. Alternating updates produce a different discretisation that can orbit rather than converge. The problem is not that the equilibrium is missing. It is that the joint field contains a rotational component.

The Jacobian of the continuous bilinear field is antisymmetric. Its eigenvalues are imaginary, which means the flow rotates instead of descending a scalar potential. Mescheder and colleagues used a closely related one-parameter generator and one-parameter discriminator, the Dirac-GAN, to show that unregularised GAN training need not converge when distributions lie on lower-dimensional supports.[6] Their counterexample does not claim every GAN diverges. It demonstrates that a correct equilibrium plus gradient updates is insufficient.

When the signal disappears

The original divergence picture creates another trap. Real images and generated images often concentrate near thin regions of a very high-dimensional pixel space. Early in training, those regions may barely overlap. A powerful discriminator can then separate them almost perfectly. For the original saturating loss, its output becomes locally flat around generated samples and the generator gradient approaches zero. The judge is accurate but unhelpful.[3]

Weakening the critic is not a complete solution. If it cannot see an obvious artefact, the generator can exploit the blind spot. If it memorises the training set, it may learn sharp boundaries that do not generalise. If it is updated too slowly, the generator optimises against an obsolete surface. The critic must be informative without becoming either omniscient on the sample or irrelevant to the next move.

Why one plausible sample is not coverage

Consider a second thought experiment. The real distribution occupies eight islands. A generator receives a high critic score for reaching any one of them. If early stochastic updates make island four slightly easier, more latent inputs move there. The critic learns to reject that crowd, but the generator may already have mapped broad latent regions into the same basin. The pursuit can repeat among a few islands while ignoring the rest.

This is mode collapse: generated examples can look individually plausible while the generator distribution covers too little of the data. A discriminator that sees one sample at a time may detect poor fidelity without directly observing a batch-level coverage failure. PacGAN changes only this causal feature by showing the discriminator packs of samples, making certain collapsed distributions easier to distinguish.[10] Unrolled GANs instead let a generator update anticipate several future discriminator steps, reducing exploitation of the current critic in the studied settings.[9]

A useful control is to freeze the critic after it has learned an initial separator. The generator now faces a stationary differentiable loss, so cycling caused by mutual adaptation disappears. Training may look smooth and the generator score may improve quickly. Yet the generator can discover a narrow adversarial route through that frozen surface, concentrating on examples that retain a high score while ignoring distinctions the critic would have learned from later batches.

This control separates two desirable properties that are often bundled together. A fixed judge gives predictable optimisation; an adaptive judge can repair discovered blind spots. The former can be stable and wrong, while the latter can be informative and unstable. A strong design needs enough critic adaptation to keep the score relevant, plus enough damping, time-scale separation or lookahead to keep the pursuit observable. Training calmness and critic adequacy are independent claims.

The negative control now has force. Fixed quadratic descent converges under the same numerical idea, while the minimal game can rotate. Real GANs add non-convexity, stochastic gradients, finite data and representation limits, so no toy proof explains every failure. It does establish that adaptive opposition introduces a distinct failure source that ordinary loss curves do not diagnose.

The hard failure boundary is critic observability. If the critic's inputs and representation cannot distinguish a required property, the generator receives no pressure to learn it. If the property concerns provenance, legality, causal validity or downstream safety rather than resemblance to the training sample, an adversarial image score cannot authorise it. Better optimisation cannot recover a criterion that the game never contains.

Part III. Stability methods edit the interaction

GAN stabilisation is often presented as a shelf of tricks. A more useful view asks which part of the joint field each intervention changes. Methods that solve different pathologies should not be treated as interchangeable.

Change the signal geometry

Wasserstein GAN replaces the original divergence construction with a critic constrained to a Lipschitz function class. Under its assumptions, the resulting Wasserstein-1 objective changes continuously as separated distributions move, so distance can remain informative before supports overlap.[4] Weight clipping was a crude way to enforce the constraint. WGAN-GP penalised deviations of the critic gradient norm from one along interpolated samples and improved empirical training in its experiments.[5]

This does not turn every implementation into a convergent system. Mescheder and colleagues showed that WGAN and WGAN-GP with a finite number of critic steps do not universally converge even on their simple counterexample, while instance noise and zero-centred gradient penalties converge locally under the stated conditions.[6] The appropriate conclusion is bounded: smoother distances and gradient controls can repair specific signal and local-stability failures; names do not substitute for trajectory evidence.

Add damping near the data

The R1 penalty adds ½γ E_x∼pdata||∇_xD(x)||² to the discriminator objective. It discourages unnecessary discriminator slope around real data. In the local analysis, this moves troublesome Jacobian eigenvalues away from the imaginary axis and introduces contraction.[6] The lab's damped bilinear game is an intentionally simpler analogue: penalising critic magnitude removes energy from a rotational field.

Regularisation has a cost. Too little may leave oscillation; too much can erase useful distinctions. A gradient penalty also consumes computation and samples only selected regions of input space. The parameter is part of the learned game, not a generic safety margin.

Separate time scales or look ahead

Two Time-Scale Update Rule uses different stochastic-approximation rates for generator and discriminator. Under assumptions including stable local attractors, bounded iterates and suitable learning-rate schedules, Heusel and colleagues proved convergence to a stationary local Nash equilibrium and reported improved empirical behaviour in their studied models.[7] The practical lesson is not simply “train the critic more”. What matters is whether one learner changes slowly enough for the other's response to remain meaningful.

Extragradient computes a provisional move, evaluates the field there, then uses that lookahead to correct the original point. In the bilinear game, this converts an expanding spiral into contraction. Unrolling applies a related idea specifically to the generator by differentiating through future discriminator optimisation.[9] These methods spend additional compute to estimate how today's move changes tomorrow's opponent.

Control what the critic can memorise

Limited data allow a discriminator to memorise real examples and leave the generator with brittle gradients. Adaptive Discriminator Augmentation applies augmentations to both real and generated images while adjusting their strength to limit overfitting; the reported experiments improved data efficiency without changing the generator's inference path.[13] Architectural work such as StyleGAN2 and StyleGAN3 addressed characteristic image and aliasing artefacts through changes to normalisation, generator design and signal processing.[14][15] These advances show that game stability and representation design interact. They do not make adversarial objectives self-validating.

State which kind of stability is being claimed

Numerical stability means computations remain finite: no exploding activations, NaNs or optimiser states. Dynamical stability means the joint update trajectory approaches a bounded attractor or equilibrium rather than cycling or diverging. Distributional adequacy means the final generator has acceptable fidelity, coverage and memorisation behaviour for the task. A system can satisfy any one while failing the others.

Gradient clipping may prevent numerical explosion without repairing a rotational field. A zero-centred penalty may create local contraction while the critic remains blind to a rare mode. Early stopping may capture one attractive checkpoint even though neighbouring seeds collapse. A low-variance training curve can arise because both players have stopped learning. For this reason, every stability claim should name the state variables observed, the perturbations applied, the time horizon and the external generator criteria.

Recovery also belongs in the claim. If a critic spike appears, can training resume from a checkpoint without reproducing the same collapse? Does reducing the generator rate recover coverage, or only lower losses? Does a changed data mix invalidate optimiser state? A reproducible recovery path is stronger evidence than one uninterrupted successful run.

The GAN stability lab

The accompanying artefact is deliberately small enough to inspect. It contains a fixed-loss control, the bilinear game under four update rules and a stochastic eight-mode categorical GAN. The bounded configuration uses moderate rates, two critic steps and critic regularisation. The stress configuration intentionally combines a tiny batch, no regularisation and extreme rates. It is designed to collapse, not to represent a sensible default.

Across ten deterministic seeds, the bounded configuration's worst mode mass remained above 0.119, its worst Jensen-Shannon divergence was 0.000109, and its mean effective mode count was 7.998 out of eight. The stress configuration's mean effective mode count was 1.000, with one mode receiving almost all probability. These are measured synthetic results from this artefact, not evidence about image GANs.

#!/usr/bin/env python3
"""GAN stability lab for 'Adversarial Generation and the Moving Target'.

The lab uses only Python's standard library. It contains two synthetic tests:

1. A matched one-dimensional control and game:
   * fixed quadratic descent (stationary loss),
   * simultaneous bilinear descent-ascent,
   * alternating bilinear descent-ascent,
   * extragradient,
   * critic damping.

2. An eight-mode discrete GAN:
   * a bounded, regularised configuration that covers all modes,
   * an intentionally aggressive stress configuration that collapses.

The purpose is causal diagnosis, not image-quality benchmarking. Run:

    python gan_stability_lab.py --output-dir gan_stability_outputs

The command writes JSON summaries and CSV trajectories and exits non-zero if
its positive and negative controls do not show the expected qualitative result.
"""

from __future__ import annotations

import argparse
import csv
import json
import math
import random
import statistics
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Iterable, Sequence

EPSILON = 1e-12


@dataclass(frozen=True)
class Point:
    step: int
    generator: float
    critic: float
    radius: float


@dataclass(frozen=True)
class ModeRun:
    seed: int
    js_divergence: float
    entropy: float
    effective_modes: float
    minimum_mass: float
    maximum_mass: float
    probabilities: list[float]


def _radius(generator: float, critic: float) -> float:
    return math.hypot(generator, critic)


def simulate_fixed_quadratic(
    learning_rate: float = 0.2,
    steps: int = 200,
    start: float = 1.0,
) -> list[Point]:
    """Gradient descent on 0.5 * x^2, stored in the generator coordinate."""
    x = start
    trajectory = [Point(0, x, 0.0, abs(x))]
    for step in range(1, steps + 1):
        x -= learning_rate * x
        trajectory.append(Point(step, x, 0.0, abs(x)))
    return trajectory


def simulate_bilinear(
    method: str,
    learning_rate: float = 0.2,
    steps: int = 200,
    damping: float = 0.5,
    start: tuple[float, float] = (1.0, 1.0),
) -> list[Point]:
    """Simulate min_theta max_phi L(theta, phi) = theta * phi.

    The generator minimises and the critic maximises the same bilinear payoff.
    """
    theta, phi = start
    trajectory = [Point(0, theta, phi, _radius(theta, phi))]

    for step in range(1, steps + 1):
        if method == "simultaneous":
            theta_next = theta - learning_rate * phi
            phi_next = phi + learning_rate * theta
        elif method == "alternating":
            theta_next = theta - learning_rate * phi
            phi_next = phi + learning_rate * theta_next
        elif method == "extragradient":
            theta_lookahead = theta - learning_rate * phi
            phi_lookahead = phi + learning_rate * theta
            theta_next = theta - learning_rate * phi_lookahead
            phi_next = phi + learning_rate * theta_lookahead
        elif method == "damped":
            theta_next = theta - learning_rate * phi
            phi_next = phi + learning_rate * (theta - damping * phi)
        else:
            raise ValueError(f"Unknown bilinear method: {method}")

        theta, phi = theta_next, phi_next
        trajectory.append(Point(step, theta, phi, _radius(theta, phi)))

    return trajectory


def _sigmoid(value: float) -> float:
    if value >= 0:
        exp_neg = math.exp(-value)
        return 1.0 / (1.0 + exp_neg)
    exp_pos = math.exp(value)
    return exp_pos / (1.0 + exp_pos)


def _softmax(logits: Sequence[float]) -> list[float]:
    maximum = max(logits)
    shifted = [math.exp(value - maximum) for value in logits]
    normaliser = sum(shifted)
    return [value / normaliser for value in shifted]


def _sample_index(probabilities: Sequence[float], rng: random.Random) -> int:
    threshold = rng.random()
    cumulative = 0.0
    for index, probability in enumerate(probabilities):
        cumulative += probability
        if threshold <= cumulative:
            return index
    return len(probabilities) - 1


def _js_divergence(p: Sequence[float], q: Sequence[float]) -> float:
    midpoint = [(left + right) / 2.0 for left, right in zip(p, q)]

    def kl(left: Sequence[float], right: Sequence[float]) -> float:
        return sum(
            value * math.log((value + EPSILON) / (reference + EPSILON))
            for value, reference in zip(left, right)
            if value > 0.0
        )

    return 0.5 * kl(p, midpoint) + 0.5 * kl(q, midpoint)


def _entropy(probabilities: Sequence[float]) -> float:
    return -sum(
        probability * math.log(probability + EPSILON)
        for probability in probabilities
        if probability > 0.0
    )


def train_discrete_gan(
    *,
    seed: int,
    modes: int = 8,
    steps: int = 2_000,
    batch_size: int = 16,
    generator_learning_rate: float = 0.1,
    critic_learning_rate: float = 0.5,
    critic_steps: int = 2,
    critic_regularisation: float = 0.1,
    record_every: int = 20,
) -> tuple[ModeRun, list[dict[str, float]]]:
    """Train a discrete non-saturating GAN on a uniform mode distribution.

    The generator is a categorical softmax. The critic has one logit per mode.
    Real and generated minibatches supply stochastic critic gradients; the
    generator uses the exact gradient of -E_p[log D] under its current p.
    """
    if modes < 2:
        raise ValueError("modes must be at least 2")
    if min(steps, batch_size, critic_steps, record_every) <= 0:
        raise ValueError("steps, batch_size, critic_steps and record_every must be positive")

    rng = random.Random(seed)
    target = [1.0 / modes] * modes
    generator_logits = [rng.uniform(-0.03, 0.03) for _ in range(modes)]
    critic_logits = [0.0] * modes
    trajectory: list[dict[str, float]] = []

    for step in range(steps + 1):
        generator_probabilities = _softmax(generator_logits)

        if step % record_every == 0 or step == steps:
            entropy = _entropy(generator_probabilities)
            trajectory.append(
                {
                    "step": float(step),
                    "js_divergence": _js_divergence(generator_probabilities, target),
                    "entropy": entropy,
                    "effective_modes": math.exp(entropy),
                    "minimum_mass": min(generator_probabilities),
                    "maximum_mass": max(generator_probabilities),
                }
            )

        if step == steps:
            break

        for _ in range(critic_steps):
            generator_probabilities = _softmax(generator_logits)
            real_counts = [0] * modes
            fake_counts = [0] * modes
            for _sample in range(batch_size):
                real_counts[rng.randrange(modes)] += 1
                fake_counts[_sample_index(generator_probabilities, rng)] += 1

            for index in range(modes):
                score = _sigmoid(critic_logits[index])
                real_frequency = real_counts[index] / batch_size
                fake_frequency = fake_counts[index] / batch_size
                gradient = (
                    real_frequency * (1.0 - score)
                    - fake_frequency * score
                    - critic_regularisation * critic_logits[index]
                )
                critic_logits[index] += critic_learning_rate * gradient

        generator_probabilities = _softmax(generator_logits)
        critic_probabilities = [
            min(max(_sigmoid(value), EPSILON), 1.0 - EPSILON)
            for value in critic_logits
        ]
        sample_costs = [-math.log(value) for value in critic_probabilities]
        expected_cost = sum(
            probability * cost
            for probability, cost in zip(generator_probabilities, sample_costs)
        )
        for index in range(modes):
            gradient = generator_probabilities[index] * (
                sample_costs[index] - expected_cost
            )
            generator_logits[index] -= generator_learning_rate * gradient

    final_probabilities = _softmax(generator_logits)
    final_entropy = _entropy(final_probabilities)
    run = ModeRun(
        seed=seed,
        js_divergence=_js_divergence(final_probabilities, target),
        entropy=final_entropy,
        effective_modes=math.exp(final_entropy),
        minimum_mass=min(final_probabilities),
        maximum_mass=max(final_probabilities),
        probabilities=final_probabilities,
    )
    return run, trajectory


def _write_points(path: Path, trajectories: dict[str, Iterable[Point]]) -> None:
    with path.open("w", encoding="utf-8", newline="") as handle:
        writer = csv.writer(handle)
        writer.writerow(["method", "step", "generator", "critic", "radius"])
        for method, points in trajectories.items():
            for point in points:
                writer.writerow(
                    [method, point.step, point.generator, point.critic, point.radius]
                )


def _write_mode_trajectory(path: Path, rows: Sequence[dict[str, float]]) -> None:
    with path.open("w", encoding="utf-8", newline="") as handle:
        fieldnames = [
            "step",
            "js_divergence",
            "entropy",
            "effective_modes",
            "minimum_mass",
            "maximum_mass",
        ]
        writer = csv.DictWriter(handle, fieldnames=fieldnames)
        writer.writeheader()
        writer.writerows(rows)


def _aggregate_mode_runs(runs: Sequence[ModeRun]) -> dict[str, float]:
    return {
        "seeds": float(len(runs)),
        "mean_js_divergence": statistics.fmean(run.js_divergence for run in runs),
        "worst_js_divergence": max(run.js_divergence for run in runs),
        "mean_effective_modes": statistics.fmean(run.effective_modes for run in runs),
        "worst_minimum_mass": min(run.minimum_mass for run in runs),
        "worst_maximum_mass": max(run.maximum_mass for run in runs),
    }


def run_lab(output_dir: Path) -> dict[str, object]:
    output_dir.mkdir(parents=True, exist_ok=True)

    control = simulate_fixed_quadratic()
    simultaneous = simulate_bilinear("simultaneous")
    alternating = simulate_bilinear("alternating")
    extragradient = simulate_bilinear("extragradient")
    damped = simulate_bilinear(
        "damped", learning_rate=0.1, steps=200, damping=0.5
    )

    bilinear_trajectories = {
        "fixed_quadratic": control,
        "simultaneous": simultaneous,
        "alternating": alternating,
        "extragradient": extragradient,
        "damped": damped,
    }
    _write_points(output_dir / "bilinear_trajectories.csv", bilinear_trajectories)

    stable_runs: list[ModeRun] = []
    stress_runs: list[ModeRun] = []
    stable_trajectory: list[dict[str, float]] = []
    stress_trajectory: list[dict[str, float]] = []

    for seed in range(10):
        stable_run, stable_rows = train_discrete_gan(seed=seed)
        stress_run, stress_rows = train_discrete_gan(
            seed=seed,
            steps=1_000,
            batch_size=2,
            generator_learning_rate=10.0,
            critic_learning_rate=10.0,
            critic_steps=1,
            critic_regularisation=0.0,
            record_every=10,
        )
        stable_runs.append(stable_run)
        stress_runs.append(stress_run)
        if seed == 7:
            stable_trajectory = stable_rows
            stress_trajectory = stress_rows

    _write_mode_trajectory(output_dir / "stable_mode_trajectory.csv", stable_trajectory)
    _write_mode_trajectory(output_dir / "stress_mode_trajectory.csv", stress_trajectory)

    summary: dict[str, object] = {
        "experiment": "Adversarial Generation and the Moving Target: GAN stability lab",
        "assumptions": [
            "All examples are synthetic and low dimensional.",
            "The bilinear game isolates game rotation rather than image modelling.",
            "The eight-mode stress test intentionally uses extreme learning rates and a tiny batch.",
            "Passing the lab does not establish stability for a neural image generator.",
        ],
        "fixed_and_bilinear": {
            method: {
                "initial_radius": points[0].radius,
                "final_radius": points[-1].radius,
                "minimum_radius": min(point.radius for point in points),
                "maximum_radius": max(point.radius for point in points),
            }
            for method, points in bilinear_trajectories.items()
        },
        "stable_eight_mode": {
            "configuration": {
                "steps": 2_000,
                "batch_size": 16,
                "generator_learning_rate": 0.1,
                "critic_learning_rate": 0.5,
                "critic_steps": 2,
                "critic_regularisation": 0.1,
            },
            "aggregate": _aggregate_mode_runs(stable_runs),
            "runs": [asdict(run) for run in stable_runs],
        },
        "stress_eight_mode": {
            "configuration": {
                "steps": 1_000,
                "batch_size": 2,
                "generator_learning_rate": 10.0,
                "critic_learning_rate": 10.0,
                "critic_steps": 1,
                "critic_regularisation": 0.0,
            },
            "aggregate": _aggregate_mode_runs(stress_runs),
            "runs": [asdict(run) for run in stress_runs],
        },
    }

    # Positive and negative controls. These thresholds are deliberately wide;
    # they test qualitative mechanisms, not benchmark leadership.
    fixed = summary["fixed_and_bilinear"]  # type: ignore[assignment]
    assert fixed["fixed_quadratic"]["final_radius"] < 1e-12  # type: ignore[index]
    assert fixed["simultaneous"]["final_radius"] > 20.0  # type: ignore[index]
    assert fixed["alternating"]["minimum_radius"] > 0.8  # type: ignore[index]
    assert fixed["extragradient"]["final_radius"] < 0.05  # type: ignore[index]
    assert fixed["damped"]["final_radius"] < 0.05  # type: ignore[index]

    stable_aggregate = summary["stable_eight_mode"]["aggregate"]  # type: ignore[index]
    stress_aggregate = summary["stress_eight_mode"]["aggregate"]  # type: ignore[index]
    assert stable_aggregate["worst_js_divergence"] < 0.002  # type: ignore[index]
    assert stable_aggregate["worst_minimum_mass"] > 0.10  # type: ignore[index]
    assert stress_aggregate["mean_js_divergence"] > 0.40  # type: ignore[index]
    assert stress_aggregate["worst_maximum_mass"] > 0.99  # type: ignore[index]

    summary["checks"] = {
        "fixed_loss_converges": True,
        "simultaneous_game_diverges": True,
        "alternating_game_cycles": True,
        "extragradient_contracts": True,
        "critic_damping_contracts": True,
        "bounded_eight_mode_run_covers_all_modes": True,
        "aggressive_stress_run_collapses": True,
    }

    with (output_dir / "summary.json").open("w", encoding="utf-8") as handle:
        json.dump(summary, handle, indent=2)
        handle.write("\n")

    return summary


def main() -> int:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument(
        "--output-dir",
        type=Path,
        default=Path("gan_stability_outputs"),
        help="Directory for summary.json and CSV trajectories.",
    )
    args = parser.parse_args()

    summary = run_lab(args.output_dir)
    print(json.dumps(summary["checks"], indent=2))
    print(f"Wrote results to {args.output_dir.resolve()}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Part IV. Evidence must be outside the game

A generator and critic can agree for the wrong reason. The critic may have forgotten a feature, exhausted its capacity or overfitted the available sample. A loss near the nominal equilibrium therefore does not prove that the generator distribution is faithful. Release evidence must come from instruments that the generator did not train to fool.

Fréchet Inception Distance compares the mean and covariance of real and generated examples in an Inception feature space.[7] It is useful for coarse distributional comparison, yet it compresses each feature distribution to two moments and inherits the chosen encoder. It does not by itself prove semantic coverage, absence of memorisation or fitness for a downstream task. Large-scale controlled comparisons also found substantial sensitivity to hyperparameter budget and seeds, and no tested GAN variant consistently dominated the non-saturating baseline across their setup.[8]

Precision and recall style measures separate two questions that a single score can blend. Precision asks whether generated examples lie near the data manifold; recall asks whether generated examples cover the data manifold.[11][12] They remain estimator-dependent, but they expose the central adversarial trade: sharp samples can coexist with missing variety.

The evaluator can become another moving target if it is repeatedly consulted during tuning. Once researchers select architectures, checkpoints and seeds against the same holdout metric, that metric becomes part of the effective training loop even when no gradient passes through it. A defensible experiment therefore separates a tuning instrument from a final locked evaluation, records the number of trials and keeps a simple baseline under the same compute budget.

Memorisation needs its own test because distributional proximity can reward copied training examples. Nearest-neighbour inspection is only a start; useful audits compare train and holdout distances, examine duplicates under transformations and measure whether performance deteriorates when near-duplicates are removed. No single threshold proves originality. The point is to make copying a visible competing explanation for apparent fidelity.

A production-shaped worked scenario

The team first freezes a baseline classifier and a feature encoder before GAN tuning. It defines three data splits: training examples for the game, a holdout for distributional evaluation and a near-duplicate audit set for memorisation. It also keeps a non-adversarial augmentation baseline with the same downstream training budget. This prevents a beautiful sample grid from becoming the comparison.

Each GAN configuration receives the same tuning budget and at least five seeds. The team records generator and critic losses, gradient norms, critic accuracy, precision, recall, nearest-neighbour distance, per-defect coverage and downstream classifier change. It injects three failures: remove one defect family from generator minibatches, corrupt a fraction of critic labels and freeze the critic for a late interval. A useful monitoring stack should distinguish these causes rather than merely report that “training became unstable”.

The release criterion is not “FID improved”. It requires no material reduction in holdout recall, no suspicious nearest-neighbour concentration, acceptable per-family coverage, bounded seed variance and a downstream gain over the fixed augmentation baseline. The final generator remains a sample-producing component. It does not certify that a synthetic defect is physically possible or safe to use in an engineering decision.

This scenario reveals a practical advantage of adversarial generation: inference can be a single forward pass through the generator, and a learned critic can place pressure on details that are awkward to write as a pixel loss. Large adversarial systems have shown that this approach can scale to high-resolution, fast synthesis in specific settings.[16] The same scenario also reveals the price: the training signal, stability regime and evaluation budget become part of the model specification.

The adversarial training evidence card

The card below turns symptoms into discriminating tests. It is deliberately mechanism-first. Applying every stabiliser at once may improve a score while leaving the causal reason unknown and the next domain change unmanageable.

A final selection rule follows. Choose a GAN when an adaptive critic captures a quality distinction that fixed losses or simpler augmentations miss, when fast direct sampling matters, and when the team can afford multi-seed, distribution-level evaluation. Prefer another mechanism when likelihood, calibrated uncertainty, broad mode coverage or straightforward optimisation carries the decision. The learned judge must earn its complexity against a fixed baseline.

Choose the game, then prove it stays playable

Adversarial generation replaces a hand-written discrepancy with an adaptive test. That is its intellectual power. A critic can discover texture, structure and distributional differences that are difficult to encode in a fixed reconstruction loss. The generator can then learn through ordinary backpropagation even though no explicit likelihood is available.

The price is equally structural. The test changes because the generator changes. Ideal best-response analysis identifies a desirable equilibrium, but practical training follows a coupled stochastic field. That field may contain descent, rotation, saturation, overfitting and exploitation at the same time. A judge can be accurate yet provide no route forward; easy to fool yet produce calm losses; locally stabilised yet blind to missing modes.

The architecture decision is to specify a GAN as a training system, not merely a generator network. The specification includes the critic objective and representation, update ordering, learning-rate relationship, regularisation, augmentation, seed budget, stopping rule, recovery behaviour and frozen external evaluators. Removing any of these can change what the generator actually learns.

For research, vary one causal feature at a time. Start with a fixed-loss control. Add the adaptive critic. Test a known multimodal distribution. Inject a stale critic, a strong critic and a missing mode. Compare damping or lookahead against matched compute. Report distributions across seeds. A visually compelling run is an observation; a controlled contrast begins an explanation.

The moving target is not a defect to eliminate completely. Without a changing critic, adversarial generation loses the mechanism that makes it useful. The goal is narrower and more defensible: design a game whose feedback remains informative, whose dynamics are observable and recoverable, and whose final generator passes tests it never learned to game.