The path is the hidden curriculum

Begin with a source distribution p₀, such as a standard Gaussian, and a target data distribution p₁. A continuous generative model introduces distributions p_t for times t between zero and one. The individual sample state x_t evolves under a time-dependent velocity field v_t. Neural ordinary differential equations made this continuous-depth construction a practical modelling primitive, while continuous normalising flows connected the dynamics to density transformation and likelihood.[1][2]

The population must move consistently with the sample dynamics. The continuity equation supplies that conservation law. It says that density inside a small region changes because probability mass flows across its boundary. Mass is neither created nor destroyed by the deterministic ODE.

Thought experiment: identical endpoints, different journeys

Fix one source draw z and one target y. Path A moves linearly: x_t = (1 − t)z + t y. Its target velocity is the constant vector y − z. Path B reaches the same y, but rotates the intermediate state by an angle that rises and then returns to zero. The endpoints, training pairs and final distribution remain unchanged. Only the path changes.

A one-step Euler solver is exact for Path A when the learned field is constant along the trajectory. The same solver can miss Path B badly because it extrapolates the initial tangent across an entire curved journey. A higher-order solver or more time steps can repair the approximation. The endpoint modelling problem and the numerical transport problem are separable. A generator can represent the correct destination yet still be expensive or inaccurate to sample.

This thought experiment also limits a common claim. Predicting velocity is not itself a speed guarantee. Velocity is simply the derivative of the chosen path. If that derivative changes sharply with time or state, the solver must resolve the change. The executable experiment later in the article varies only this causal feature.

Conditional arrows become a marginal flow

If the exact marginal velocity u_t(x) were known at every intermediate state, flow matching could train a neural field v_θ by ordinary squared-error regression. The direct objective is conceptually simple:

The obstacle is that sampling from the marginal p_t and evaluating its velocity can be as difficult as the original generative problem. Conditional flow matching avoids this circularity. Draw a data example y, a source example z, and a time t. Construct a conditional intermediate state with a known formula, then regress against its known derivative. For the linear interpolation

The original flow-matching derivation shows that, under its regularity conditions, the conditional objective has the same parameter gradients as the intractable marginal objective.[5] The optimum of squared-error regression has an intuitive form:

Conditional flow matching converts an inaccessible population velocity into accessible per-example arrows. The conversion is powerful, but the conditional expectation exposes the main design debt. If many incompatible journeys meet at the same x and t, regression returns their average. That average can point somewhere no individual pair requested.

Worked example: an affine Gaussian transport

Let z be standard normal in one dimension and define a paired target y = 3 + 1.5z. The source is N(0, 1); the target is N(3, 1.5²). Choose the linear path. Substitution gives

Every source value has a deterministic paired target. There is no hidden disagreement about which way to move. At t = 0 the field is affine in x, and the exact solution reaches the target Gaussian at t = 1. This case is deliberately easy. It demonstrates that the training target can be derived without simulating the ODE and that a structured coupling can make the field identifiable.

Schedules are part of the target

The linear interpolation is only one member of a broader Gaussian conditional family. Write x_t = α(t)y + σ(t)z, where α controls the data contribution and σ controls the source-noise contribution. Differentiation gives the supervised target u_t = α′(t)y + σ′(t)z. Changing the schedule therefore changes the magnitude, direction and time concentration of the labels even when the endpoints remain fixed.

This has two operational consequences. First, time sampling and loss weighting allocate model capacity across the journey. Uniformly sampling t does not imply uniform difficulty: a short endpoint interval may contain the largest target norms or fastest field variation. Second, a parameterisation that is numerically well scaled for one α-σ schedule may be poorly scaled for another. Targets should be inspected by time, not only through one average loss.

A useful diagnostic bins held-out examples by t and records target norm, conditional target variance, model residual and endpoint sensitivity after a perturbation at that time. A schedule is successful when it distributes learnable transport work, not merely when its endpoint equations are elegant. This diagnostic also provides an early warning for stiff or under-trained regions before an expensive image-quality sweep.

Thought experiment: crossing pairs and lost identity

Now use two-dimensional pairs. Send (−1, −1) to (1, 1), and send (−1, 1) to (1, −1), both along straight conditional lines. At t = 0.5, both paths pass through (0, 0). The first asks for velocity (2, 2); the second asks for (2, −2). A deterministic field cannot assign two velocities to the same state and time. Squared-error regression returns their average, (2, 0).

This is not merely noisy supervision. It is a structural collision. A well-posed ODE has one future from one state and time, so trajectories cannot cross there and then depart with different velocities. Rectified flow interprets the learned ODE as a rewiring of the original coupling: the endpoint marginals can be preserved even when individual source-target identities change.[6]

This matters differently across tasks. In unconditional image generation, preserving which Gaussian noise point was originally paired with which image may have no semantic value. In unpaired translation, molecular transport, trajectories between observed cell states or any application claiming individual correspondence, rewiring is a substantive assumption. Distribution matching does not certify identity-preserving transport.

Coupling methods therefore belong to the model, even when they add no neural parameters. Conditional flow matching with minibatch optimal transport and multisample flow matching use non-trivial pairings to reduce target variance or crossings. Their reported experiments connect such couplings with simpler or straighter learned flows and better low-cost sampling.[7][8] Yet an optimal conditional line for sampled pairs does not automatically prove that the resulting marginal flow is the globally optimal transport map. The coupling approximation, minibatch size and problem geometry remain part of the evidence boundary.

flowchart LR
  A["The path is the hidden curriculum"] --> B["Conditional arrows become a<br/>marginal flow"]
  B --> C["Flow matching and denoising<br/>are neighbours"]
  C --> D["Geometry decides whether few<br/>steps work"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#7B4CD8,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Flow matching and denoising are neighbours

The most misleading comparison says that diffusion learns to denoise through many stochastic steps while flow matching learns one deterministic route. It mixes training target, probability path and sampler into one slogan. Each dimension can be varied separately.

Denoising diffusion probabilistic models define a forward corruption process that gradually turns data into noise, then learn a reverse generative process.[3] In continuous time, score-based models learn the score ∇_x log p_t(x), the direction in state space along which log density rises most rapidly. The score enables a reverse-time stochastic differential equation. The same marginal distributions also admit a deterministic probability-flow ODE.[4]

Flow matching starts from a prescribed probability path and trains the velocity field that transports its marginals. The original framework includes a family of Gaussian paths, including paths associated with diffusion constructions.[5] Stochastic interpolants make the shared structure even more explicit: one interpolating family can support deterministic transport equations and families of stochastic forward or backward dynamics.[10]

The scientifically useful contrast is not “flow versus diffusion”; it is which object is supervised, which path is induced, and which dynamics are integrated at inference. A diffusion model can be sampled deterministically. A flow-matching model can use a diffusion probability path. A network can predict noise, score, clean data or a velocity-like linear combination under different parameterisations. The output name alone does not identify the model family.

One diffusion path, two learned objects

The relation can be made concrete. Consider a forward stochastic differential equation dx = f(x, t)dt + g(t)dw. Its score is s_t(x) = ∇_x log p_t(x). Under the score-SDE construction, the associated probability-flow ODE has velocity

A score network can therefore be converted into a velocity for this specific path. Flow matching can instead regress the corresponding velocity directly from conditional path samples. The two objectives expose different errors and may weight time differently, but their samplers can target the same marginal evolution. This is why a fair experiment should preserve the probability path before attributing a result to score learning or velocity learning.

The equation also shows why “score points towards data” is too crude. The actual transport velocity includes the forward drift f, the diffusion scale g, the score and the chosen time direction. A larger score norm does not by itself mean a faster sample trajectory. The learned object becomes an action only after the governing dynamics specify how it is used.

Where the analogy breaks

It is helpful to imagine denoising as reading density geometry and flow matching as reading transport geometry. The analogy fails if it suggests a unique route from either object. Many velocity fields can generate the same marginal density evolution, especially when divergence-free components are possible. A score identifies a log-density gradient but needs the specified diffusion dynamics to become a reverse drift. The bridge equations, schedules and boundary conditions matter.

Likewise, “simulation-free training” has a bounded meaning. Flow matching can compute regression targets without solving the generative ODE inside each training example. Sampling still integrates an ODE. Exact likelihood, when required, still involves divergence accumulation and numerical error, as in continuous normalising flows. The solver has moved out of the training target, not out of the system.

Geometry decides whether few steps work

A numerical ODE solver never sees the whole trajectory at once. Euler's method evaluates the field at the current point and takes a tangent step. Higher-order methods sample several intermediate slopes. Adaptive solvers shrink the step when their local error estimate rises. All of them pay in neural function evaluations, commonly abbreviated NFE.

For an exactly constant velocity, one Euler step reaches the endpoint. When direction or speed changes, the same step extrapolates stale information. Research on ODE-based generators has related trajectory curvature to numerical truncation error and sampling cost.[9] More recent analysis introduces piecewise straightness measures and bounds Wasserstein sampling error using path geometry under stated assumptions.[12]

Curvature is not the only difficulty. A field can be nearly straight yet stiff, meaning small state changes cause large velocity changes in a narrow time region. It can have sharp endpoint behaviour. Classifier-free guidance or another conditioning intervention can alter the sampling field after training. Approximation error in the neural field can dominate solver error. “Straight” is a useful diagnostic, not a complete certificate of cheap generation.

Four mechanisms that shape low-NFE performance

First, coupling controls conditional ambiguity. Independent noise-data pairing is easy to sample but may produce crossing conditional lines and high variance in target velocities. Minibatch optimal transport, multisample coupling, or matched pairs generated by a teacher can reduce disagreement. The gain must be weighed against coupling cost, bias and any semantic meaning assigned to the pairing.

Second, the probability path controls local geometry. Linear conditional interpolants make each pair's target velocity constant, but the marginal field can still curve because many conditional paths are averaged. Diffusion paths change signal and noise scales according to a schedule. Curved paths can sometimes avoid low-density or semantically awkward regions. The shortest Euclidean line in latent coordinates need not be the easiest learned route.

Third, the trained field controls approximation regularity. Finite data, finite capacity and optimisation error mean the learned field differs from the prescribed target. Even a theoretically easy path can become irregular. Time weighting determines which regions receive model capacity. A low average regression loss can hide a narrow interval that dominates endpoint error.

Fourth, the sampler controls numerical error. One-step Euler, four-step Runge-Kutta and an adaptive solver are different algorithms, not interchangeable reporting details. A comparison by “steps” is unfair when each step uses a different number of model calls. Compare total NFE, wall-clock latency, memory traffic and output quality under identical guidance and precision.

Negative control: straightness may be the wrong explanation

A serious causal claim needs a result that could make the preferred explanation unnecessary. Rectified Diffusion argues that the benefit commonly attributed to rectified-flow straightness can arise from training on matched noise-sample pairs and learning a path that is well approximated to first order, even when that path is inherently curved. Its authors report that the flow-matching form and a particular velocity parameterisation are not essential to their rectification procedure.[11]

This does not invalidate path geometry. It changes the hypothesis. Instead of asking only whether trajectories look straight, test whether the field's one-step or few-step approximation error falls after pairing, while holding the path family and solver fixed. Then change the path while holding pairing fixed. If matched pairing improves low-NFE quality without straightening the path, coupling was the active mechanism. If a straighter path improves only Euler but not a higher-order solver at matched NFE, the gain was solver-specific.

Run the flow experiment

The companion artefact uses synthetic two-dimensional Gaussian samples. It fixes the source distribution, the exact endpoint map and the target distribution. The only experimental treatment is the path. The control path is a constant translation. The detour rotates the moving cloud by an angle that rises and returns to zero, so every sample reaches the same exact endpoint. Euler and fourth-order Runge-Kutta integration then estimate that endpoint at several NFE budgets.

The script is intentionally model-free. There is no neural approximation to blame. That makes it a strong negative case for the slogan that velocity prediction creates fast sampling. Even an exact velocity field can be expensive for a coarse solver when its trajectory bends.

The mean detour path length in the experiment is 2.038 times the direct translation length. Euler's detour RMSE falls from 4.902 at one NFE to 0.267 at 32 NFE. RK4 reaches 0.010 at 16 NFE and approximately 0.0007 at 32 NFE. These are synthetic measurements, not image-generation quality claims. They isolate the causal relation between path geometry, solver order and endpoint error.

"""Flow-path stress test for 'Flow Matching and Continuous Generative Transport'.

Python 3.10+. Standard library only.
All data, paths and thresholds are synthetic teaching fixtures.

The experiment keeps the source distribution, target distribution and exact endpoint
map fixed. It changes only the path between them, then measures numerical integration
error. This isolates path geometry from endpoint quality and model capacity.
"""
from __future__ import annotations

from dataclasses import dataclass
import math
import random
from typing import Callable, Iterable

Vec = tuple[float, float]
Velocity = Callable[[float, Vec], Vec]
Integrator = Callable[[Vec, Velocity, int], Vec]

SHIFT: Vec = (2.0, -1.0)
SEED = 7
SAMPLES = 512


def add(a: Vec, b: Vec) -> Vec:
    return (a[0] + b[0], a[1] + b[1])


def scale(s: float, a: Vec) -> Vec:
    return (s * a[0], s * a[1])


def rotate(theta: float, x: Vec) -> Vec:
    c, s = math.cos(theta), math.sin(theta)
    return (c * x[0] - s * x[1], s * x[0] + c * x[1])


def quarter_turn(x: Vec) -> Vec:
    return (-x[1], x[0])


def norm(x: Vec) -> float:
    return math.hypot(x[0], x[1])


def straight_velocity(_: float, __: Vec) -> Vec:
    return SHIFT


def detour_velocity(amplitude: float) -> Velocity:
    """Return the exact velocity of a rotating detour with unchanged endpoints."""

    def velocity(t: float, x: Vec) -> Vec:
        angle = amplitude * math.sin(math.pi * t)
        angular_speed = amplitude * math.pi * math.cos(math.pi * t)
        rotational = scale(angular_speed, quarter_turn(x))
        translational = rotate(angle, SHIFT)
        return add(rotational, translational)

    return velocity


def exact_state(t: float, z: Vec, amplitude: float) -> Vec:
    """Analytic path: rotate(z + t*SHIFT), returning to zero rotation at t=1."""
    angle = amplitude * math.sin(math.pi * t)
    return rotate(angle, add(z, scale(t, SHIFT)))


def euler(z: Vec, velocity: Velocity, steps: int) -> Vec:
    x = z
    h = 1.0 / steps
    for k in range(steps):
        t = k * h
        x = add(x, scale(h, velocity(t, x)))
    return x


def rk4(z: Vec, velocity: Velocity, steps: int) -> Vec:
    x = z
    h = 1.0 / steps
    for k in range(steps):
        t = k * h
        k1 = velocity(t, x)
        k2 = velocity(t + h / 2, add(x, scale(h / 2, k1)))
        k3 = velocity(t + h / 2, add(x, scale(h / 2, k2)))
        k4 = velocity(t + h, add(x, scale(h, k3)))
        weighted = add(add(k1, scale(2, k2)), add(scale(2, k3), k4))
        x = add(x, scale(h / 6, weighted))
    return x


def endpoint_rmse(samples: Iterable[Vec], velocity: Velocity,
                  integrator: Integrator, steps: int) -> float:
    squared_errors: list[float] = []
    for z in samples:
        estimate = integrator(z, velocity, steps)
        target = add(z, SHIFT)
        squared_errors.append(
            (estimate[0] - target[0]) ** 2 + (estimate[1] - target[1]) ** 2
        )
    return math.sqrt(sum(squared_errors) / len(squared_errors))


def path_length_ratio(z: Vec, amplitude: float, slices: int = 2000) -> float:
    """Midpoint estimate of path length divided by direct endpoint distance."""
    velocity = detour_velocity(amplitude)
    h = 1.0 / slices
    length = 0.0
    for k in range(slices):
        t = (k + 0.5) * h
        x = exact_state(t, z, amplitude)
        length += h * norm(velocity(t, x))
    return length / norm(SHIFT)


@dataclass(frozen=True)
class Result:
    solver: str
    path: str
    steps: int
    nfe: int
    rmse: float


def run() -> None:
    rng = random.Random(SEED)
    samples = [(rng.gauss(0, 1), rng.gauss(0, 1)) for _ in range(SAMPLES)]
    amplitude = 1.1
    detour = detour_velocity(amplitude)

    results: list[Result] = []
    for solver_name, integrator, calls_per_step in (
        ("Euler", euler, 1),
        ("RK4", rk4, 4),
    ):
        for path_name, velocity in (
            ("straight", straight_velocity),
            ("detour", detour),
        ):
            for steps in (1, 2, 4, 8, 16, 32):
                results.append(
                    Result(
                        solver=solver_name,
                        path=path_name,
                        steps=steps,
                        nfe=calls_per_step * steps,
                        rmse=endpoint_rmse(samples, velocity, integrator, steps),
                    )
                )

    ratios = [path_length_ratio(z, amplitude) for z in samples]
    print("Same source, same endpoint map, different path geometry")
    print(f"Mean detour path-length ratio: {sum(ratios) / len(ratios):.3f}x")
    print()
    print(f"{'solver':<8} {'path':<9} {'steps':>5} {'NFE':>5} {'endpoint RMSE':>15}")
    for row in results:
        print(
            f"{row.solver:<8} {row.path:<9} {row.steps:>5} "
            f"{row.nfe:>5} {row.rmse:>15.6f}"
        )

    # Positive case: a straight constant field is exactly solved by one Euler step.
    assert endpoint_rmse(samples, straight_velocity, euler, 1) < 1e-12

    # Negative case: calling the target a velocity does not make a curved path cheap.
    assert endpoint_rmse(samples, detour, euler, 1) > 1.0

    # Solver control: a higher-order solver resolves the same curved field accurately.
    assert endpoint_rmse(samples, detour, rk4, 4) < 0.02

    # Path control: zero detour amplitude collapses exactly to the straight path.
    assert endpoint_rmse(samples, detour_velocity(0.0), euler, 1) < 1e-12

    print("\nAll positive, negative and control assertions passed.")


if __name__ == "__main__":
    run()

stateDiagram-v2
  [*] --> Claim
  state "Thought experiment: identical<br/>endpoints, different journeys" as Claim
  state "Worked example: an affine<br/>Gaussian transport" as Test
  state "Schedules are part of the target" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Build and evaluate a transport model

Worked example: a conditioned latent flow

Consider a production-shaped but synthetic text-conditioned image system. A fixed image encoder maps training images to continuous latent vectors y. A text encoder produces condition c. Draw Gaussian noise z, sample time t, and construct x_t = (1 − t)z + t y. A time-conditioned U-Net or transformer receives (x_t, t, c) and predicts y − z. At inference, draw fresh z, integrate the learned ODE, and decode its endpoint.

This description is short because the mechanism is short. The evidence burden is not. The encoder defines the geometry in which a straight line is being privileged. The text condition changes which data modes are relevant. Independent noise-data pairing can make conditional targets ambiguous. Guidance changes the field used at sampling. The decoder can turn a small latent endpoint error into a visible artefact or hide a large one. A latent flow is a compound system whose transport claims must be localised to the latent state actually modelled.

A matched experiment, not a model-family tournament

Use one dataset split, one latent encoder and decoder, one conditioning stack, and architectures with comparable capacity. Train a flow-matching model and a denoising baseline under declared compute budgets. Preserve a high-accuracy reference sampler for each model. Then sweep NFE and solver choices. At every point, record at least three kinds of evidence:

Transport evidence measures conditional target variance, field regression error by time, trajectory curvature or local truncation proxies, and endpoint deviation from a high-accuracy integration. Generative evidence measures fidelity, diversity, conditional compliance and memorisation with uncertainty. Operating evidence measures wall-clock latency, peak memory, batch sensitivity, numerical precision and failure rate on the intended hardware.

A low-NFE model should not be compared with a high-quality baseline using different guidance strength or a stronger decoder. Nor should one system be allowed adaptive high-order integration while the other is counted in nominal steps. Report model evaluations, not just solver steps. If distillation, reflow or teacher-generated matched pairs are used, include their training cost and lineage. A fast student may be operationally attractive even when its total creation cost is high; those are separate decisions.

Failure injection before release

A benchmark generated under nominal settings can conceal the exact regions that will fail in service. Deliberately perturb the sampler. Remove one intermediate time band from training, coarsen the time grid near each endpoint, raise guidance beyond its validated range, reduce numerical precision, and inject a transient non-finite field value. Record whether the system detects the deviation, falls back to a safer sampler, or quietly emits a plausible but altered result.

Separate four outcomes. A model failure persists even under high-accuracy integration. A solver failure shrinks when tolerance or order improves. A conditioning failure appears only for particular prompts, classes or guidance settings. A decoder failure emerges after a latent endpoint that otherwise satisfies the transport checks. Without this separation, an attractive output can hide a wrong latent trajectory, while an unattractive output can falsely condemn a correct flow.

Recovery should also be bounded. An adaptive retry is useful only when the trigger is observable and the maximum extra NFE is declared. Repeating an unstable integration without changing its conditions is not recovery. For consequential generation, retain a decision receipt containing model version, path and coupling version, sampler, time grid, precision, random seed, guidance, NFE and the postcondition checks applied to the result.

The transport stress test

The following instrument turns the argument into an architecture decision record. It is intentionally mechanism-first. A paper or system passes only when its proposed efficiency gain can be located in an observable change rather than inferred from the method label.

The instrument changes the order of work. Do not begin by asking whether to adopt flow matching. First ask whether a continuous latent path is appropriate, whether individual correspondence matters, and where the present sampler loses quality. Then run the coupling and path controls. Only after those results should the team select an objective, parameterisation and solver.

Compact glossary

Probability path

A time-indexed family of distributions connecting a source to a target.

Velocity field

A function assigning the instantaneous direction and speed to every state and time.

Continuous normalising flow

An invertible distribution transformation defined by an ODE, with density change governed by divergence.

Coupling

A joint distribution over source and target samples whose marginals equal the two endpoint distributions.

Score

The spatial gradient of log density, ∇_x log p_t(x), at a fixed time.

NFE

Number of neural function evaluations used by a sampler. It is more comparable than nominal solver steps.

Rectification or reflow

Retraining on pairs produced by an existing flow to obtain a coupling that may support easier few-step transport.

Stiffness

A numerical condition in which stable or accurate integration requires very small steps in some regions.

sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: The path is the hidden curriculum
  M->>E: Thought experiment: identical endpoints, different journeys
  E-->>M: compare rival explanations
  E->>D: Choose the path before the fashion
  D-->>Q: revise scope or act

Choose the path before the fashion

Flow matching earns its place by making continuous generative transport trainable through ordinary regression. It supplies tractable conditional arrows and, under the framework's conditions, recovers the marginal velocity that carries one distribution into another. That is a precise and valuable mechanism.

Its strongest lesson is not that denoising has been replaced. Diffusion paths live inside the broader construction, and score models can generate through deterministic probability-flow ODEs. The decisive variables are lower level: the coupling between endpoints, the intermediate probability path, the regularity of the learned field, and the numerical method used to integrate it.

The architecture decision changed by this argument is simple: treat flow matching as a transport hypothesis, not a speed label. Require a matched experiment that varies coupling, path and solver separately. Preserve endpoint semantics where they matter. Compare by total function evaluations and operating latency. Keep a high-accuracy trajectory as a reference. Reject one-step claims that disappear under a different pairing or solver control.

The resulting question is more demanding but more useful. Do not ask, “Is flow matching faster?” Ask: Which path makes the required probability mass movement unambiguous to learn, cheap to integrate, and faithful to the outcome we actually care about? That question can fail in an experiment. Because it can fail, it can also accumulate defensible evidence.