The missing causal channel

A prediction is a conditional claim. Given observations, assumptions and a proposed action, it estimates a future state, distribution or cost. A capacity model may estimate tomorrow’s request rate. A route model may estimate arrival time under a planned path. A language model may estimate the likely result of a tool call. These estimates can be exceptionally useful, yet none of them by itself changes the governed process.

Control begins when an action is selected to influence a variable. Open-loop control can be sophisticated: optimise a trajectory, allocate resources, schedule doses, choose a prompt, or compute a sequence of tool calls. The sequence may rest on an excellent world model. It is still open loop if the realised consequences of those actions cannot alter the sequence while it matters.

Feedback adds a consequence channel. A sensor or evaluator observes some effect of action. A comparator relates that observation to a reference, constraint or acceptable set. A controller maps the resulting discrepancy into a new intervention. The plant, environment or task responds, and the cycle repeats. Cybernetics made this circular organisation a general object of study across machines and organisms, while modern control theory supplied precise accounts of stability, estimation, robustness and performance.[2][4]

The counterfactual test

The cleanest test for genuine loop closure is counterfactual. Hold the initial prediction, initial plan and environment constant until a disturbance occurs. Then ask: if the measured consequence had been different, would a later action have changed? If not, the observation is telemetry, not feedback. A dashboard that no decision process reads does not close a loop. A retrospective evaluator that runs after the outcome becomes irreversible may improve a future policy, but it did not control the present episode.

The distinction is architectural rather than tribal. A neural predictor can sit inside a feedback controller. A rule-based thermostat is a controller despite having no learned model. A large language model that repeatedly observes tool results and revises its next action is operating in a feedback architecture, even if the model was trained by next-token prediction. Conversely, a policy learned by reinforcement learning can be deployed as a fixed open-loop sequence. What matters is the runtime relation among observation, discrepancy and action.

Reflexive World Models provide a recent hybrid example. A slower reinforcement-learning policy supplies a nominal trajectory in latent space; a faster adaptive controller changes actions when realised state diverges from the world model’s prediction. The separation is useful because it makes the feedback error explicit instead of asking retraining to absorb every disturbance. Its guarantees remain conditional on model accuracy, adequate control authority, bounded perturbations and a differentiable latent model. Learned references do not repeal control assumptions. They move those assumptions into a representation that must itself be audited.

Prediction remains part of the solution

Calling prediction “feedforward” does not diminish it. A feedforward path uses known or forecast conditions to act before error appears. In the opening case, pre-scaling twelve replicas was sensible because waiting for latency to rise would have incurred avoidable harm. The closed-loop system did not reject the forecast; it used the forecast as a baseline and feedback as insurance against residual uncertainty.

Prediction is feedforward when it drives action from anticipated causes rather than measured error. Feedforward can be fast and quiet because it need not wait for deviation. It fails when important causes are unknown, mismeasured or change after the plan is fixed. Feedback can handle some of those residuals, but it waits for evidence and therefore pays in delay, sensing cost and potential instability. Well-designed systems combine them rather than forcing a false choice.

Feedback is not the same as learning

A feedback controller can correct a trajectory without changing its own policy. A thermostat may use the same control law for years while continually correcting temperature. Learning changes a model, policy or controller from accumulated experience. The two can be nested: a fast control loop regulates the current episode, an adaptation loop tunes parameters across episodes, and a governance loop changes objectives or authority on a slower cadence.

This distinction matters for machine intelligence. Reinforcement learning uses evaluative consequences to improve behaviour during training, but a trained policy can later be deployed either with continuing state feedback or as a fixed sequence.[9] Likewise, asking a language model for a second answer creates another generation step, not necessarily a corrective loop. Correction requires evidence that discriminates better from worse, a rule that lets that evidence change the next action, and a terminal check that the relevant error actually fell.

Conflating control with learning creates two opposite mistakes. One is to assume that a non-learning controller cannot be intelligent because its gains are fixed. The other is to assume that a learning system is automatically well controlled because its policy adapts. An adaptive policy can still chase a poor reward signal, learn more slowly than the process changes, or destabilise an inner loop. Identify which parameters change, from what evidence, on which timescale, and under whose authority.

Build the loop from first principles

A useful feedback design starts by naming the controlled variable, not by choosing an algorithm. “Keep the service healthy” is not yet a control objective. “Keep p95 latency below 400 milliseconds while the queue remains below 2,000 requests and replicas stay within an approved cost envelope” is closer. It separates a reference or acceptable region from the measurements used to judge it.

The minimal discrete-time model below is deliberately plain. It is enough to expose the logic without pretending that every machine-intelligence system is linear:

Here x[t] is the process state at step t; u[t] is the intervention; d[t] is an external disturbance; y[t] is the measured output; n[t] is measurement noise; r[t] is the desired reference; and e[t] is the measured error. The coefficients a and b describe how the process carries state forward and responds to action. u_ff[t] is a predictive or planned feedforward action. Kp and Ki are proportional and integral gains.

The equation makes loop closure visible. The next action depends on an error created by the previous consequence. If the disturbance changes x, that change reaches y, changes e, and therefore changes u. Remove the path from y to u and the plan cannot respond to a disturbance that was not already encoded in u_ff.

Five design obligations

Regulation, tracking and optimisation

Three control tasks are often blurred. Regulation holds a variable near a fixed reference despite disturbance. Tracking follows a reference that changes over time. Optimisation chooses actions, and sometimes the reference itself, to trade performance against cost and constraints. The same feedback architecture can support all three, but their evidence and failure criteria differ.

A system can regulate the wrong variable perfectly. An engagement controller may hold clicks above target while trust declines; an autoscaler may hold latency below target by spending beyond budget; an answer improver may raise an internal score while factual support deteriorates. Stable convergence is therefore not sufficient for good governance. The controlled objective must be justified, harder constraints must dominate softer preferences, and any mechanism that changes the reference requires its own authority boundary.

It is useful to separate setpoint generation from setpoint regulation. A planner, optimiser or human policy process may choose the desired operating region. The feedback controller then keeps the process inside that region. This separation makes disagreements inspectable: failure may lie in choosing the target, estimating state, applying control, or verifying effect. Without it, a loop that faithfully pursues a bad target can be misdescribed as a control failure when it is actually an objective failure.

Observability is not controllability

A system may infer that it is failing yet lack the means to change the relevant state. It may also possess a powerful actuator while being unable to infer the state that matters. In control theory, observability concerns whether hidden state can be reconstructed from available outputs, while controllability concerns whether admissible inputs can move the state as required. State estimation, including the family of ideas associated with Kalman filtering, addresses the first problem; a controller addresses the second.[7]

Observability is not controllability. An evaluator that detects hallucinated citations does not by itself give a model access to a trustworthy source. A monitor that sees a fraudulent transfer after settlement does not create a reversible payment rail. A queue metric that reveals overload is useless if the scaling API is rate-limited beyond the remaining latency budget. The boxes must be connected by real authority, not by vocabulary.

Why integral action matters

Proportional control reacts to present error. If a constant disturbance pushes a process away from target, proportional action often reduces the error but leaves a residual offset because some continuing error is needed to sustain the compensating action. Integral action accumulates error over time. It keeps increasing or decreasing the intervention until the persistent offset disappears, provided the loop remains stable and the actuator is not saturated.

Integral action stores unresolved error. This is a minimal form of memory directed at regulation. The internal model principle states, under its technical assumptions, that robust perfect regulation against a class of reference or disturbance signals requires the controller to embody dynamics capable of reproducing that class.[5] A controller that must reject constant bias therefore needs something like an integrator; one that must track a sinusoidal disturbance needs a richer internal structure. This is not a licence to say that every successful regulator contains a detailed world simulator. It is a precise claim about the structure needed for specified regulation tasks.

Worked example 1: a minimal thermal process

Take a normalised first-order thermal process whose temperature moves gradually towards the heater command. The target is 1.0. At time 20, a door is opened and introduces a constant cooling disturbance of 0.25. An open-loop schedule holds the original heater setting. A proportional controller reacts to the measured drop. A proportional-integral controller reacts and also accumulates the remaining offset.

In the synthetic run below, all three controllers use the same nominal plant model. Before the disturbance, their trajectories are similar. Afterwards, open loop settles well below target. Proportional feedback recovers part of the loss but retains an offset. PI feedback returns close to target after a transient. The result does not prove that PI is universally superior. It shows exactly what new capacity is purchased by consequence-sensitive memory in this disturbance class.

Model and regulator

A regulator usually benefits from a model because it must anticipate how action will propagate through the process. Conant and Ashby’s good regulator theorem is often summarised as “every good regulator must be a model of that system”. The formal result is narrower than the slogan and depends on the problem formulation.[6] The safe lesson is that regulation quality depends on representing distinctions relevant to the controlled outcome. A controller need not mirror every detail of the world; it must model enough of the action-consequence structure and disturbance class to choose effective interventions.

This clarifies why prediction alone and feedback alone are both incomplete descriptions. Prediction supplies anticipatory structure. Feedback supplies error-sensitive revision. A model-predictive controller repeatedly solves a constrained forecast problem from the latest estimated state, applies part of the plan, then solves again as new evidence arrives.[8] Its intelligence lies not in forecasting instead of feedback, but in placing forecasting inside a disciplined feedback cycle.

flowchart LR
  A["Two systems make the same forecast"] --> B["The missing causal channel"]
  B --> C["Build the loop from first<br/>principles"]
  C --> D["When correction becomes<br/>instability"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#7B4CD8,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

When correction becomes instability

Feedback is attractive because it reduces dependence on a perfect model. That robustness is purchased by coupling the controller to a living process. Once coupled, the controller’s own timing and gain become part of the dynamics. The system can now fail because it corrects too much, too late, on the wrong evidence, through a saturated actuator, while another controller is correcting the same variable.

Robustness is always relative to a declared uncertainty set. A loop may tolerate twenty per cent error in plant gain and a two-sample delay yet fail under a sign error, an unavailable actuator or a disturbance larger than its remaining authority. Production tests should therefore vary model parameters, disturbance shape, missing observations, timing and constraint activation rather than replaying only nominal traces. The claim to defend is not “the controller adapts”. It is “for these bounded departures from the model, the governed variables remain inside these limits or enter this safe fallback”.

Delay and gain

A controller acts on an error that describes the past. Sensor sampling, network transport, inference, authorisation, actuator warm-up and the plant’s physical response all consume time. If the process moves substantially during that interval, the correction may arrive after the error has changed sign. The controller then pushes in the wrong direction, sees the resulting overshoot later, and pushes back. Sufficient gain and delay create sustained oscillation or divergence.

Delay spends stability margin. Faster or stronger reaction is not automatically safer. In a service autoscaler, a replica may take ninety seconds to become ready while metrics arrive every ten seconds. A controller that treats each sample as independent can issue nine scale-out actions before the first one has any effect. In a tool-using agent, retrying after a timeout can duplicate a side effect whose success acknowledgement was merely delayed. Stability here includes both numerical dynamics and business semantics.

Noise and proxy error

Feedback acts on what is measured, not on what the designer intended to measure. High-frequency sensor noise can be converted into high-frequency control effort. A reward proxy can be optimised while the actual objective degrades. An automated reviewer can systematically prefer stylistic confidence over factual support. Filtering, hysteresis, deadbands, multi-signal checks and independent verification can reduce this risk, but each filter also adds delay or hides small real changes.

The deeper problem is specification. A target such as “maximise engagement” defines an attractor whether or not the designer meant to create one. The loop will search for actions that move the measured variable. Goodhart-style failure is therefore not external to control; it is control succeeding against an impoverished measurement. The remedy is not simply lower gain. It is a richer objective, hard constraints, audit signals and authority boundaries.

Saturation and windup

Every actuator has limits: maximum replicas, tool-call budget, safe dose, steering angle, human-review capacity or account permission. When the requested intervention exceeds that limit, the actual plant no longer follows the controller’s assumed law. An integral term may continue accumulating error even though the actuator cannot deliver more. When the constraint later clears, the stored integral drives a large overshoot.

Saturation changes the mathematics. Anti-windup logic pauses or back-calculates integral accumulation when the actuator is clamped. Rate limits constrain how quickly action may change. Supervisory modes can replace smooth control with safe fallback. These are not implementation details added after the “intelligent” controller. They are part of the real closed-loop system.

Interacting loops

A production system rarely has one controller. A client retries to maintain success rate; a gateway rate-limits to protect capacity; an autoscaler adds workers to reduce latency; a cost controller removes workers to meet budget; a model router shifts traffic to preserve quality. Each loop may be stable in isolation and unstable in combination. Their sampling intervals, thresholds and objectives can create limit cycles: retries increase load, throttling causes timeouts, timeouts cause retries, scaling arrives late, and cost logic then removes the new capacity.

Design therefore needs a loop inventory. Name which variable each controller observes and influences, its cadence, delay, priority, authority and fallback. Where two loops act on the same variable, establish hierarchy or coordinate them through a shared state model. Where a fast inner loop and slow outer loop coexist, keep their timescales sufficiently separated that the outer loop does not chase the inner loop’s transient.

The one-way transaction

Suppose an autonomous purchasing system submits a payment instruction and receives a network timeout. The underlying bank may have executed the transfer while the acknowledgement was lost. Reissuing the instruction is not a harmless correction. The observation “no response” is ambiguous between failure and unknown outcome; the actuator is not safely repeatable; and the consequence may be irreversible. A competent loop needs an idempotency key, an execution receipt or a state query that can distinguish these cases before acting again.

This example generalises. The value of evidence depends on its relation to action semantics. “Tool returned an error” is not enough if the tool may have partially committed. “Model produced a revised answer” is not enough if the source has not been checked. “Scale request accepted” is not enough if replicas failed readiness. Verification closes the operational loop by testing the world-state postcondition, not merely the controller’s intention.

Operate a production-shaped loop

Return to the model-serving cluster. A production controller must do more than map high latency to “add replicas”. It must distinguish demand from downstream failure, know the warm-up delay, account for in-flight scale actions, enforce cost and quota constraints, and avoid amplifying noisy percentiles. A useful design combines an anticipatory path with several feedback paths at different timescales.

Worked example 2: constrained model-serving autoscaling

The controlled outcome is a service envelope: p95 latency below 400 milliseconds, queue depth below 2,000, error rate below two per cent, and replicas between 20 and 120. The main manipulated variable is desired replica count. Measurements arrive every ten seconds. A new replica requires between sixty and ninety seconds to load weights and pass readiness. Removing a replica requires draining active requests.

The feedforward path forecasts request rate fifteen minutes ahead and computes a baseline capacity plan. The fast feedback path monitors queue growth and admission pressure every ten seconds. The slower path estimates effective service rate after replicas become ready and corrects model mismatch. A supervisory constraint layer blocks scale-in when error rate is elevated, limits replica changes per minute, reserves headroom for known regional failover, and moves to protected degradation when the quota ceiling is reached.

At 10:07, the retry storm raises arrivals from 8,000 to 13,500 requests per minute. The forecast baseline has already raised replicas from 60 to 72. Queue slope turns positive for three consecutive samples and p95 latency reaches 430 milliseconds. The controller calculates a further need of eighteen replicas but can add only twelve in the current rate window. It commands twelve, marks them as pending rather than available, and suppresses duplicate corrections until readiness evidence arrives. During warm-up, admission control sheds low-priority work within policy.

At 10:09, nine replicas pass readiness; three fail model loading. Effective service rate rises, queue slope falls, and latency returns below 400 milliseconds at 10:11. The verifier reconciles desired, scheduled, ready and serving replica counts. It then permits a smaller second correction for the remaining deficit. When retries subside, scale-in waits through a dwell period and drains workers gradually. The loop has not merely emitted scale commands. It has tracked their consequences through the state transition that matters.

What the controller comparison should measure

Average latency is an inadequate score because a loop can improve the average while creating severe oscillation, excessive control effort or repeated saturation. Compare controllers on a portfolio: integral absolute error, worst deviation, settling time, overshoot, control variation, time at actuator limits, constraint violations, cost, and recovery under model mismatch. Report the disturbance sequence and initial conditions so that the comparison can be reproduced.

Also compare against meaningful baselines. The open-loop plan is not a straw man; it should use the same predictive model and constraints available to the feedback design. A simple proportional controller should be included before a more elaborate adaptive or model-predictive method. The sealed-oven negative control should remain in the suite. A method that wins only when its own preferred disturbance is injected has not established a general advantage.

Controller selection is a systems decision

The choice is rarely “AI controller or no intelligence”. It is a choice among causal architectures. Open-loop feedforward is suitable when uncertainty is low, the process is well modelled, sensing is costly or noisy, and interventions are precomputed safely. Proportional feedback is useful when deviations are observable and quick correction matters, but a residual offset is acceptable. PI control suits persistent biases when saturation and delay are controlled. Model-predictive control is useful when multivariable constraints and anticipated dynamics matter enough to justify online optimisation. Human or supervisory control is necessary where objectives are disputed, consequences are hard to reverse, or authority cannot safely be delegated.

This admission test changes the order of work. Teams often begin with a powerful optimiser or an “agentic” loop and add controls later. The better order is to establish whether a valid feedback opportunity exists, then choose the least complex controller that meets the disturbance, constraint and recovery requirements. Feedback earns its complexity only when it demonstrably improves consequence under a declared uncertainty set.

Executable artefact

Controller comparison laboratory

The laboratory below simulates a normalised first-order plant. All controllers share the same nominal feedforward model. You can vary a persistent disturbance, plant-gain mismatch, sensor noise, loop delay and feedback gains. The chart and metrics are generated in the browser from synthetic data; no data leaves the page.

The positive preset introduces a cooling disturbance and model mismatch. Expected result: open loop retains a large offset, P reduces it, and stable PI approaches the target. The negative preset starts at the exact target with no process disturbance but adds noisy, delayed measurement and aggressive gains. Expected result: open loop remains quiet while feedback converts noise into unnecessary action and may oscillate.

state starts from the selected preset
for each time step:
    delayed_measurement = state_history[t - measurement_delay] + shared_seeded_noise[t]
    feedforward = target / nominal_gain
    P_action  = clamp(feedforward + Kp * error)
    PI_action = clamp(feedforward + Kp * error + Ki * integral_error)
    apply anti-windup when PI action is saturated
    update true plant with true_gain, current action and step disturbance
report error, post-disturbance deviation, effort and saturation for each controller

Reference

stateDiagram-v2
  [*] --> Claim
  state "The counterfactual test" as Claim
  state "Thought experiment 1, the<br/>invisible gust" as Test
  state "Prediction remains part of the<br/>solution" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Compact glossary

Actuator

The mechanism through which a controller changes the governed process.

Controlled variable

The measured or estimated quantity the loop is responsible for regulating.

Disturbance

An influence on the process not commanded by the controller.

Feedforward

Action based on anticipated causes or forecasts before measured error appears.

Feedback

A causal path by which observed consequence changes subsequent action.

Gain

The strength with which a controller responds to error or estimated state.

Observability

The ability to infer relevant hidden state from available outputs over time.

Postcondition

The world-state fact that must hold after an intervention, independently verified.

Saturation

The condition in which the actuator cannot deliver the controller’s requested action.

Stability

The property that bounded perturbations and internal dynamics do not produce unbounded or persistent harmful behaviour.