Three control objects

Plans, policies and replanning are often treated as different implementations of the same idea. They answer different questions. A plan answers, “given this starting condition and goal, which linked actions should occur?” A policy answers, “given the state I recognise now, which action should I choose?” Replanning answers, “given new evidence, which earlier commitments should be retained, repaired or discarded?” Confusing them hides the source of both robustness and failure.

A plan stores causal dependence

Take the route S → A → B → G. It is more than a list. The action from B to G assumes the courier reaches B, the edge remains open, the parcel is still destined for G, the time window remains feasible and the courier retains authority to enter the dispatch area. A classical planner may express these as preconditions, effects, ordering constraints and resources. The adjacent article Planning Before Action develops that machinery. Here the concern begins after execution exposes the model to reality.

Calling the list a “plan” without its supporting conditions invites a dangerous shortcut. The executor can check that the next tool call is syntactically available and still miss that its business meaning has expired. A plan’s identity lies in its dependency structure, not in the prose that describes its steps. This is why merely asking a language model to rewrite the remaining checklist does not establish that the revised sequence is valid.

A policy trades commitment for coverage

In reinforcement learning, a policy is commonly defined as a mapping from perceived state to action.¹ That mapping may be a table, a control law, a neural network or a set of rules. Schoppers’ “universal plans” were an early planning formulation of a related ambition: specify an appropriate reaction for the situations that may arise, rather than commit to one predicted sequence.² The terminology differs across fields, but the operational distinction is clear. A policy moves branching work before execution. Runtime becomes cheap because many contingencies have already been represented.

The saving has a price. The policy only distinguishes what its input state distinguishes. A warehouse policy that sees current location and destination can handle an unexpected move to aisle C or a new goal H. If it does not receive the status of a remote edge, it may repeatedly choose a locally plausible action whose downstream value estimate is stale. More model capacity does not repair an omitted variable. Policy robustness is bounded by state representation, coverage and the validity of the dynamics under which it was produced.

Replanning changes the problem instance

Replanning runs planning again, perhaps reusing previous search, after the planning instance has changed. The new instance may have a different current state, goal, available action, resource level, duration, cost, constraint or world model. This is distinct from asking a policy for its next action. A policy selects inside an existing control problem. Replanning may reconstruct the problem itself.

Receding-horizon control offers a useful comparison. Model predictive control repeatedly solves a finite-horizon optimisation from the latest state, executes the first control and solves again.³ The structural lesson is valuable: plans can be provisional while action remains closed-loop. The analogy has limits. An enterprise agent may face discrete permissions, untrusted observations, irreversible effects and uncertain execution receipts. It cannot assume that a fresh optimisation is safe to execute merely because it is mathematically feasible.

A plan has a validity envelope

A common execution loop compares observed state with predicted state. That is necessary and insufficient. Two states can match on visible task variables while differing on an expired permission, a superseded rule or an action whose outcome is still unknown. Conversely, a harmless off-path change can produce a large state difference without affecting the plan. A better monitor asks which evidence supports the remaining suffix.

This is not a claim that every planner already computes such an envelope. It is a design instrument derived from plan monitoring, incremental search, control and governed action systems. The footprint can be exact for a symbolic plan, conservative for a learned planner, or partly elicited through tests. Even an incomplete footprint improves on monitoring every field equally because it states what could make the commitment false.

Thought experiment one: move only the closure

Return to the courier after it reaches A. In world one, edge C→D closes. In world two, edge B→G closes. Everything else is held constant: the sensor reliability, timing, number of changed fields and current location. The first closure sits outside the remaining route A→B→G. It may matter to backup options, but it does not invalidate the committed suffix. The second closure removes a required causal link. An event count, anomaly score or generic “environment changed” flag treats the worlds alike. A support-footprint monitor separates them immediately.

This distinction explains why monitoring and replanning cadence should not be conflated. Monitoring can remain frequent because observations are cheap or safety-critical. Planning can remain sparse because only some observations change a decision. Recent controlled work on data-centric tool use reports that full-horizon planning with lazy replanning matched step-wise planning in the authors’ studied tasks while using fewer tokens, though the result is bounded to well-defined knowledge and question-answering environments.⁴ A separate preprint on the Crafter environment found that always planning could be costly and less effective than learning when to allocate planning compute.⁵ These studies do not establish a universal trigger, but they strengthen the case against replanning by reflex.

Trigger on expected loss, not surprise alone

A binary footprint check is useful when conditions are explicit. Many agents face uncertain observations and approximate models. Then the trigger becomes an economic decision. Let the controller compare the risk of continuing with the cost and instability of planning again:

Here p_invalid is the current probability that the remaining suffix has lost a necessary support. L_failure is the expected loss if execution continues under that invalid suffix. R_keep is the regret of retaining a still-feasible but now inferior route. On the other side, C_plan is planning compute and latency, C_switch is the cost of abandoning or compensating current commitments, and C_churn is the instability introduced by changing course.

In the courier example, a reliable report that B→G is closed makes p_invalid close to one. If the parcel can wait safely, failure loss may be modest and a local repair is enough. If the parcel has a hard cold-chain deadline, the loss term rises and the controller should plan before taking another move. If C→D closes, p_invalid for the current suffix remains near zero. A generic anomaly score may rise, but the decision-relevant risk does not.

The equation is a decision scaffold, not a claim that every term can be estimated precisely. Its value is diagnostic. It exposes a controller that replans because planning is available, or one that continues because replanning is expensive, without accounting for consequence. The correct commitment horizon shrinks as uncertainty, consequence and irreversibility rise, and expands when the world is stable and actions are cheap to reverse.

Replanning can destabilise the system it is trying to save

A trigger needs memory of its own decisions. Imagine that two routes have almost equal expected cost and a noisy estimate alternates between them. A controller that replans whenever the current route loses by any amount can switch on every observation. Each switch discards work, changes resource reservations and creates new observations, which can reverse the ranking again. The planner is then part of the disturbance.

Control systems handle this with ideas such as hysteresis, minimum dwell time and switching penalties. An agent can use the same discipline. Require the replacement plan to improve expected outcome by more than a margin. Keep a recently selected route for a minimum interval unless a hard support fails. Increase C_churn when downstream systems have already reserved capacity or when humans have begun acting on the plan. These mechanisms do not prevent emergency replanning. They prevent indecision from masquerading as adaptivity.

The monitor should also distinguish invalidity from opportunity. A new route may be slightly cheaper while the current route remains valid. That is an optimisation trigger, not a safety trigger. The two deserve different thresholds, evidence standards and approval paths. A high-consequence system may immediately abandon an invalid route yet ignore small opportunities until the next natural checkpoint. Without this separation, every improvement signal competes with every failure signal, and the agent cannot explain why it changed course.

flowchart TD
  A["The aisle that closed"] --> B["Three control objects"]
  B --> C["A plan has a validity envelope"]
  C --> D["Repair when the disturbance is<br/>local"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#5B5CF6,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Repair when the disturbance is local

Once a suffix is invalid, “replan” still leaves an important choice. Should the controller preserve most of the old plan, or formulate the task again from the current world? The answer depends on the locality of the disturbance and on whether the old search structure remains informative.

Reuse can be powerful, and it is not free

Incremental search provides a clean positive case. D* Lite reuses information across a series of related path-finding problems and combines that reuse with heuristic search; its authors demonstrated advantages in the navigation applications they studied.⁶ Plan-repair methods similarly try to remove obstructing constraints and add what is needed rather than synthesize everything again. Van der Krogt and de Weerdt reported that their repair system outperformed replanning from scratch and a comparison repair method in their experiments.⁷ More recent PDDL+ work found that plan-fixing reformulations could outperform scratch replanning in the authors’ tested problems.⁸

The negative evidence matters just as much. Nebel and Koehler showed that plan reuse has no general worst-case efficiency advantage over generation, and that conservative modification can be strictly harder in some formulations.⁹ Repair is therefore an algorithmic option, not a natural law. A system can spend more effort protecting an obsolete plan than generating a clean one. The reused fragment may also anchor the search around a poor commitment, a phenomenon the runtime metric alone will not reveal.

A practical repair test asks four questions. Is the goal unchanged? Is the planning model still adequate? Can the affected constraints be localised? Are the retained actions still feasible, authorised and worthwhile? When all four answers are yes, repair can preserve useful search and execution work. When the goal or model changes, or the disruption touches many dependencies, reconstruction becomes cleaner.

Thought experiment two: change only what the policy can see

Compile a goal-conditioned courier policy from the original graph. Give it current location, destination and available outgoing edges. First, teleport the courier from A to C. The policy recognises C and chooses D, then G. Next, change the destination from G to H. Because goal is an input, it chooses a route to H. Now restore the original location and goal, but close the remote edge B→G. At A, all local observations match training. The policy chooses B. At B it discovers the closure and may return to A. If its value estimates still assume B→G exists, it can oscillate.

The only changed feature is whether the disturbance appears in the policy’s represented state soon enough to change its action. This is the core limit of “reactivity”. A policy can be highly responsive to represented variation and blind to unrepresented causal change. Under partial observability, the appropriate input may be a belief state rather than a directly observed state. The classical POMDP formulation treats that belief as the sufficient statistic for action choice under its assumptions.¹⁰ If the observation model is wrong, the belief can still be confidently wrong.

The negative control: when commitment should win

Consider a deterministic batch conversion over an immutable input snapshot. Every operation is idempotent, its output is locally verified, no external actor can change the data, and the goal cannot change during the run. A validated topological plan already contains the necessary order. Replanning after every successful operation adds compute, latency and another source of nondeterminism. A policy adds little because there are no meaningful branches. This is the negative control for the thesis: when support is stable, consequences are reversible and the task is fully represented, a committed plan is the stronger design.

This case also prevents a fashionable error. Closed-loop control does not imply maximum deliberation. The loop can observe and verify without reconstructing. A robust executor may simply advance after each postcondition succeeds, stop on failure and escalate a bounded exception. The design earns its simplicity from the environment, not from confidence in the model.

Repair can preserve the wrong thing

Repair systems often optimise some notion of stability: retain actions, preserve ordering or minimise edit distance from the old plan. Stability is useful when commitments have external cost. It is misleading when similarity becomes the objective. Two plans can differ by one action and impose very different risk, while two semantically equivalent plans can look distant under a syntactic metric. A planner that proudly retains ninety per cent of a suffix may be preserving the ninety per cent that depended on the obsolete assumption.

The antidote is a scratch-plan counterfactual. For a sample of disruptions, generate both the best repair available under the reuse constraint and a new plan without that constraint. Compare outcome quality, computation, retained irreversible commitments and explanation complexity. The scratch plan is not automatically preferable. It provides a baseline that reveals the price of loyalty to the old route. Nebel and Koehler’s negative result makes this baseline intellectually necessary rather than optional.

For learned agents, the equivalent test is to remove the previous plan from context and ask whether independent planning reaches a materially better course. Because two language-model samples share correlated errors, this is weak evidence unless the comparison uses different prompts, models, search procedures or formal validators. Plan continuity is operational evidence only when retained commitments are revalidated, not when the new text resembles the old text.

Situated execution sharpens this boundary. Cashmore and colleagues show that replanning for robots with durative or concurrent actions cannot be reduced to giving a planner a clean symbolic “current state”. Their formulation accounts for actions still in flight and introduces bailout actions for interruptible execution.¹¹ The broader lesson reaches beyond robotics: the act of stopping, compensating or waiting is itself part of the planning problem.

Production control needs more than a planner

Language-agent frameworks often interleave reasoning, action and observation. ReAct made that loop explicit for language models and demonstrated its usefulness across the authors’ tasks.¹² Interleaving is an execution pattern, not a complete control theory. An observation can be untrusted, an action can require separate authority, and a fluent explanation can obscure that the system has not verified the world.

The initial plan contains five stages: reconstruct the case state, validate identity evidence, run screening, assess eligibility, and prepare the next communication. Three surprises arrive. First, a document unrelated to eligibility changes its display label. That sits outside the current support footprint. Second, the identity document expires before screening. The remaining suffix depends on valid identity evidence, so the controller should repair by requesting a current document. Third, the applicable policy version changes with an effective-time rule. That change can alter the criteria, evidence requirements and explanation. The safer response is to reconstruct from the current case state and applicable policy rather than preserve a cosmetically similar suffix.

A fourth event is more subtle. The communication service times out after accepting a request, and no effect receipt returns. The planner does not know whether the message was sent. A naive replan may choose “send communication” again. The correct next action is readback or reconciliation, not semantic improvisation. Execution uncertainty must be resolved before planning treats the world as changed.

The ten objects that must remain distinct

In a consequential agent, the planning loop sits inside a larger control system. The intent states the desired business change. Identity binds the requester, subject and service actors. World state records current reality. Context supplies permitted evidence. Reasoning proposes a route. Authority decides which effects are allowed. Action invokes a typed operation. Evidence records observations and receipts. Outcome verifies the business state. Release and recovery govern confirmation, timeout, compensation and escalation.

Plans may reference all ten, but they do not collapse them. A graph edge does not grant permission. A fresh policy document does not prove that it applies to this subject at this effective time. A tool success response does not prove the intended postcondition. A new plan does not erase an earlier unknown outcome. Replanning changes proposed coordination; it must not manufacture authority or evidence.

Observation quality changes the trigger

A support-footprint trigger is only as reliable as its evidence. Each monitored condition needs provenance, effective time, confidence and a rule for supersession. A closure reported by one noisy sensor may justify a slower verification step. A signed operational event may justify immediate repair. A natural-language tool message copied from untrusted content should not directly rewrite the plan’s control state.

This is where context compilation matters. Raw documents and tool outputs should become typed claims such as “edge B→G unavailable from 14:05 to 14:40” or “policy version 7 applies to applications received after effective time T”. The planner can then reason over explicit scope and time. Without that transformation, “new context” becomes an undifferentiated prompt, and the model may treat an irrelevant sentence as a task-changing instruction.

Choose the control mode explicitly

The design choice can be made before implementation. Start with the environment and consequence, then choose the cheapest control object that preserves the required evidence. The following questions form a compact architecture decision record.

1. What is committed? Record the remaining actions, causal links, resources, deadlines, goal and irreversible effects.
2. Which observations can invalidate it? Name the support footprint and the authoritative source for each monitored condition.
3. Which variation is already represented? State the policy inputs, belief update, validation region and known out-of-distribution indicators.
4. How local is the disruption? Estimate the fraction of retained actions whose preconditions, value and authority remain valid.
5. What is the cost of delay and churn? Compare planning latency, switching cost and instability with expected failure loss and regret.
6. Is an effect in flight or outcome unknown? Read back, reconcile or compensate before a new plan assumes the action did or did not occur.
7. Who may approve the changed route? Keep proposal, veto, approval and execution rights explicit, especially when the new route raises consequence.

Test the trigger as a measurement system

A replanning trigger deserves its own evaluation set. Create disturbances that vary causal relevance rather than visual magnitude: an off-path edge and an on-path edge, an expired fact that is superseded by a current one, a goal change hidden inside similar wording, a policy-covered state deviation and an unrepresented structural change. For each case, label whether continuation, policy, repair, reconstruction, observation or escalation is the least-cost defensible response.

Measure false replans and missed replans separately. A false replan wastes compute and can create churn. A missed replan carries the loss of invalid execution. Also measure detection delay, planning delay, fraction of retained commitments revalidated, duplicate-effect rate after timeouts, and the rate at which a human overturns the chosen control mode. Final task success alone can hide a trigger that replans constantly until it gets lucky.

Failure injection should target the evidence chain as well as the route. Delay a closure event. Supply two sources with different effective times. Corrupt the state projection while leaving the raw event correct. Return an accepted command without an effect receipt. Remove authority after planning but before execution. These tests establish whether the system knows why its plan remains valid. The trigger is a metrology component: it converts observations into a claim about commitment, and its errors must be calibrated like any other decision instrument.

Executable laboratory: four controllers, five surprises

The accompanying Python laboratory turns the opening case into a falsifiable comparison. It uses a synthetic directed graph and four controllers: a committed shortest path, an offline goal-conditioned local policy, an event-triggered replanner and a planner that solves again before every action. Disturbances close an irrelevant edge, close a required edge, move the agent, or change the goal. Online planning counts exclude offline policy compilation.

The event trigger monitors three conditions: whether a changed edge intersects the remaining plan, whether execution has left the expected state, and whether the goal has changed. It uses fresh Dijkstra search after a breach. This is intentionally simpler than D* Lite, so the experiment isolates trigger semantics rather than claiming an incremental-search implementation.

def should_replan(plan, cursor, state, goal, changed_edges):
    remaining = {
        (plan[i], plan[i + 1])
        for i in range(cursor, len(plan) - 1)
    }
    support_breached = bool(changed_edges & remaining)
    state_invalid = cursor >= len(plan) or plan[cursor] != state
    goal_invalid = not plan or plan[-1] != goal
    return support_breached or state_invalid or goal_invalid

# Positive case: stable world
# Expected: event-triggered succeeds with one search;
# eager replanning succeeds with three searches.

# Negative case: an off-path edge closes
# Expected: event-triggered does not replan merely because
# some environmental fact changed.

All four strategies reach the goal in the stable world. The eager controller performs three searches, while the event trigger performs one. Closing an off-path edge changes nothing and again causes no event-triggered search. Closing B→G makes the committed plan fail. The local policy also fails within the step limit because its baseline values omit the non-local closure and send it around an A↔B loop. Both replanners recover through A→C→D→G, but the event-triggered controller uses two searches against four for eager replanning.

The policy wins a different contest. After a forced state deviation to C, or a goal change to H, it adapts with no online search because location and goal are represented inputs. The committed sequence fails both cases. The experiment does not crown one controller. It exposes the causal condition under which each controller earns its cost.

stateDiagram-v2
  [*] --> Claim
  state "Plan" as Claim
  state "Policy" as Test
  state "Replanning" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Compact glossary

Committed suffix

The unexecuted portion of a plan whose earlier actions have already fixed part of the state.

Support footprint

The decision-relevant facts, links, resources, goal and authority assumptions required by that suffix.

Validity envelope

The set of world and evidence states in which the remaining commitment is still justified.

Policy coverage

The region of represented states or beliefs for which a policy has adequate evidence and validation.

Plan repair

Modification that preserves useful parts of a previous plan while removing or replacing invalid constraints.

Reconstruction

Formulating the planning problem again because the goal, model, authority or broad state has changed.

Unknown outcome

A command was accepted or timed out, but the intended effect has not yet been verified.

Decision receipt

A record of the proposal, evidence, policy and authority decision that permitted or rejected an action.