The gap is a deformation, not a distance

People often speak about closing the reality gap as if simulation and reality were two points separated by one measurable distance. That picture hides the mechanism. A policy interacts with a sequence: it receives an observation, updates internal state, selects an action, meets physical dynamics, and receives another observation. Mismatch can enter at every link. It can also alter the task itself, such as a new obstacle class, a changed payload distribution or a human behaviour never represented during training.

Let a simulator be a family of models indexed by a vector z. The vector can include mass, friction, camera exposure, packet delay, actuator gain, contact parameters, object geometry and task conditions. A policy π has an outcome function J(π, z), such as successful grasps per attempt, time to goal, energy use or braking clearance. It also has constraints, such as maximum collision probability or force.

A local change can be sketched as ΔJ ≈ ∇J · Δz. The model mismatch is Δz; the policy sensitivity is the gradient ∇J. Either term can be small while the other is large. A simulator can be inaccurate in harmless directions, or accurate on average while missing a narrow direction that drives failure. The linear expression is only a local intuition. Contacts, mode switches and learned policies often create cliffs where a small change produces a discontinuous outcome.

Here, A_π is the policy's tested acceptance region. J_min is the minimum useful performance. ε is the tolerated constraint-violation probability. p_real(z | D_t) is the current belief about the physical system after observing data D_t. The release confidence is 1 - α. This does not prove the model complete. It makes the transfer claim explicit enough to attack.

Thought experiment: equal error, unequal danger

Consider two braking simulators with the same mean absolute error in predicted stopping distance. Simulator A is wrong by 20 centimetres at every speed. Simulator B is nearly exact in normal driving but wrong by two metres when tyre friction falls below 0.3. If the robot normally stops with three metres of clearance, A is visibly inaccurate yet operationally tolerable. If the robot sometimes enters a wet loading bay with only one metre of clearance, B is more accurate on average and more dangerous.

The causal feature changed is the placement of error relative to the decision boundary. Aggregate fidelity scores erase that placement. The same problem appears when a perception model has excellent average accuracy but fails on objects located precisely where an action becomes irreversible. Mismatch concentrated at a policy cliff matters more than larger mismatch in a flat region.

Build fidelity around the decision

The warehouse example contains at least five different gaps. Treating them as one makes engineering discussions vague and tests incomplete.

Five mismatch families

Observation mismatch changes what the policy can infer. Examples include lens distortion, lighting, rolling shutter, missing depth returns, microphone reverberation, timestamp skew and data preprocessing. Visual domain randomisation deliberately varies textures, lighting, camera pose and other rendering factors so that the real image is less exceptional to the model. Tobin and colleagues showed the value of this idea for object localisation and grasping with non-realistic random textures.³ Later manipulation work found that carefully selected visual randomisation parameters could support strong transfer across its tested tasks, but that result belongs to those tasks and conditions.¹²

State mismatch occurs when the simulator exposes variables the real system cannot know. A training policy may receive perfect object pose, exact velocity or privileged contact state. The deployed policy receives estimates, delays and missing values. Replacing privileged state with a learned estimator can introduce a new failure mode even if the physical simulator is excellent.

Dynamics mismatch covers mass, inertia, friction, compliance, aerodynamics, contacts, fluid effects and disturbances. Peng and colleagues randomised simulator dynamics to obtain policies that transferred to a physical pushing task.⁴ The mechanism is coverage: a policy trained across varied dynamics is discouraged from relying on one narrow model.

Actuation mismatch includes saturation, backlash, dead zones, thermal limits, control-loop frequency, packet loss and execution delay. It is often buried inside a generic dynamics label, but it deserves separate tests because a correct desired force is not the same as the force the hardware produces.

Task and distribution mismatch changes what situations occur and what success means. Payloads may become heavier, objects may be deformable, workers may enter from a new doorway, maintenance may alter a calibration, or business rules may change the permitted response. No amount of pixel fidelity repairs a simulator that generates the wrong world distribution.

Four bridges and their limits

Most sim-to-real methods can be understood as four moves. They are complementary, and each fails differently.

Randomise what is uncertain. Domain randomisation samples many simulated worlds. It can break brittle dependence on one texture or one friction value. Its burden is choosing a distribution. Vuong and colleagues showed that the choice of randomisation parameters materially affects real performance and should itself be optimised rather than treated as an arbitrary engineering detail.⁷ Randomise too narrowly and the real system may fall outside coverage. Randomise too widely and the policy may become unnecessarily conservative or learn around physically impossible combinations.

Calibrate what is identifiable. System identification uses real trajectories to estimate plausible simulator parameters. SimOpt adapts a distribution of simulator parameters using real rollouts rather than relying only on hand tuning.⁶ BayesSim treats those parameters probabilistically, producing a posterior rather than one best fit.⁸ Calibration is valuable when observed behaviour distinguishes the parameters. If friction and motor gain produce nearly the same trajectory, fitting one number may hide ambiguity rather than remove it.

Adapt the policy online. A universal policy can condition on an estimated dynamics context. Yu and colleagues paired a policy trained across models with online system identification based on recent state and action history.⁵ Rapid Motor Adaptation later used an adaptation module to infer changing extrinsics for quadruped locomotion.¹⁰ Adaptation buys response to change, but it adds a new estimator whose delay, observability and failure modes must be tested.

Learn where simulation is trustworthy. The transferability approach does not require the simulator to be accurate everywhere. It learns which solution regions transfer using a small number of physical tests, then avoids high-scoring simulated behaviours predicted to exploit model error.² This is close to the paper's central claim: the useful object is not global simulator fidelity, but a map from policy behaviour to transfer risk.

flowchart TD
  A["Two twins enter the same aisle"] --> B["The gap is a deformation, not<br/>a distance"]
  B --> C["Build fidelity around the decision"]
  C --> D["A serious negative control,<br/>perhaps no twin is needed"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#B96D00,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

A serious negative control: perhaps no twin is needed

There is an important objection to the whole discussion. Classical control engineering already offers robust control, gain and phase margins, observers, fault detection, system identification, model predictive control and hardware testing. Why invent a digital twin vocabulary around familiar methods?

The objection is partly correct. Suppose a motor's uncertain inertia and friction are bounded, the control law has a proven stability margin over those bounds, the task has no learned perception, and a physical test can cheaply cover the operational extremes. A continuously synchronised asset model may add cost without changing the release decision. The appropriate baseline may be a robust controller plus a conservative safety envelope.

Evidence from learned locomotion also warns against treating dynamics randomisation as mandatory. Xie and colleagues reported direct transfer for their Laikago quadruped case without dynamics randomisation or on-robot adaptation, and used ablations to identify other design choices that affected robustness.⁹ This does not overturn randomisation results. It shows that a successful method in one stack can be unnecessary in another. Simulator quality, action representation, low-level control, policy architecture and task all move the boundary.

A digital twin earns its place when asset-specific change, partial observation or counterfactual decision support alters what can safely be done. Examples include wear that accumulates differently across individual machines, damage that changes hidden state, coupled models that cannot be exhaustively tested physically, or operations where a model must ingest live evidence before comparing interventions. If those burdens are absent, the simpler control and test system is the negative control the twin must beat.

A digital twin is a belief loop, not a mirror

The label digital twin is often applied to a static simulation, a three-dimensional dashboard or a stream of sensor readings. Those artefacts can be useful, but they do not carry the full burden. The 2024 National Academies report describes a digital twin as virtual information constructs that mimic a physical counterpart, update dynamically from it, have predictive capability and inform decisions that realise value.¹⁶ ISO 23247 supplies a manufacturing framework and reference architecture rather than a promise of correctness.¹⁴ Earlier aerospace formulations likewise coupled high-fidelity simulation with vehicle health, maintenance and historical data over an asset's life.¹³

The common feature is not graphic resemblance. It is a maintained relation between a physical system, observations, models and decisions. A defensible twin therefore contains at least four kinds of uncertainty: uncertain current state, uncertain parameters, known model discrepancy and unrepresented possibilities. The last category cannot be assigned a trustworthy probability merely because a software interface demands one.

Calibration is not validation

Calibration adjusts uncertain parameters so that model outputs better match observations. Validation asks whether the model is adequate for an intended use. A model can be calibrated on one trajectory and invalid for a different manoeuvre. Kennedy and O'Hagan's Bayesian calibration framework made a further distinction: even the best-fitting parameters can leave systematic discrepancy between a computer model and reality.¹⁷ If that discrepancy is forced into physical parameters, the twin may fit yesterday's data for the wrong reason and extrapolate badly tomorrow.

Continuous twins add a temporal burden. Ward and colleagues compared particle filtering and Bayesian calibration for continuously streamed data, illustrating how parameter distributions can be updated rather than manually refitted once.¹⁸ But synchronisation frequency must follow the decision. A structural-health model may tolerate hourly updates. A collision-avoidance controller cannot rely on a twin that is one second behind.

NIST's work on digital-twin credibility treats verification, validation and uncertainty quantification as necessary to establish fitness for intended purpose, and stresses that credibility extends beyond a one-time model check.¹⁵ ASME similarly distinguishes verification, which asks whether the computational model solves its mathematical description correctly, from validation, which asks whether it represents the real application accurately enough, and from uncertainty quantification, which measures the effects of uncertain inputs.¹⁹

Thought experiment: the perfectly synchronised wrong model

A twin receives flawless, millisecond telemetry from a robot arm. Its estimated joint angles and temperatures are exact. The model omits cable flex, which becomes important only when the arm carries a new long tool. A second twin receives noisy telemetry every 100 milliseconds but includes cable flex and maintains uncertainty over tool load. Which is safer for predicting end-effector overshoot?

The causal feature changed is model class, not data freshness. Better synchronisation reduces state uncertainty inside a model. It does not eliminate model-form error. A live feed can keep the wrong equations perfectly up to date. The twin must therefore report both state uncertainty and evidence that its model family remains adequate under the current intervention.

Three clocks must agree

The physical system changes on one clock, the twin updates on a second, and the policy is released on a third. A bearing wears gradually, the twin's parameter estimate tracks it, but the policy was validated against last month's acceptance region. The twin may accurately announce that the system moved outside the policy's safe envelope. That is useful only if the operating system can withdraw authority, reduce speed, switch controller or escalate.

This separates twin validity from policy validity. A current twin can invalidate an old policy. A stale twin can also leave a valid policy under-informed. The release mechanism must bind a policy version to the model family, parameter envelope, data window, thresholds and fallback under which it was accepted.

A minimal transfer laboratory

Consider a one-dimensional robot travelling towards an obstacle. It observes remaining distance and speed, estimates stopping distance, and commands maximum braking when that estimate reaches the obstacle. The physical system differs from the training simulator in only two variables: tyre-floor friction and command-to-brake latency.

The nominal policy assumes friction of 0.60 and latency of 0.05 seconds. The coverage policy assumes lower friction, greater latency and an additional margin. It is not smarter. It simply acts as if a wider part of the deformation field were plausible. This makes the mechanism visible: robustness buys coverage by spending stopping efficiency.

In the well-modelled case, both policies stop. The nominal policy uses more of the available distance, while the coverage policy stops early. Under the shifted case, lower friction and longer latency move the physical system across the nominal policy's cliff; the coverage policy remains safe. Outside both policies' represented envelope, both fail. That third case is essential. A robust policy is not permission to stop measuring reality.

The artefact also plots a grid. The coloured cells are not a claim about a robot. They are a compact version of the experiment every transfer programme needs: vary each plausible mismatch, include coupled extremes, locate constraint boundaries and compare a preferred policy with a simpler baseline. Replace the toy dynamics with a validated simulator and physical tests, then preserve the same logic.

const policies = {
  nominal: { assumedFriction: 0.60, assumedLatency: 0.05, margin: 0.05 },
  coverage: { assumedFriction: 0.35, assumedLatency: 0.18, margin: 0.20 }
};

function simulate(policy, physicalFriction, physicalLatency) {
  const dt = 0.001;
  const obstacle = 5.5;
  let x = 0, velocity = 3.2, time = 0;
  let brakeStartsAt = null;

  while (time < 8) {
    const remaining = obstacle - x;
    const estimatedStop =
      velocity * velocity / (2 * policy.assumedFriction * 9.81) +
      velocity * policy.assumedLatency + policy.margin;

    if (brakeStartsAt === null && estimatedStop >= remaining) {
      brakeStartsAt = time + physicalLatency;
    }

    const acceleration =
      brakeStartsAt !== null && time >= brakeStartsAt
        ? -physicalFriction * 9.81
        : 0;

    x += velocity * dt + 0.5 * acceleration * dt * dt;
    velocity = Math.max(0, velocity + acceleration * dt);
    time += dt;

    if (x >= obstacle && velocity > 0.05) return { safe: false };
    if (velocity <= 0.01) return { safe: true, clearance: obstacle - x };
  }
  return { safe: false };
}

stateDiagram-v2
  [*] --> Claim
  state "Thought experiment: equal<br/>error, unequal danger" as Claim
  state "Five mismatch families" as Test
  state "Four bridges and their limits" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

A production-shaped transfer: the shared crossing

Return to the warehouse robot, now as a synthetic operating case rather than a thought experiment. An autonomous mobile robot carries variable payloads between storage and packing. It approaches a shared crossing with a 1.4 metre stop boundary. Cameras detect people and forklifts; wheel encoders and an inertial unit estimate motion; a low-level controller converts velocity commands into motor torque. The proposed learned policy chooses approach speed and braking onset.

The engineering team has a digital twin built from the facility map, robot kinematics, measured motor response, wheel-floor friction tests, camera latency, historical payloads and traffic traces. The twin receives current battery temperature, payload estimate, wheel slip and software timing. It can simulate candidate approaches before the live system acts. None of that makes the policy authorised.

The transfer claim must be traced across ten distinct objects. Intent is to cross without violating the stop boundary. Identity binds the command to the specific robot and policy version. World state records current position, velocity, payload and traffic. Context contains the observations and applicable safety envelope. Reasoning proposes an approach. Authority determines whether this policy may control speed in this zone. Action is a typed velocity or stop command. Evidence links the proposal to sensor and twin state. Outcome is the observed physical stop. Release and recovery decide whether the policy remains active after drift or failure.

Step 1: define the operating envelope before training

The team records friction by floor zone and contamination state, payload mass and centre-of-mass bounds, camera and control latency distributions, minimum observable object size, battery and motor-temperature limits, maximum approach speed, crossing geometries and human traffic patterns. Correlations are preserved. Wet floors occur more often near loading doors; heavy pallets alter both inertia and wheel slip. Independent random sampling would generate some impossible cases and miss some plausible clusters.

The envelope is not merely a training range. It is a contract for evaluation and release. Each variable has provenance, uncertainty, last-observed time and a rule for what happens when it leaves range. An unmeasured floor coating is not silently clipped to the nearest known friction value.

Step 2: map the policy's acceptance region

Training randomises observation noise, latency, payload, friction and actuator response. Evaluation then uses a separate design. It sweeps single variables to expose sensitivities, combines correlated extremes, searches for failures near the stop boundary and compares against a deterministic braking controller. The team records clearance distributions, collision rate, unnecessary stops, energy and travel time. A policy that never collides because it refuses to move fails the usefulness threshold even if it satisfies the safety threshold.

Search should focus on the boundary rather than uniformly spend simulation everywhere. Adversarial optimisation, active learning or simple adaptive grids can locate the narrow region where a few milliseconds or a small friction change flips the outcome. These points become physical-test candidates because they discriminate between competing transfer claims.

Step 3: introduce real timing and hardware

Hardware-in-the-loop replaces simulated control timing, motor interfaces and sensor pipelines while retaining a safe virtual environment. The test injects packet loss, clock skew, delayed frames, stale state, actuator saturation and restart. This stage often reveals a different reality gap: the policy's mathematics may be adequate while the software execution path violates its assumed control frequency.

Selected low-speed physical tests then update the twin's posterior and discrepancy model. They are chosen for information, not spectacle. A manoeuvre that excites both acceleration and braking may distinguish motor gain from rolling resistance better than another nominal crossing. If parameters remain confounded, the release envelope must cover the ambiguity rather than select the most convenient fit.

Step 4: shadow before acting

In shadow mode the policy receives live observations and proposes actions, but the existing controller retains authority. The system compares proposed and executed trajectories, twin predictions and physical outcomes. It looks for systematic residuals, missing objects, timing differences and regions where the policy would have crossed a constraint. This captures the operational distribution without granting the new policy physical effect.

Shadow success is necessary but weak when the existing controller changes the trajectory. The new policy's counterfactual action was not executed, so its downstream state is partly unobserved. The twin can estimate that counterfactual, but the estimate inherits the very model uncertainty under examination. A bounded pilot must eventually create carefully limited real interventions.

Step 5: grant narrow authority and verify the effect

The pilot limits speed, zones, payloads, hours and traffic conditions. A deterministic authority kernel checks robot identity, policy version, current envelope membership, command schema, speed ceiling and emergency-stop availability. The model proposes; mechanical controls enforce the hard boundary. Every accepted action receives an effect observation: did the robot slow, stop and remain behind the line?

This last question matters because an accepted command can have an unknown outcome. A network timeout after a stop command does not reveal whether the motor controller received it. Retrying may be safe for an idempotent stop, but not for every physical effect. The system needs readback from authoritative sensors, a timeout policy and a conservative fallback. Simulation transfer without outcome verification proves the proposal, not the action.

The transfer release gate

A release review should be able to reject a policy even when its simulator score is excellent. The following instrument is deliberately framed as evidence questions rather than method names. Domain randomisation, a twin platform or a famous benchmark is not an answer by itself.

1. Name the physical decision and postcondition. State what the policy may propose, what it may execute, the unit of success, the safety constraints and how the physical outcome will be observed.
2. Define the operating envelope as a joint distribution. Include parameter correlations, discrete modes, timing, observation quality, external actors and the conditions that are explicitly out of scope.
3. Separate verification, calibration, validation and discrepancy. Show that code implements the equations, parameters fit evidence, the model is adequate for the intended intervention, and residual error is not disguised as a parameter.
4. Map the policy acceptance region. Use sweeps, boundary search, fault injection and coupled extremes. Report both usefulness and constraint surfaces, with uncertainty and repeated seeds where learning is stochastic.
5. Compare a serious baseline. Test a deterministic controller, robust control design, simpler policy or physical rule. The twin-backed learned policy must earn its added complexity.
6. Use discriminating real tests. Choose safe experiments that distinguish model explanations and probe the simulated cliff. Do not spend scarce physical trials repeating comfortable nominal cases.
7. Bind authority to evidence. Release only named policy versions, assets, zones, speeds and envelopes. Define automatic reduction, fallback and human escalation when evidence expires or the twin reports departure.
8. Verify effects and keep a transfer ledger. Record proposal, authorisation, action, readback, outcome, model version, posterior, discrepancy and unresolved risk so a later reviewer can reconstruct why the action was allowed.

What would weaken this paper's thesis?

The argument would be weakened if a global simulator-fidelity metric consistently predicted transfer across materially different policies and tasks better than policy-specific acceptance regions. It would also be weakened if continual twin calibration reliably removed the need for discrepancy modelling and boundary tests under extrapolation. Current evidence instead shows task dependence, method dependence and continuing concern with VVUQ. The National Academies explicitly calls for continual verification, validation and uncertainty quantification, including under extrapolatory conditions.¹⁶

An empirical research programme follows. For a set of physical tasks, construct several simulators that trade visual, dynamics, timing and distribution fidelity. Train policies with different sensitivities. Compare three predictors of transfer: global simulator similarity, trajectory similarity under nominal actions, and posterior coverage of each policy's measured acceptance region. The paper's open hypothesis is that the third will better predict constraint failures, especially near mode changes.

{
  "asset_id": "amr-synthetic-07",
  "policy_version": "crossing-policy-3.2",
  "intended_use": "approach-speed and brake-onset in zones A3 and A4",
  "twin_model_family": "wheel-slip-v4 + latency-v2 + perception-residual-v1",
  "evidence_window": "last 10 operating hours plus quarterly friction survey",
  "real_posterior": {
    "friction": {"credible_interval": [0.34, 0.61]},
    "control_latency_ms": {"credible_interval": [62, 141]}
  },
  "policy_acceptance": {
    "minimum_clearance_m": 0.25,
    "maximum_violation_probability": 0.0001,
    "posterior_coverage_required": 0.999
  },
  "authority": {
    "maximum_speed_mps": 1.2,
    "permitted_payload_kg": [0, 180],
    "fallback": "deterministic-stop-controller"
  },
  "outcome_readback": "safety_lidar_stop_pose + wheel_encoder_zero_velocity",
  "withdraw_on": ["posterior_outside_acceptance", "unknown_outcome", "new_residual_mode"]
}

Stop asking whether the twin is realistic

The wrong release question is, “How realistic is the simulator?” It invites a tour of graphics, model detail and platform features. The better question is, “Which physical changes can reverse this policy's decision, and what evidence shows those changes remain covered?”

That question changes the architecture. The simulator becomes a model family with explicit modes and uncertainty. Evaluation becomes a search for policy cliffs, not a repetition of average cases. Real experiments are selected to discriminate model explanations. The digital twin becomes a maintained posterior and discrepancy ledger, not a perfect virtual copy. Authority becomes conditional on asset identity, policy version, operating envelope and current evidence. Outcome readback closes the gap between a plausible command and a verified physical effect.

Randomisation can broaden coverage. Calibration can concentrate it around observed reality. Online adaptation can respond within a represented family. Transferability models can steer optimisation away from known simulator exploits. None removes the need to test the acceptance boundary, compare a simpler baseline and admit missing modes.

The decisive practical rule is therefore modest: do not release simulated capability; release a narrow, evidenced slice of it. Begin in simulation, move through real timing and hardware, observe in shadow, intervene under bounded authority, and withdraw when the real-system posterior or discrepancy leaves the tested region. A twin is valuable when it helps the system know that boundary, update it and act on departure.

The reality gap will not disappear, because every model omits. What can improve is the discipline of the claim. A policy need not inherit the whole world's complexity. It must survive the particular deformations that matter before its next action becomes irreversible.