The interval that learns

“Offline” should not mean unconscious, asleep or disconnected from a network. It names a computational relation. A learner is offline with respect to a task when its internal update is partly decoupled from the next external observation and action. The organism may be asleep, sitting quietly or mind-wandering. A machine may be in a shadow-training window while its live policy remains frozen. In each case, stored or generated traces can become the effective training data.

This distinction separates inactivity from offline cognition. A powered-down device is inactive. A sleeping brain is not. A deployed model whose weights are frozen while logs are merely archived is not learning offline. A second model that replays selected traces against a candidate update is. The relevant variable is not movement. It is whether an internally constructed sample stream is driving plastic change.

Human memory research gives this possibility substance. Sleep can support both quantitative retention and qualitative reorganisation, although effects vary by memory system, task, timing and comparison condition. Quiet wake can also help. A recent meta-analysis of 37 wakeful-rest studies reported an overall benefit for later memory, which means that sleep cannot own the entire category of offline processing (Weng et al., 2025). The safer claim is that reduced external interference can create conditions in which internally driven reactivation matters.

Fast capture creates a stability problem

Suppose a learner changes its general knowledge immediately after every episode. One unusual encounter can overwrite a useful regularity. Suppose instead that it changes only slowly. It preserves stability but fails to absorb a genuinely new fact before that fact disappears. This is the stability-plasticity problem in miniature.

Complementary learning systems theory proposes a useful division of labour: a fast system captures particular episodes, while a slower system extracts structure through interleaved learning. The original connectionist argument was that gradual, interleaved exposure lets new knowledge enter distributed representations without catastrophic interference (McClelland, McNaughton & O’Reilly, 1995). Later revisions emphasised that rapid generalisation can also occur when new material fits existing structure (Kumaran, Hassabis & McClelland, 2016).

Four jobs hidden inside one word

“Consolidation” often sounds like a single operation that makes a memory harder to erase. That description is too coarse. Hold the original experience constant and change the test given tomorrow. A word pair may need exact retention. A map may need a new shortcut. A motor sequence may need smoother execution. A painful prediction may need reduced control over future behaviour. Each outcome can improve after an offline interval, but the required transformation is different.

The first job is stabilisation: protect a fragile trace from interference. The second is integration: relate the trace to older structure so it can be retrieved through more than one route. The third is abstraction: preserve a regularity while discarding episode-specific detail. The fourth is prospection: use fragments of the past to evaluate a situation not yet encountered. Calling all four “strengthening” hides the direction of change.

Consider a trainee who sees three variants of the same equipment fault. Exact replay can preserve each inspection. Interleaved replay can reveal the common causal pattern. Compression can form a reusable fault category. Recombination can simulate a fourth variant. A later test of recognition may reward the first operation, while a transfer test rewards the latter three. The same learner can therefore look better or worse depending on which offline job the test exposes.

This matters because a memory can become less detailed yet more useful. It can also become more fluent yet less accurate. An offline benefit must name the changed capability, rather than only report that performance rose. Retention, transfer, inference, calibration and revision should be measured separately. Otherwise a system that memorises yesterday and a system that reorganises yesterday receive the same label even though they will fail in different ways.

This yields the paper’s causal thesis. Offline cognition improves adaptive intelligence when it changes the learner’s effective sample distribution and update conditions. A quiet interval with no replay does nothing. Replay without selection may waste capacity. Selection without interleaving can create obsession. Interleaving without a commit test can spread distortion. The useful unit is therefore not “sleep” or “replay” alone, but a governed offline update cycle.

What the night chooses

Imagine that a day produces ten thousand candidate traces. The night cannot faithfully rehearse each one at equal resolution. Even if it could, equal replay would preserve the day’s accidental frequency distribution, including noise, routine repetition and misleading salience. The central engineering problem appears in biology too: which traces receive scarce offline updates?

Six words that must not collapse

Rodent studies first made replay concrete by showing that hippocampal place-cell activity associated with a waking trajectory reappears during subsequent sleep (Wilson & McNaughton, 1994). Replay can run forwards, backwards and at compressed timescales. It also occurs during quiet wake, where reverse sequences can follow a run and may support evaluation or planning (Foster & Wilson, 2006). Human studies use less direct windows, including multivariate fMRI, MEG and intracranial recordings, so the same word covers measurements with different resolution. That translation should remain explicit.

During human non-REM sleep, a leading account links cortical slow oscillations, thalamocortical spindles and hippocampal ripples (Diekelmann & Born, 2010). Their timing may open windows in which reactivated traces influence wider cortical networks. In an invasive closed-loop study, stimulation synchronised to medial temporal slow-wave phases increased cross-region coupling and improved recognition accuracy, while the same stimulation without precise timing did not produce the same benefit (Geva-Sagiv et al., 2023). This is stronger than correlation, but it remains evidence from a particular human sample, task and intervention.

Timing is a routing variable

A cue does not have to contain the lesson to alter what is learned. Suppose two object-location sets are trained with different sounds. During a later quiet interval, one sound is presented again without the objects. If the cued set is subsequently changed more than the uncued set, the cue acted as an address into the replay queue. Targeted memory reactivation uses this logic to test whether biasing reactivation changes later memory. Reviews find reliable effects in some conditions, alongside substantial dependence on task, sleep state, cue intensity and analysis choice (Carbone & Diekelmann, 2024).

Now change only the phase at which the cue arrives. The sound is identical, the earlier learning is identical and total stimulation is identical. Yet the effect may differ because slow oscillations, spindles and ripples create transient opportunities for communication among memory systems. The closed-loop human study described above is informative for exactly this reason: synchronised stimulation changed coupling and memory, while poorly timed stimulation did not. Offline processing is governed not only by content, but by when a trace becomes eligible to influence another system.

This temporal sensitivity blocks a common inference. If an overnight intervention fails, it may have selected the wrong trace, arrived in the wrong state, disrupted sleep or targeted a memory that no longer needed help. A null result does not collapse these possibilities into “sleep has no effect”. Conversely, a positive result does not prove a universal sleep-learning channel. The intervention must identify the cue, the state, the proposed bridge and the later capability it is expected to change.

Replay has a queue

Evidence increasingly suggests that replay is prioritised rather than exhaustive. In one human fMRI study, weaker items were replayed more during subsequent rest, and greater replay predicted better later memory (Schapiro et al., 2018). In rats, replay frequency reflected exposure and familiarity: more traversals increased replay, while familiar tracks received less (Huelin Gorriz, Takigawa & Bendor, 2023). Reward and behavioural relevance can also bias reactivation.

A recent mouse study adds a finer result. Within non-REM sleep, pupil-linked substates segregated replay of recent and older memories. Disrupting sharp-wave ripples in the substate dominated by recent replay impaired recall of recent memories (Chang, Tang et al., 2025). The result suggests temporal multiplexing: old and new material need not compete in one undifferentiated replay pool.

Yet the hippocampus is not a universal night manager. A 2026 study found replay of a learned procedural sequence in mouse dorsal striatum, including prioritisation by behavioural outcome, even after bilateral hippocampal lesions (Thompson et al., 2026). Replay may be a family of offline mechanisms distributed across parallel memory systems, with the hippocampus central for some forms of memory but not obligatory for every skill.

A useful queue has four fields

A replay priority should not be a single scalar copied from “importance”. At minimum, it needs four distinguishable fields. Need estimates fragility or current error. Future value estimates whether the trace is likely to matter under anticipated conditions. Coverage prevents common classes and recent episodes from consuming the budget. Trust asks whether the observation, label and outcome are valid enough to teach from at all.

These fields behave differently. Need and future value can increase priority. Coverage acts at the batch level: an item may enter because a region of experience would otherwise disappear. Trust should usually be a gate, not a weak negative term. A corrupted trace with extreme loss can outrank every clean example if distrust merely subtracts a small score. The result is an apparently diligent learner that rehearses the least reliable part of its history.

The queue must also change over time. A weak memory may cease to need replay after successful retrieval. A routine event may become valuable after a new instruction arrives, as in the post-learning memo. An old example may become essential when recent experience narrows the learner’s coverage. Priority is a decision about the next update, not an intrinsic property stored permanently on the event.

This produces a discriminating prediction. Two learners can share the same buffer, model and compute budget yet diverge because their selectors construct different effective histories. Compare them on old-task retention, new-task learning, minority-case coverage and calibration. If their outcomes remain identical across strong distribution shifts, the proposed priority fields add little. If they diverge in the predicted directions, selection is not bookkeeping. It is part of the learning mechanism.

flowchart LR
  A["The interval that learns"] --> B["What the night chooses"]
  B --> C["Replaying is not copying"]
  C --> D["Build the offline learner"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#007D68,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Replaying is not copying

If a night merely duplicated the day, consolidation would be an expensive backup service. Useful offline cognition does something more selective. It may preserve a fragile detail, integrate an episode into a schema, weaken an obsolete association, infer a relation that was never directly observed or simulate a future path. The same freedom creates the main risk: a system capable of recombination is also capable of distortion.

Dreams sit beside the mechanism, not on top of it

Dream reports often contain fragments of recent experience, transformed rather than replayed as literal episodes. Task-related dreaming has been associated with better subsequent performance in navigation studies (Wamsley et al., 2010), and a later meta-analysis found a positive relation between learning-related dream content and memory outcomes across studies (Hudachek & Wamsley, 2023). These results support a link between dreaming and offline processing. They do not yet show that consciously experiencing the dream caused the gain.

Dream research faces a severe observation filter. A morning report depends on whether dreaming occurred, whether the person awakened near it, whether the content survived state transition and whether it can be expressed. Dreaming is reported from non-REM as well as REM sleep, although vivid recall is often associated with REM-rich periods. Individual differences in recall further complicate “no dream” as a control. The scientifically disciplined position is neither dismissal nor metaphysical inflation: dreams are valuable phenomenological data, but their causal role remains contested.

Recombination can create a shortcut

This is a crucial distinction. Consolidation can change the geometry of knowledge rather than only the strength of stored items. A compressed shortcut saves future computation, but it hides its derivation. When B changes, an explicit A→B→C chain can be revised locally. A cached A→C relation may persist. Offline abstraction purchases speed with update debt.

Forgetting is part of the bargain

Imagine a system that never loses a detail. It retains every view of every object, every obsolete route and every accidental correlation. Such completeness sounds desirable until the system must choose. Retrieval becomes crowded, old contingencies compete with current ones and a rare but valid exception is indistinguishable from a one-off error. Useful memory therefore depends on controlled loss as well as preservation.

Offline transformation can weaken a trace, absorb it into a schema or make it reachable only through a summary. That may improve generalisation while reducing episodic fidelity. The trade-off is visible in the A→C shortcut. A direct relation makes inference cheap, but the intermediate evidence becomes less available to revision. For human memory, this is one reason that a later confident answer is not a perfect record of the learning episode. For engineered systems, it is a reason not to treat a changed model as the only surviving history.

Keep two things separate. The evidence archive should preserve authorised source traces, provenance and adjudication according to retention rules. The learner may compress, weight or forget those traces in its parameters. When a shortcut fails, the archive allows reconstruction without pretending that the model still contains a faithful copy. Biological memory does not offer this clean separation, but engineered systems can and often should.

The boundary is consequential. If the task is poetry, associative recombination may be valuable even when provenance is diffuse. If the task is medication dosing, legal entitlement or aircraft inspection, a fluent abstraction must remain subordinate to verifiable source evidence. The same offline mechanism should not be granted the same commit authority across these domains.

When selection becomes rumination

For a person, repeated internally generated thought can support planning or become perseverative rumination. For a machine, prioritised replay can focus learning on high-error transitions, but error is also where corrupted labels, adversarial examples and model misspecification concentrate. Experience replay helped stabilise deep reinforcement learning by breaking correlations in successive experience (Mnih et al., 2015). Prioritised experience replay then made the sampling distribution explicit by favouring transitions with larger learning signals (Schaul et al., 2016).

Generative replay changes a second variable. Instead of retaining every old trace, a model produces pseudo-examples representing earlier tasks and interleaves them with new data (Shin et al., 2017). This can reduce storage pressure, but the learner now depends on the generator’s omissions. A class that the generator slightly under-represents receives fewer future updates, so the omission can compound. Generated traces therefore need generator identity, source-task coverage and comparison against held-out real traces. Self-produced experience is a proposal, not independent evidence.

The dream analogy is tempting here, but it should remain bounded. Dreams are not synthetic training batches in any established formal sense. The preserved relation is narrower: both biological and artificial systems may use internally generated content to revisit experience when direct action is absent. The comparison fails at phenomenology, physiology, objective function and implementation.

Build the offline learner

The practical contribution of this paper is an offline learning kernel with four explicit operations. It is not a brain model. It is a control structure for any system that learns from stored or generated traces while live action is paused.

Gate

Define which external inputs and live effects are disabled during the update.

Select

Construct a replay distribution from weakness, novelty, future value, coverage and provenance.

Transform

Choose whether traces are copied, interleaved, recombined, compressed or generated.

Commit

Evaluate the candidate learner against retention, adaptation, calibration and safety invariants before release.

The kernel makes a hidden fact visible. Every offline learner already has these operations, even when they are implicit. “Train on yesterday’s logs” is a selector. “Summarise the logs first” is a transform. “Replace the live model at midnight” is a commit policy. Naming them creates places to test.

Separate the learner from the release decision

The four operations form two different planes. Gate, Select and Transform decide what the candidate learner experiences. Commit decides whether the resulting change may affect the world. Mixing these planes creates a dangerous shortcut: because training completed without an exception, the new model is assumed fit for release. Successful optimisation says only that the candidate moved towards its training objective.

The gate first freezes live authority and validates traces. This includes model version, observation time, outcome source, consent or entitlement where relevant, and any expiry condition. The selector then constructs the replay distribution. The transform chooses ordering, interleaving, augmentation, compression or generation. Each transformed item should retain a lineage to its source or generator. The candidate update runs in an isolated workspace with a reproducible seed and configuration.

Commit requires a different path. Tests should cover the intended gain, old capabilities that must survive, groups that could be hidden by an average, calibration, known rare events and deliberate replay poisoning. A readback confirms the model and configuration actually released. A rollback target is fixed before release, not improvised after an incident. The commit gate converts offline plasticity from self-modification into a governed proposal.

A negative gate result has several meanings. The replay policy may be wrong, the budget may be too small, the model may lack capacity, the traces may be untrustworthy or the proposed improvement may conflict with a protected invariant. Rejecting the candidate is therefore useful evidence. It prevents the system from converting every learning attempt into an operational fact.

The smallest useful formalism

Let the slow learner have parameters θ. Let each stored trace be T_i. The selector assigns probability q_i that trace i appears in an offline batch. The update is:

Here η is the offline learning rate, L is the loss induced by a trace, and q is the replay distribution. The equation assumes that sampled gradients are meaningful for the slow learner and that the trace labels or outcomes remain valid. Change q, and the same buffer teaches a different lesson.

Executable lab: selection changes the stability-plasticity trade-off

The lab below trains a small three-class linear learner on synthetic data. Phase A contains classes 0 and 1. Phase B contains classes 1 and 2, so class 0 disappears from the live stream. The learner then receives an offline budget. “Error only” samples high-loss traces, while “need + coverage” alternates classes and then favours high-loss items within each class. The model is deliberately small enough that you can see the trade-off rather than admire a benchmark.

A positive result means replay recovered some old-class performance without erasing the new task at the chosen budget. A negative result means that this selector, model and budget did not find a useful compromise. It does not prove that replay is useless. The lab cannot establish anything about dreams, neural oscillations or subjective experience. It isolates one causal idea: replay policy changes what is learned from a fixed history.

from __future__ import annotations

import math
import random
from dataclasses import dataclass

@dataclass(frozen=True)
class Trace:
    x1: float
    x2: float
    label: int
    phase: str


def softmax(values: list[float]) -> list[float]:
    peak = max(values)
    exps = [math.exp(v - peak) for v in values]
    total = sum(exps)
    return [v / total for v in exps]


def gaussian(rng: random.Random, mean: float, sd: float) -> float:
    return rng.gauss(mean, sd)


def make_data(seed: int, n: int = 80) -> tuple[list[Trace], list[Trace], list[Trace]]:
    rng = random.Random(seed)
    centres = {0: (-1.1, -0.3), 1: (1.1, -0.3), 2: (-0.8, 0.75)}

    def samples(classes: list[int], phase: str, count: int) -> list[Trace]:
        out: list[Trace] = []
        for label in classes:
            cx, cy = centres[label]
            for _ in range(count):
                out.append(Trace(gaussian(rng, cx, 0.45), gaussian(rng, cy, 0.45), label, phase))
        return out

    return samples([0, 1], "A", n), samples([1, 2], "B", n), samples([0, 1, 2], "test", 200)


def run(policy: str = "balanced", budget: int = 5, seed: int = 42) -> dict[str, float]:
    phase_a, phase_b, test = make_data(seed)
    rng = random.Random(seed + 1)
    weights = [[rng.gauss(0.0, 0.05) for _ in range(3)] for _ in range(3)]

    def probabilities(trace: Trace) -> list[float]:
        features = [trace.x1, trace.x2, 1.0]
        logits = [sum(features[i] * weights[i][c] for i in range(3)) for c in range(3)]
        return softmax(logits)

    def update(trace: Trace, rate: float) -> None:
        features = [trace.x1, trace.x2, 1.0]
        probs = probabilities(trace)
        for i, feature in enumerate(features):
            for c in range(3):
                gradient = feature * (probs[c] - (1.0 if c == trace.label else 0.0))
                weights[i][c] -= rate * gradient

    def loss(trace: Trace) -> float:
        return -math.log(max(probabilities(trace)[trace.label], 1e-12))

    for _ in range(20):
        rng.shuffle(phase_a)
        for trace in phase_a:
            update(trace, 0.05)

    for _ in range(30):
        rng.shuffle(phase_b)
        for trace in phase_b:
            update(trace, 0.06)

    buffer = phase_a + phase_b
    losses = [loss(trace) for trace in buffer]

    for step in range(budget * 40):
        if policy == "none":
            break
        if policy == "uniform":
            index = rng.randrange(len(buffer))
        elif policy == "error":
            scaled = [(value + 1e-4) ** 0.8 for value in losses]
            index = rng.choices(range(len(buffer)), weights=scaled, k=1)[0]
        else:
            target_class = step % 3
            candidates = [i for i, trace in enumerate(buffer) if trace.label == target_class]
            if rng.random() < 0.7:
                scaled = [(losses[i] + 0.05) ** 0.8 for i in candidates]
                index = rng.choices(candidates, weights=scaled, k=1)[0]
            else:
                index = rng.choice(candidates)
        update(buffer[index], 0.025)
        losses[index] = loss(buffer[index])

    correct = {0: 0, 1: 0, 2: 0}
    total = {0: 0, 1: 0, 2: 0}
    for trace in test:
        prediction = max(range(3), key=lambda c: probabilities(trace)[c])
        total[trace.label] += 1
        correct[trace.label] += int(prediction == trace.label)

    accuracy = {c: correct[c] / total[c] for c in range(3)}
    return {
        "old_retention": accuracy[0],
        "new_adaptation": (accuracy[1] + accuracy[2]) / 2,
        "overall": sum(accuracy.values()) / 3,
    }


if __name__ == "__main__":
    for name in ("none", "uniform", "error", "balanced"):
        print(name, run(name, budget=5, seed=42))

A configured-system example

This example keeps concepts that are often collapsed into “memory” separate. The episodic buffer stores traceable events. The slow model contains learned parameters. The current inspection map is world state. The evidence ledger records adjudication and test receipts. The candidate update has no authority to alter production until the commit gate succeeds.

The decision instrument

Before adding offline learning, locate the proposed system on two axes. Selection pressure asks how concentrated the replay distribution is. Transformation strength asks how far replayed content departs from recorded traces. Low values on both axes produce an archive. High selection with low transformation produces focused rehearsal. Low selection with high transformation produces broad simulation. High values on both axes produce powerful abstraction and the largest risk of self-confirming drift.