The canyon is made of questions, not tokens

A long sequence creates cost, but it does not tell us what must cross time intact. The hidden assumption behind “make the context larger” is that the past is one homogeneous object and every future use needs the same access to it.

Consider two histories, each containing a million tokens. In the first task, the only future question is “How many red alerts occurred?” A counter can replace the whole history without loss. In the second, the question may later be “What exact phrase preceded the third red alert?” The same counter is useless. Sequence length is identical. Required memory is not.

This gives a practical equivalence relation. Two histories may be merged when every permitted future query gives the same answer on both. They must remain distinct when at least one permitted query separates them. A memory failure is therefore a destructive collision between histories that the future still needs to distinguish.

The smallest useful memory is a partition of possible histories

The equivalence relation gives more than a metaphor. Suppose the set of possible histories is finite and every answer must be exact. The future-question family divides those histories into classes. Histories inside one class are interchangeable because no promised query separates them. Histories in different classes are not. Any adequate deterministic memory needs at least as many distinguishable states as there are classes. In bits, that is at least the ceiling of log₂ of the class count.

Take four binary events. There are sixteen possible histories. If the only query is the total number of ones, the histories collapse into five classes, corresponding to counts zero through four. Three bits can encode those five answers. If the future may request the exact four-event sequence, all sixteen histories remain separate and four bits are necessary. If it may ask only whether the first and last events match, two classes are enough. Compression is not intrinsically lossy; it is lossy relative to a question family.

The lower bound assumes exact deterministic answers, a known query family and no external source that can restore discarded information. Real systems often permit approximation. Then adequacy becomes a tolerance statement: the decoder must keep error below a declared threshold for a declared distribution of histories and queries. This makes measurement part of the contract. A summary that is excellent on frequent queries may still erase a rare safety exception. Average accuracy alone can hide that failure.

Queries also include future updates. An operational state must not only answer “What is the current status?” It must correctly apply the next authorised transition. Two histories that yield the same current value may still need different treatment when one includes an unclosed obligation, a pending compensation or a version conflict. The relevant equivalence classes are therefore defined by every future read and write the system promises to support, not by natural-language questions alone.

Published evaluations reinforce the distinction between nominal capacity and effective use. Models with long advertised windows often perform differently depending on where evidence appears, and performance falls as tasks add multiple needles, aggregation or multi-hop tracing. “Lost in the Middle”, RULER, LongBench v2 and NoLiMa each expose a different part of this gap [12][13][16][17]. These results do not show that long windows are useless. They show that being present in the input is not the same as being reliably available to the computation that matters.

flowchart LR
  A["The canyon is made of<br/>questions, not tokens"] --> B["Four ways to preserve the past"]
  B --> C["A matched temporal-canyon test"]
  C --> D["A system that remembers correctly"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#7B4CD8,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Four ways to preserve the past

The useful distinction is not “Transformer versus everything else”. It is the preservation operator implemented by the configured system. Architectures may combine operators, but each operator incurs a characteristic loss.

Attend when the relation is not known in advance

Self-attention keeps token representations available for query-dependent interaction. A query can emphasise one remote item, compare several items or distribute weight across many. This flexibility is why attention is attractive when the future relation cannot be specified ahead of time. The original Transformer made all-to-all interaction a direct layer operation [1].

The cost follows from that flexibility. Dense attention considers a quadratic number of token pairs during a full pass. Exact kernels such as FlashAttention reduce expensive movement through the hardware memory hierarchy, making the same attention operation much faster and less memory hungry in practice, but they do not change the underlying set of pairwise interactions [8]. Sparse designs such as Longformer and BigBird reduce edges through local windows, global tokens or selected patterns [4][5]. That is more than an optimisation. It changes which paths information can take.

Efficient attention and memory semantics are separate questions. One asks how cheaply an interaction pattern can run. The other asks which distinctions remain accessible. A faster exact kernel may make full attention feasible at a larger length. A sparse pattern may be cheaper still, but only if its connectivity matches the task.

Retrieve when a small addressable subset is enough

Retrieval separates a large archive from the small working set presented to the model. The archive may hold raw documents, chunks, key-value pairs or internal representations. A query selects a few candidates, and the model reasons over them. Retrieval-augmented generation made this separation explicit for knowledge-intensive tasks, while Memorizing Transformers and Landmark Attention explored retrieval using model representations or learned landmarks [6][10][11].

Retrieval is strongest when relevance is sparse and addressable. Exact identifiers, stable entities and distinctive semantic cues create good access paths. It is weaker when the required answer depends on complete coverage, subtle order, absence, or an aggregate over items that top-k selection omits. Chunking can cut a relation in half. Embeddings can miss a paraphrase. A retriever can return a convincing superseded record. NoLiMa is especially useful here because it reduces literal overlap between questions and relevant evidence, making latent association rather than string matching carry more of the burden [17].

Retrieval preserves addressability, not completeness. It can be made more complete by retrieving more, by adding structured filters or by querying iteratively. Each repair changes cost and, in some cases, turns the retriever into a more explicit database query system.

Recur and compress when a stable statistic can carry the task

Recurrence specifies an update path: new input changes a carried state. Compression specifies what distinctions are discarded when that state is formed. They often travel together, but they are not the same idea. A recurrent system may carry a large uncompressed memory bank. A compression stage may summarise old segments without recurrent token-by-token processing.

Transformer-XL introduced segment-level recurrence; the Compressive Transformer added a lower-resolution store for older activations; Recurrent Memory Transformer passed learned memory tokens between segments; Infini-attention combined local attention with a compressive long-term memory [2][3][9][15]. State-space models take a more direct route. S4 and Mamba process sequences through structured state updates, with Mamba making propagation and forgetting input dependent [7][14].

A fixed state can be exceptionally efficient when the future depends on a compact sufficient statistic: a count, filter state, low-dimensional dynamical estimate, recent mode or slowly changing latent condition. It becomes brittle when the query family grows after compression. A preliminary study of several recurrent long-context language models found that selecting one relevant chunk could outperform processing the full long context, suggesting that fixed-state memories were not always exploiting the available horizon as intended [18]. That observation is not a proof of a universal architectural limit, but it is consistent with state overflow and destructive collision.

Carry explicit state when truth can change and actions depend on it

A transcript is a record of statements. An explicit state store is a governed representation of what the system currently treats as true. It has a schema, identity, version, update rule and authority boundary. A field such as breaker_17.isolated = true is not equivalent to a retrieved sentence saying “breaker 17 is isolated”. The sentence may be a proposal, a stale report, a hypothetical, an error or a superseded observation.

Persistent state is therefore not merely memory that survives a process restart. It is semantic persistence under controlled change. The system knows which writes are permitted, what event caused the transition, which version was read, and whether the effect was verified. An event ledger may preserve history for audit; the current state view intentionally collapses superseded events according to declared rules.

Do not confuse a cache with a durable memory contract

During autoregressive generation, a Transformer commonly keeps key and value activations from earlier tokens so they do not need to be recomputed at every step. This KV cache can persist across many generated tokens and can consume substantial device memory. Serving systems such as PagedAttention improve how those blocks are allocated and shared [20]. Yet a KV cache is still an execution cache. Its entries have layer-specific numerical meaning, not a declared business schema, write authority or independent truth status.

The distinction matters during restart, migration and audit. A cache may be evicted without changing the authoritative world, provided it can be rebuilt from the prefix. A recurrent hidden state may be impossible to rebuild unless the stream is replayed. A retrieval archive may survive indefinitely while containing mutually inconsistent statements. A typed state record may be tiny but decisive because later actions are required to read its current version. Persistence has at least three axes: survival across computation, recoverability from evidence, and semantic authority under change.

Hybrids should decompose obligations, rather than only stack fashionable components

Combining mechanisms is justified when the future-question family itself decomposes. A local attention window can handle immediate syntax and nearby causal order. Retrieval can supply a small set of remote records. A recurrent state can carry a trend estimate. A typed store can answer current status. The model may attend jointly over the outputs, but each source should retain its own semantics and failure signal.

A weak hybrid simply adds everything to the prompt. It retrieves documents, serialises state, appends summaries and trusts the model to infer which source is current. That recreates the original ambiguity at a larger scale. A strong hybrid gives each source a typed interface. Retrieved evidence includes provenance and retrieval score. Recurrent statistics identify their update rule and coverage interval. State reads include version and validity. The model proposal cites which inputs support it, while an authority component checks the facts that govern action.

The break condition is also compositional. If retrieval recall falls, the system should not silently substitute a recurrent summary. If state freshness is unknown, a fluent answer over old context is not a repair. If a relation spans chunks that the sparse topology cannot connect, a larger top-k may help only when the missing chunks can be found. A hybrid is reliable when one mechanism’s failure remains visible rather than being laundered through another mechanism’s prose.

A matched temporal-canyon test

Architectures are often compared on different datasets, budgets and training recipes. The included lab instead holds one synthetic event stream fixed and changes only the future obligation.

The stream contains 4,096 signals, notes and asset-status statements. Some status statements are proposals; others are committed updates. The harness asks 256 questions in each of four families:

1. Exact note: recover the phrase stored under a note identifier.
2. Ordered relation: count how often one tag immediately preceded another across the complete stream.
3. Global count: count all events carrying a selected tag.
4. Current committed state: return the latest committed status for an asset while ignoring later proposals.

Five configured mechanisms receive the same events. The full-history replay oracle retains everything and provides an information-preservation upper bound. A 128-event attention window keeps only the recent tail. Top-8 retrieval provides exact note lookup and a small candidate set. The recurrent summary keeps tag counts, selected pair counts, thirty-two notes and twenty-four recent asset states. The typed store keeps only the latest committed status for every asset.

The top-8 retriever recovers every exact note because note identifiers are perfect addresses. It fails the global count because the model sees at most eight matches, not the posting-list cardinality. Its state accuracy is 76.6 per cent because it ranks recent status text without understanding which records are committed. Add a commit filter and version rule, and the mechanism begins to implement explicit state semantics.

The recurrent summary answers every global-count query because the required statistic is updated at each event. It performs poorly on arbitrary notes and states because its fixed slots intentionally merge most histories. The typed state store is perfect on committed current status and silent elsewhere. Silence outside the schema is a virtue when pretending to remember would be dangerous.

Read the matrix as a set of interventions

The useful result is not the ranking. It is what happens when one preservation variable changes. Increase the attention window from 128 events towards the full stream and exact-note or old-state accuracy rises only when the relevant item enters the window. The global count remains wrong until the complete counting horizon is visible or the model receives an aggregate. The relation query remains wrong whenever part of the required order lies outside the retained band.

Increase retrieval k and more candidates become visible. Exact-note performance is already saturated because the identifier provides a direct address. Count accuracy does not become exact merely because k grows from eight to sixteen; the interface still withholds the fact that more matches exist. State accuracy improves when the latest committed event appears among candidates, but a newer uncommitted proposal can still outrank it. A metadata filter for committed = true changes the causal mechanism, not just the parameter.

Add note slots to the recurrent summary and arbitrary note recall increases roughly with retained coverage. To guarantee any note query, storage must eventually scale with the number of distinguishable notes or acquire an external addressable store. Add counters for more tag pairs and ordered-relation accuracy rises for those configured pairs. This is useful when the pair family is known. It does not solve a later request for an untracked relation.

Extend the explicit state schema with a pressure-warning counter and the state store can answer that aggregate exactly, provided every relevant event reaches the update path once and only once. At that point the store has not “become intelligent”. Its schema now preserves another equivalence class required by the future. This is why the labels in the matrix are less important than the interfaces and invariants behind them.

A good benchmark should make the loss recoverable by a specific intervention. If no one can say which retained distinction would repair a failure, the benchmark may be measuring a mixture of representation, optimisation, prompting and decoding. The harness deliberately strips most of those variables away so the information boundary is inspectable.

#!/usr/bin/env python3
"""Temporal obligations benchmark for Paper 48.

This synthetic harness holds one event stream fixed and compares configured
representations on four future-question families. It tests information
preservation under explicit budgets. It does not benchmark trained language
models, neural attention quality, or hardware throughput.
"""

from __future__ import annotations

import argparse
from collections import Counter, OrderedDict, defaultdict
from dataclasses import dataclass
from random import Random
from typing import Any, Iterable, Protocol

STATUSES = ("OPEN", "HOLD", "CLOSED")
TAGS = tuple(f"T{i:02d}" for i in range(16))


@dataclass(frozen=True)
class Event:
    step: int
    event_id: str
    kind: str
    key: str
    value: str
    tag: str
    committed: bool = True


class Memory(Protocol):
    name: str

    def note(self, key: str) -> str | None: ...
    def adjacency(self, first: str, second: str) -> int | None: ...
    def count(self, tag: str) -> int | None: ...
    def state(self, key: str) -> str | None: ...


def build_stream(n: int = 4096, seed: int = 48) -> list[Event]:
    """Generate a deterministic mixture of signals, notes, and status records."""
    if n < 256:
        raise ValueError("n must be at least 256 so every query family is populated")

    rng = Random(seed)
    events: list[Event] = []
    note_number = 0

    for step in range(n):
        draw = rng.random()
        tag = rng.choice(TAGS)

        if draw < 0.48:
            kind = "signal"
            key = f"sensor-{rng.randrange(96):03d}"
            value = f"reading-{rng.randrange(1000):03d}"
            committed = True
        elif draw < 0.68:
            kind = "note"
            key = f"note-{note_number:04d}"
            note_number += 1
            value = f"phrase-{rng.randrange(10_000_000):07d}"
            committed = True
        else:
            kind = "status"
            key = f"asset-{rng.randrange(128):03d}"
            value = rng.choice(STATUSES)
            committed = rng.random() < 0.68

        events.append(
            Event(
                step=step,
                event_id=f"E{rng.randrange(10**8):08d}",
                kind=kind,
                key=key,
                value=value,
                tag=tag,
                committed=committed,
            )
        )

    return events


class Replay:
    """Exact full-history access used as an information-preservation oracle."""

    name = "Full-history replay oracle"

    def __init__(self, events: list[Event]) -> None:
        self.events = events

    def note(self, key: str) -> str | None:
        for event in self.events:
            if event.kind == "note" and event.key == key:
                return event.value
        return None

    def adjacency(self, first: str, second: str) -> int:
        return sum(
            1
            for left, right in zip(self.events, self.events[1:])
            if left.tag == first and right.tag == second
        )

    def count(self, tag: str) -> int:
        return sum(event.tag == tag for event in self.events)

    def state(self, key: str) -> str | None:
        for event in reversed(self.events):
            if event.kind == "status" and event.key == key and event.committed:
                return event.value
        return None


class SlidingAttention(Replay):
    """A bounded recent window standing in for cost-constrained attention."""

    def __init__(self, events: list[Event], width: int = 128) -> None:
        if width < 1:
            raise ValueError("attention width must be positive")
        self.width = width
        self.events = events[-width:]
        self.name = f"{width}-event attention"


class Retrieval:
    """An addressable archive whose model-facing interface exposes top-k items."""

    def __init__(self, events: list[Event], k: int = 8) -> None:
        if k < 1:
            raise ValueError("retrieval k must be positive")
        self.k = k
        self.name = f"Top-{k} retrieval"
        self.by_note: dict[str, Event] = {}
        self.by_tag: dict[str, list[Event]] = defaultdict(list)
        self.by_key: dict[str, list[Event]] = defaultdict(list)

        for event in events:
            if event.kind == "note":
                self.by_note[event.key] = event
            self.by_tag[event.tag].append(event)
            if event.kind == "status":
                self.by_key[event.key].append(event)

    def note(self, key: str) -> str | None:
        event = self.by_note.get(key)
        return event.value if event else None

    def adjacency(self, first: str, second: str) -> int:
        # The model sees only the most recent k matches for each tag.
        candidates = {
            event.step: event
            for event in self.by_tag[first][-self.k :]
            + self.by_tag[second][-self.k :]
        }
        ordered = [candidates[index] for index in sorted(candidates)]
        return sum(
            1
            for left, right in zip(ordered, ordered[1:])
            if right.step == left.step + 1
            and left.tag == first
            and right.tag == second
        )

    def count(self, tag: str) -> int:
        # The interface does not reveal posting-list cardinality.
        return len(self.by_tag[tag][-self.k :])

    def state(self, key: str) -> str | None:
        # Recency is used without commit authority, so proposals can win.
        matches = self.by_key.get(key, [])[-self.k :]
        return matches[-1].value if matches else None


class RecurrentSummary:
    """A fixed-size, task-shaped summary updated once per event."""

    name = "Fixed recurrent summary"

    def __init__(
        self,
        events: list[Event],
        note_slots: int = 32,
        state_slots: int = 24,
        pair_slots: int = 32,
    ) -> None:
        if min(note_slots, state_slots, pair_slots) < 0:
            raise ValueError("slot counts cannot be negative")

        all_pairs = [(left, right) for left in TAGS for right in TAGS]
        self.selected_pairs = set(all_pairs[:pair_slots])
        self.tag_counts: Counter[str] = Counter()
        self.pair_counts: Counter[tuple[str, str]] = Counter()
        self.notes: dict[str, str] = {}
        self.states: OrderedDict[str, str] = OrderedDict()
        self.note_slots = note_slots
        self.state_slots = state_slots

        previous: Event | None = None
        notes_seen = 0

        for event in events:
            self.tag_counts[event.tag] += 1

            if previous and (previous.tag, event.tag) in self.selected_pairs:
                self.pair_counts[(previous.tag, event.tag)] += 1
            previous = event

            if event.kind == "note":
                # Deterministic sparse retention: one note in every 25.
                if notes_seen % 25 == 0 and len(self.notes) < note_slots:
                    self.notes[event.key] = event.value
                notes_seen += 1

            if event.kind == "status" and event.committed:
                if event.key in self.states:
                    self.states.pop(event.key)
                self.states[event.key] = event.value
                while len(self.states) > state_slots:
                    self.states.popitem(last=False)

    def note(self, key: str) -> str | None:
        return self.notes.get(key)

    def adjacency(self, first: str, second: str) -> int:
        return self.pair_counts.get((first, second), 0)

    def count(self, tag: str) -> int:
        return self.tag_counts[tag]

    def state(self, key: str) -> str | None:
        return self.states.get(key)


class ExplicitState:
    """A typed current-state view that advances only on committed updates."""

    name = "Typed current state"

    def __init__(self, events: list[Event]) -> None:
        self.current: dict[str, tuple[int, str]] = {}
        for event in events:
            if event.kind == "status" and event.committed:
                self.current[event.key] = (event.step, event.value)

    def note(self, key: str) -> None:
        return None

    def adjacency(self, first: str, second: str) -> None:
        return None

    def count(self, tag: str) -> None:
        return None

    def state(self, key: str) -> str | None:
        item = self.current.get(key)
        return item[1] if item else None


def exact_accuracy(got: Iterable[Any], expected: Iterable[Any]) -> float:
    pairs = list(zip(got, expected))
    if not pairs:
        raise ValueError("at least one query is required")
    return 100.0 * sum(actual == target for actual, target in pairs) / len(pairs)


def run_benchmark(
    n: int = 4096,
    seed: int = 48,
    queries: int = 256,
    attention_width: int = 128,
    retrieval_k: int = 8,
) -> list[tuple[str, float, float, float, float]]:
    if queries < 1:
        raise ValueError("queries must be positive")

    rng = Random(seed + 1)
    events = build_stream(n=n, seed=seed)
    oracle = Replay(events)

    notes = [event.key for event in events if event.kind == "note"]
    assets = sorted(
        {
            event.key
            for event in events
            if event.kind == "status" and event.committed
        }
    )
    if not notes or not assets:
        raise RuntimeError("generated stream did not contain required event types")

    note_queries = [rng.choice(notes) for _ in range(queries)]
    pair_queries = [(rng.choice(TAGS), rng.choice(TAGS)) for _ in range(queries)]
    count_queries = [rng.choice(TAGS) for _ in range(queries)]
    state_queries = [rng.choice(assets) for _ in range(queries)]

    expected = {
        "note": [oracle.note(key) for key in note_queries],
        "relation": [oracle.adjacency(a, b) for a, b in pair_queries],
        "count": [oracle.count(tag) for tag in count_queries],
        "state": [oracle.state(key) for key in state_queries],
    }

    systems: list[Memory] = [
        oracle,
        SlidingAttention(events, width=attention_width),
        Retrieval(events, k=retrieval_k),
        RecurrentSummary(events),
        ExplicitState(events),
    ]

    rows: list[tuple[str, float, float, float, float]] = []
    for system in systems:
        rows.append(
            (
                system.name,
                exact_accuracy(
                    (system.note(key) for key in note_queries), expected["note"]
                ),
                exact_accuracy(
                    (system.adjacency(a, b) for a, b in pair_queries),
                    expected["relation"],
                ),
                exact_accuracy(
                    (system.count(tag) for tag in count_queries), expected["count"]
                ),
                exact_accuracy(
                    (system.state(key) for key in state_queries), expected["state"]
                ),
            )
        )
    return rows


def print_table(rows: list[tuple[str, float, float, float, float]]) -> None:
    print(
        f"{'Mechanism':30} {'Exact note':>11} {'Relation':>11} "
        f"{'Count':>11} {'State':>11}"
    )
    for name, note, relation, count, state in rows:
        print(f"{name:30} {note:10.1f}% {relation:10.1f}% {count:10.1f}% {state:10.1f}%")


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="Compare memory representations on matched temporal obligations."
    )
    parser.add_argument("--events", type=int, default=4096, help="stream length")
    parser.add_argument("--queries", type=int, default=256, help="queries per family")
    parser.add_argument("--seed", type=int, default=48, help="random seed")
    parser.add_argument("--attention-width", type=int, default=128)
    parser.add_argument("--retrieval-k", type=int, default=8)
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    rows = run_benchmark(
        n=args.events,
        seed=args.seed,
        queries=args.queries,
        attention_width=args.attention_width,
        retrieval_k=args.retrieval_k,
    )
    print_table(rows)


if __name__ == "__main__":
    main()

stateDiagram-v2
  [*] --> Claim
  state "The smallest useful memory is<br/>a partition of possible<br/>histories" as Claim
  state "Attend when the relation is<br/>not known in advance" as Test
  state "Retrieve when a small<br/>addressable subset is enough" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

A system that remembers correctly

A realistic long-running assistant usually needs a composition, not a winner. The composition should follow temporal obligations and keep proposal, evidence, state and authority separate.

Return to the operations assistant. A recent local window supports immediate dialogue and nearby event relations. A retrieval index supplies exact historical notes with source identifiers. A recurrent telemetry component maintains counts, rates and filters. An operational state store holds current asset states and versions. The language model proposes an interpretation or action, but a deterministic authority path decides whether the proposal is permitted against fresh state.

An implementation protocol for temporal obligations

A design team can turn the framework into a concrete review in five passes.

1. Inventory temporal objects. Separate recent conversation, raw evidence, derived statistics, current world state, pending proposals, permissions and effect receipts. Do not place them all under a column called “memory”.
2. Declare future operations. For each object, list the reads, comparisons, aggregates and updates that must remain possible. Include negative queries such as “Did no alert occur?”, because absence often requires exhaustive coverage.
3. Choose a preserving representation. Use attention when relations are open-ended, retrieval when addresses are sparse, recurrent updates when a sufficient statistic is stable, and typed state when supersession or authority matters. Keep an immutable event record when later reconstruction or audit is required.
4. Attach invariants and evidence. A recurrent counter needs exactly-once update assumptions. Retrieval needs recall tests and source identifiers. State needs subject, field, value, version, validity, authority and evidence reference. Actions need typed requests, idempotency and effect receipts.
5. Inject collisions. Construct paired histories that each mechanism is likely to merge. Swap order, insert a rare exception, add a superseding update, restart the process, remove one retrieved chunk and delay a state write. Check whether the promised decision changes.

The protocol makes cost comparison more honest. Dense attention may be expensive but operationally simple for a bounded document. Retrieval adds indexing, recall evaluation and provenance. Recurrence adds state training, reset semantics and overflow risk. Explicit state adds schema governance and transaction logic. The cheapest component in isolation may create the most expensive failure investigation when its obligation is unclear.

Retention policy also follows the distinction. Raw archives can carry privacy and storage costs. Recurrent summaries may be sufficient for telemetry that has no later forensic value. Authoritative state may need only the current view plus a controlled event ledger. Keeping every token forever is not automatically safer. It may increase exposure while still failing to identify which statement is current.

This layout also makes failure diagnosis sharper. A wrong answer may come from failed retrieval, a lossy recurrent statistic, stale state, an invalid model proposal, a permissive authority decision or an unverified effect. “The model forgot” is too vague to locate the fault.

sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: The canyon is made of questions, not tokens
  M->>E: The smallest useful memory is a partition of possible histories
  E-->>M: compare rival explanations
  E->>D: Preserve every distinction that can change a decision
  D-->>Q: revise scope or act

Preserve every distinction that can change a decision

Do not begin a long-context design review by asking, “How many tokens can the model accept?” Begin by listing the future obligations. Must the system compare arbitrary past items? Recover one exact record? Maintain a sufficient statistic? Track a mutable fact? Use that fact to authorise an irreversible effect?

Then apply the collision test. For each proposed memory boundary, construct two histories that it maps to the same representation. Ask whether any promised future query should distinguish them. If yes, the design has already lost required information. Increase capacity, change the representation, narrow the promise or add another mechanism.

Use attention for flexible relation, retrieval for sparse address, recurrence for stable statistics, and explicit state for governed truth. Optimise each only after its obligation is clear. Flash kernels, sparse patterns, state-space layers, compression memories and vector indexes are valuable engineering choices, but none can repair a mismatch between what was preserved and what the future asks.

The practical consequence is a different architecture conversation. A system may use a smaller active context while becoming more reliable, because exact history sits behind retrieval, aggregates live in recurrent state, and consequential facts live in a versioned operational store. Conversely, a compact recurrent model may need an addressable complement when unanticipated exact recall matters. The aim is not minimum memory in isolation. It is minimum cost subject to preserving every distinction that can change a decision.