Part I
The mask defines the legal past

A decoder-only language model assigns a probability to a sequence by decomposing one difficult joint prediction into a chain of next-token predictions. For tokens x₁ to x_T, the factorisation is:

Training nevertheless processes many positions in parallel. The apparent contradiction disappears once computation and information are separated. Matrix operations for all positions can run together, while a triangular mask removes every attention edge that points forward. The original Transformer paper explicitly uses this masking to preserve the autoregressive property, setting illegal attention logits to negative infinity before softmax (Vaswani et al.).

For one attention head, the operation can be written as:

The mask is therefore more than a speed option or a display convention. It identifies which conditional model is being learned. A bidirectional encoder, a prefix language model and a left-to-right decoder can share attention machinery while representing different dependency structures because their masks differ.

Parallel training is sometimes described as if the model “generates every token at once”. That merges two different timelines. On the computational timeline, a device evaluates many query rows together. On the dependency timeline, row t still conditions only on rows before it. The mask lets hardware parallelise lawful conditional predictions without granting any prediction access to its future target.

The distinction matters for debugging. A low training loss can survive a mask defect because leaked targets make the optimisation problem easier. Generation then exposes the mismatch: the future columns are absent, so the model is asked to operate under a dependency graph it did not learn. A robust test should therefore compare the implemented mask against the intended factorisation, verify target shifting, and include a synthetic sequence whose answer becomes trivial if even one future edge opens.

The mask also shapes gradient routes. During training, a future representation that enters an earlier query can influence the loss through attention and then through shared parameters. Removing the leaked edge at inference does not merely remove a convenience; it removes a path the optimiser may have relied upon. Training-time legality and decode-time legality must be the same contract.

Thought experiment: the completed exam

Give two students the same partially completed arithmetic worksheet. Every row contains a question followed by its answer. Student A may look only at previous rows while filling the next answer. Student B receives the complete sheet and may look one cell to the right, where the target answer already appears.

Both can be trained in parallel across rows. Vary only future visibility. Student A must exploit the prefix; Student B can copy. If the evaluation later presents a blank final answer, Student B's training loss was evidence of leakage, not predictive competence. The causal mask plays the role of the opaque strip covering unearned answers.

Now restore the mask but remove the one-position shift between inputs and targets. The representation at row t can contain token x_t while the loss also asks for x_t. The diagonal becomes another route to the answer. The mask and target alignment jointly enforce the factorisation. Testing only that a triangular tensor exists is insufficient.

Causality here is statistical, not metaphysical and not a security permission. The mask says which token positions can influence another inside the forward pass. It does not establish whether a prompt is trusted, whether a tool call is authorised or whether cached state belongs to the same user.

flowchart TD
  A["Three decoders continue the<br/>same sentence"] --> B["Part I The mask defines the<br/>legal past"]
  B --> C["Part II The cache is exact reuse"]
  C --> D["Part III Faster generation has<br/>a fidelity boundary"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#D95545,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Part II
The cache is exact reuse

After the prompt has been processed, generation samples one token, appends it, then asks for another distribution. A naive worker sends the whole growing prefix through every layer on every step. Earlier positions produce the same per-layer keys and values each time because a causal model prevents later tokens from changing their representations.

That invariance creates the cache. Each attention layer stores the keys and values already computed for legal past positions. On the next step it computes a query, key and value for the new token, appends the new key and value, and lets the new query attend over the stored sequence. Queries are usually not retained because a query serves the position that issued it once; its key and value may be consulted by every later query.

The equality is local to every layer. Suppose the cached worker has processed tokens 1 through t. At layer l, its stored keys and values equal those from a full causal forward pass over the same prefix. The next token's layer input is also equal by induction from layer l minus one. The new query sees the same past keys and values in the same order, so it produces the same attention output. Repeating this argument through all layers gives the same next-token logits, subject to floating-point implementation differences.

What kind of “same” is being claimed?

Teams often test only that two decoders emit the same visible string. That can be too weak or unnecessarily strong. A small logit change may leave greedy output unchanged while changing sampled continuations. Conversely, two mathematically equivalent kernels can differ in their last floating-point bits because reductions occur in another order. The release claim should name its equivalence level.

For an ordinary exact cache, tolerance-bounded logit parity is the most revealing default. It tests the mechanism before sampling hides small differences. Sequence tests then exercise cache growth, stopping and random-state handling. Task tests remain necessary for numerical kernels and essential for any approximation, but they should not replace the lower-level comparison that localises a defect.

The identity of reusable state is stricter than a human-readable prompt. Two requests may display the same characters while using different token IDs, adapters, rotary scaling, attention windows or model revisions. Each difference can change keys and values. Cache identity is therefore a tuple of the processed token prefix and every configuration item that can affect the per-layer state, not simply a string hash.

A cache is valid because past per-layer attention state is invariant to future tokens, not because “the model remembers”. The cache is workflow state for a particular model execution. It should not be confused with episodic memory, enterprise knowledge or an evidence ledger.

Minimal worked example: four tokens, one new row

Take a four-token prompt. A full causal attention matrix contains ten legal score cells: one in row one, two in row two, three in row three and four in row four. The final row alone determines the prompt's next-token logits after the remaining layer operations. Once the first four tokens are cached, processing token five requires five score cells, all in one row. Recomputing the five-token prefix would evaluate fifteen legal cells, including ten whose outputs are unchanged.

Continue for many steps and the difference compounds. If P is prompt length and G is the number of sampled tokens, a deliberately naive full-recompute plan evaluates a growing triangle before each sample. Cached decoding evaluates the prompt triangle once, then one row for each later token:

The bill moves from compute to memory

Exact reuse is not free. For a conventional cache, approximate storage in bytes is:

A synthetic 32-layer model with eight KV heads, head width 128, an 8,192-token context and two-byte elements needs exactly 1 GiB of KV state per sequence before allocator overhead. With 32 KV heads it needs 4 GiB. This is why multi-query attention and grouped-query attention reduce serving memory and bandwidth by sharing key-value heads, while introducing architecture-level quality trade-offs that must be settled during model design (Shazeer; Ainslie et al.).

Part III
Faster generation has a fidelity boundary

“Uses a KV cache” covers several unlike interventions. One stores exact tensors in a different layout. Another shares an identical prefix across requests. A third keeps two-bit approximations. A fourth drops old tokens. A fifth lets a small model propose tokens and asks the target model to accept or reject them. These changes can all reduce cost, yet they carry different claims about the output distribution.

The first release question is not “How much faster?” but “What, if anything, has changed in the target model's conditional distribution?”

Exact changes still need parity tests

FlashAttention is an IO-aware exact attention algorithm: it changes tiling and memory traffic rather than approximating the dense result (Dao et al.). PagedAttention changes how KV blocks are allocated and shared so dynamic sequences waste less memory (Kwon et al.). Prefix caching reuses blocks when token prefixes match, reducing repeated prefill work; both vLLM and TensorRT-LLM document this behaviour (vLLM; TensorRT-LLM).

These are semantic-preserving intentions, not immunity from implementation defects. Parity must cover token IDs, model and adapter revision, position IDs or rotary offsets, attention window, KV head mapping, cache dtype, padding convention and any mask that changes by layer. Sampling tests must also hold the random stream constant if exact token-sequence comparison is required.

Speculative decoding illustrates a subtler exactness argument. A cheap model proposes several tokens; the target model evaluates them in parallel, accepts a valid prefix and corrects the sampling distribution when needed. Under the published algorithm, the target distribution is unchanged even though proposals are approximate (Leviathan et al.). Removing verification would cross the fidelity boundary.

The strongest objection: outcomes matter more than internal equality

A service owner can reasonably object that users do not consume logits. If a quantised or pruned cache meets the task's quality, latency and cost thresholds, why privilege exact execution? For many bounded workloads, the objection is correct. Exactness is not a moral property, and an approximation can be the better architecture.

The distinction still matters because evidence transfers differently. Algebraic or algorithmic equivalence supports new prompts within the mechanism's domain without retesting every semantic case. An approximate cache earns only the envelope actually evaluated. Long-range copying, code closure, numerical consistency and rare safety-relevant dependencies may respond differently even when an average benchmark barely moves. Calling both paths “the same model” silently expands evidence beyond its tested boundary.

An observation could weaken this paper's practical recommendation. Suppose an approximation is evaluated across representative and adversarial strata, its divergence remains below a decision-relevant threshold, and later incidents show no concentration in the altered state. In that use case, insisting on exact cache parity would add little. The correct conclusion would be to keep the exact-versus-approximate label for traceability while governing release primarily through the demonstrated outcome envelope.

Thought experiment: the perfect and forgetful stenographers

Two stenographers listen to the same hearing. Both avoid replaying the entire recording before typing the next sentence. The perfect stenographer keeps an exact index of every earlier statement. The forgetful stenographer retains only the opening oath and the last ten minutes.

For a routine exchange about the current witness, their transcripts match. Now vary one causal feature: ask whether today's amount contradicts a figure stated two hours earlier. The perfect index preserves the relevant dependency. The sliding record cannot recover it. The second method is faster and may be adequate for local dialogue, but it no longer implements unrestricted attention over the full hearing.

StreamingLLM shows that naive sliding-window eviction can fail and that retaining initial attention-sink tokens can recover much of the behaviour in streaming settings (Xiao et al.). KIVI reports large memory and throughput gains from asymmetric two-bit KV quantisation with near-baseline aggregate quality on evaluated models and tasks (Liu et al.). Those results support useful approximations; they do not turn eviction or quantisation into algebraic identities.

A clean negative control is a request that asks for one next-token score and stops. Both workers must process the prompt once; there is no second decode step on which stored K and V can repay their allocation. A cache may still be produced by a generic runtime, but it has no causal opportunity to save repeated work. Likewise, reranking a fixed list with a bidirectional encoder uses a different dependency structure and should not inherit a decoder-cache argument by analogy.

Another boundary appears when the system is dominated by moving cache state rather than using it. Offloading a large cache to host memory, transmitting it between disaggregated workers or restoring it after pre-emption can cost more than recomputation for short remaining outputs. The right comparison is avoided work minus allocation, lookup, transfer and synchronisation cost under the actual residual generation length.

The companion NumPy lab supplies a negative control. It reduces theoretical attention score pairs by 75 times for one synthetic workload, yet its tiny cached implementation can be slower than full recomputation because Python calls and small matrix operations dominate. On a longer synthetic prefix, the crossover can reverse. Fewer operations are a mechanism, not a wall-clock guarantee.

A 2025 production-oriented evaluation reached a related warning at larger scale: KV compression can lower memory while failing to improve throughput in existing serving paths, and altered generations can become longer, increasing end-to-end latency (Gao et al.). Aggregate accuracy alone can miss sample-specific failures. The proper unit of evidence is the configured model, kernel, scheduler, hardware and workload.

stateDiagram-v2
  [*] --> Claim
  state "Figure 1. The triangle that<br/>makes next-token learning<br/>legal" as Claim
  state "Thought experiment, the<br/>completed exam" as Test
  state "Figure 2. Time-unrolled cache<br/>cutaway" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Part IV
Operate the cache as governed state

At service scale, the cache is not one tensor attached to one request. It is a changing pool of blocks shared by batches with different prompt lengths, generated lengths and arrival times. Prefill and decode also stress hardware differently: prefill processes many prompt tokens in parallel, while decode repeatedly processes one token per active sequence. Systems work on chunked prefill, continuous batching and phase disaggregation because optimising one phase can damage the latency of the other (Agrawal et al.; Zhong et al.).

Production-shaped worked scenario

Consider a synthetic internal assistant serving 24 concurrent conversations with the 32-layer, eight-KV-head configuration from Figure 4. At the maximum 8,192-token context, raw KV state alone can reach 24 GiB. Model weights, activations, allocator metadata and temporary kernel buffers remain outside that number.

The service uses paged blocks so a short conversation does not reserve its maximum length. A common system instruction is eligible for prefix reuse. Each request then appends private conversation tokens. The scheduler admits new prefills in chunks while protecting the token cadence of active decodes. When pressure rises, it may queue a request, offload blocks, lower admission or invoke an explicitly evaluated approximate-cache policy.

Paged allocation solves a physical-memory problem: variable-length sequences no longer need one contiguous maximum-sized reservation, and shared prefixes can refer to common blocks. It does not decide whether two logical requests are entitled to share those blocks or whether their state was produced by the same model configuration. Memory indirection removes fragmentation; it does not remove semantic identity.

The architecture decision is not “turn caching on”. It is a lifecycle with typed boundaries:

Prefix reuse deserves particular care. Equality should be evaluated over token IDs and every model-state attribute that affects K and V. Text equality before tokenisation is not enough; neither is a hash that ignores model revision or adapter selection. Official vLLM code exposes prefix-cache reset for flows where weights change, which makes the invalidation dependency concrete (vLLM API).

A cache handle should bind to this identity rather than trusting caller-supplied position or length alone. When a worker resumes, it can compare the handle's declared token range with the actual cache pages, reject gaps or duplicate appends, and rebuild from the authoritative token sequence when uncertainty remains. Recompute is often safer than guessing whether partially restored state is coherent.

Cross-request reuse is also an authority boundary. A cache manager may know that two prefixes are computationally identical while the application knows they belong to different tenants, classifications or retention policies. Computational sameness does not grant permission to share state. Tenancy and entitlement should either partition the cache namespace or be included in a policy check before reuse. This is practitioner synthesis rather than a claim that one public engine supplies the complete control.

Measure four clocks, not one

A single “latency” number hides different causal paths. Time to first token includes queueing and prefill. Time per output token reflects decode cadence. Throughput depends on batching and admission. Memory headroom determines how many sequences can coexist before eviction or queueing. A cache change can improve one and worsen another.

Executable artefact: cached-decoder parity benchmark

The companion programme implements a two-layer causal decoder with synthetic NumPy weights. It compares naive full-prefix generation with per-layer KV reuse, asserts identical generated tokens and near-identical logits, then introduces a wrong-position negative case that must diverge. It also reports deterministic score-pair counts and machine-dependent timings.

"""Exact-versus-broken KV-cache laboratory for a tiny causal decoder.

Python 3.10+ and NumPy 1.24+.
All weights, tokens, dimensions and timings are synthetic teaching fixtures.
"""
from __future__ import annotations

from dataclasses import dataclass
from time import perf_counter
from typing import List
import copy
import numpy as np

Array = np.ndarray


@dataclass
class LayerWeights:
    wq: Array
    wk: Array
    wv: Array
    wo: Array
    w1: Array
    w2: Array


class TinyDecoder:
    def __init__(
        self,
        *,
        vocab_size: int = 29,
        d_model: int = 24,
        n_heads: int = 3,
        n_layers: int = 2,
        max_positions: int = 512,
        seed: int = 7,
    ) -> None:
        if d_model % n_heads:
            raise ValueError("d_model must be divisible by n_heads")
        self.vocab_size = vocab_size
        self.d_model = d_model
        self.n_heads = n_heads
        self.head_dim = d_model // n_heads
        rng = np.random.default_rng(seed)
        scale = 1.0 / np.sqrt(d_model)
        self.token_embed = rng.normal(0.0, scale, (vocab_size, d_model))
        self.position_embed = rng.normal(0.0, scale, (max_positions, d_model))
        self.layers: List[LayerWeights] = []
        for _ in range(n_layers):
            self.layers.append(
                LayerWeights(
                    wq=rng.normal(0.0, scale, (d_model, d_model)),
                    wk=rng.normal(0.0, scale, (d_model, d_model)),
                    wv=rng.normal(0.0, scale, (d_model, d_model)),
                    wo=rng.normal(0.0, scale, (d_model, d_model)),
                    w1=rng.normal(0.0, scale, (d_model, 2 * d_model)),
                    w2=rng.normal(0.0, scale, (2 * d_model, d_model)),
                )
            )
        self.unembed = rng.normal(0.0, scale, (d_model, vocab_size))

    def _split_heads(self, x: Array) -> Array:
        # [time, d_model] -> [heads, time, head_dim]
        return x.reshape(x.shape[0], self.n_heads, self.head_dim).transpose(1, 0, 2)

    @staticmethod
    def _softmax(scores: Array) -> Array:
        shifted = scores - np.max(scores, axis=-1, keepdims=True)
        exp = np.exp(shifted)
        return exp / np.sum(exp, axis=-1, keepdims=True)

    def _full_attention(self, x: Array, layer: LayerWeights) -> tuple[Array, Array, Array]:
        q = self._split_heads(x @ layer.wq)
        k = self._split_heads(x @ layer.wk)
        v = self._split_heads(x @ layer.wv)
        scores = (q @ k.transpose(0, 2, 1)) / np.sqrt(self.head_dim)
        t = x.shape[0]
        causal = np.triu(np.ones((t, t), dtype=bool), k=1)
        scores = np.where(causal[None, :, :], -np.inf, scores)
        weights = self._softmax(scores)
        mixed = weights @ v
        merged = mixed.transpose(1, 0, 2).reshape(t, self.d_model)
        return merged @ layer.wo, k, v

    def _incremental_attention(
        self,
        x: Array,
        layer: LayerWeights,
        cache: dict[str, Array],
    ) -> tuple[Array, dict[str, Array]]:
        # x has one row. No explicit triangular mask is needed because the cache
        # contains only legal past positions plus the current position.
        q = self._split_heads(x @ layer.wq)
        k_new = self._split_heads(x @ layer.wk)
        v_new = self._split_heads(x @ layer.wv)
        k_all = np.concatenate([cache["k"], k_new], axis=1)
        v_all = np.concatenate([cache["v"], v_new], axis=1)
        scores = (q @ k_all.transpose(0, 2, 1)) / np.sqrt(self.head_dim)
        weights = self._softmax(scores)
        mixed = weights @ v_all
        merged = mixed.transpose(1, 0, 2).reshape(1, self.d_model)
        return merged @ layer.wo, {"k": k_all, "v": v_all}

    @staticmethod
    def _block(x: Array, attn: Array, layer: LayerWeights) -> Array:
        x = x + attn
        ff = np.tanh(x @ layer.w1) @ layer.w2
        return x + ff

    def full_forward(self, tokens: list[int]) -> Array:
        positions = np.arange(len(tokens))
        x = self.token_embed[np.asarray(tokens)] + self.position_embed[positions]
        for layer in self.layers:
            attn, _, _ = self._full_attention(x, layer)
            x = self._block(x, attn, layer)
        return x @ self.unembed

    def prefill(self, tokens: list[int]) -> tuple[Array, list[dict[str, Array]]]:
        positions = np.arange(len(tokens))
        x = self.token_embed[np.asarray(tokens)] + self.position_embed[positions]
        caches: list[dict[str, Array]] = []
        for layer in self.layers:
            attn, k, v = self._full_attention(x, layer)
            caches.append({"k": k.copy(), "v": v.copy()})
            x = self._block(x, attn, layer)
        return (x @ self.unembed)[-1], caches

    def decode_one(
        self,
        token: int,
        *,
        position: int,
        caches: list[dict[str, Array]],
    ) -> tuple[Array, list[dict[str, Array]]]:
        x = self.token_embed[[token]] + self.position_embed[[position]]
        updated: list[dict[str, Array]] = []
        for layer, cache in zip(self.layers, caches, strict=True):
            attn, new_cache = self._incremental_attention(x, layer, cache)
            updated.append(new_cache)
            x = self._block(x, attn, layer)
        return (x @ self.unembed)[0], updated


def generate_full(model: TinyDecoder, prompt: list[int], steps: int) -> list[int]:
    seq = prompt.copy()
    logits = model.full_forward(seq)[-1]
    for _ in range(steps):
        token = int(np.argmax(logits))
        seq.append(token)
        logits = model.full_forward(seq)[-1]
    return seq


def generate_cached(
    model: TinyDecoder, prompt: list[int], steps: int
) -> tuple[list[int], float]:
    seq = prompt.copy()
    logits, caches = model.prefill(seq)
    max_delta = float(np.max(np.abs(logits - model.full_forward(seq)[-1])))
    for _ in range(steps):
        token = int(np.argmax(logits))
        seq.append(token)
        logits, caches = model.decode_one(token, position=len(seq) - 1, caches=caches)
        reference = model.full_forward(seq)[-1]
        max_delta = max(max_delta, float(np.max(np.abs(logits - reference))))
    return seq, max_delta


def score_pair_counts(prompt_tokens: int, generated_tokens: int) -> tuple[int, int]:
    """Attention score pairs per layer and query head for two execution plans.

    Full recompute runs a complete causal forward pass before every sampled token.
    Cached decode prefills once, then evaluates one new query row per later token.
    """
    full = sum((prompt_tokens + j) * (prompt_tokens + j + 1) // 2 for j in range(generated_tokens))
    cached = prompt_tokens * (prompt_tokens + 1) // 2
    cached += sum(prompt_tokens + j for j in range(1, generated_tokens))
    return full, cached


def timed_generation(model: TinyDecoder, prompt: list[int], steps: int) -> tuple[float, float]:
    start = perf_counter()
    generate_full(model, prompt, steps)
    full_ms = (perf_counter() - start) * 1000

    start = perf_counter()
    generate_cached(model, prompt, steps)
    cached_ms = (perf_counter() - start) * 1000
    return full_ms, cached_ms


def main() -> None:
    model = TinyDecoder()
    prompt = [2, 5, 1, 8, 3, 13, 21, 1]
    steps = 32

    full_sequence = generate_full(model, prompt, steps)
    cached_sequence, max_delta = generate_cached(model, prompt, steps)

    assert full_sequence == cached_sequence
    assert max_delta < 1e-10

    # Negative case: reuse the exact cache but assign the next token the wrong
    # absolute position. The causal history is intact, yet the logits change.
    next_token = full_sequence[len(prompt)]
    good_prefix = prompt + [next_token]
    good_logits = model.full_forward(good_prefix)[-1]
    _, caches = model.prefill(prompt)
    wrong_logits, _ = model.decode_one(
        next_token,
        position=len(prompt) + 1,  # should be len(prompt)
        caches=copy.deepcopy(caches),
    )
    broken_delta = float(np.max(np.abs(good_logits - wrong_logits)))
    assert broken_delta > 1e-4

    full_pairs, cached_pairs = score_pair_counts(128, 128)
    small_full_ms, small_cached_ms = timed_generation(model, prompt, 32)
    long_full_ms, long_cached_ms = timed_generation(model, prompt * 32, 64)

    print(f"positive: identical generated tokens = {full_sequence == cached_sequence}")
    print(f"positive: max absolute logit delta = {max_delta:.3e}")
    print(f"negative: wrong-position logit delta = {broken_delta:.3e}")
    print(f"score pairs, full recompute = {full_pairs:,}")
    print(f"score pairs, cached decode = {cached_pairs:,}")
    print(f"pair-count reduction = {full_pairs / cached_pairs:.1f}x")
    print(f"small workload, full/cached = {small_full_ms:.1f}/{small_cached_ms:.1f} ms")
    print(f"long workload, full/cached = {long_full_ms:.1f}/{long_cached_ms:.1f} ms")
    print("timings are machine-dependent; parity and pair counts are the assertions")


if __name__ == "__main__":
    main()

A release gate for cache and decoding changes

The decision instrument below separates semantic proof from performance evidence. A team may release an approximate method, but it should do so under an explicit quality envelope rather than an “optimisation” label that hides changed behaviour.

1. Name the intervention. Separate tensor reuse, memory layout, scheduling, proposal-and-verify, quantisation and eviction.
2. Declare the equivalence claim. Exact logits, tolerance-bounded logits, identical greedy tokens or bounded task-level degradation are different contracts.
3. Build cache identity. Include every attribute that can change K, V, mask or position.
4. Run the positive parity case. Compare cached and uncached execution across layers, lengths, batches and prefix hits.
5. Run negative cases. Wrong position, stale adapter, truncated block, changed mask and cross-request handle must be rejected or visibly diverge.
6. Measure phase-specific performance. Report time to first token, token latency, goodput, live bytes and pressure actions.
7. Slice quality where approximation enters. Long-range retrieval, exact copying, code, numbers and task-specific dependencies can fail differently.
8. Test invalidation and recovery. A model update, worker restart or cache-manager failover must not revive incompatible state.

Compact glossary

Causal mask

An attention constraint that blocks a position from reading future positions.

Prefill

The parallel processing of the supplied prompt that creates first-token logits and initial KV state.

Decode

The serial generation phase that extends the sequence, commonly one sampled token per iteration.

KV cache

Per-layer keys and values retained from already processed positions for later queries.

Prefix cache

KV blocks reused across requests that share an eligible token prefix and cache identity.

Semantic parity

Evidence that an execution change preserves the declared logits, tokens, distribution or bounded task behaviour.