A ten-question example

Real corpora make the boundary harder. Duplicate and near-duplicate material can travel through mirrors, derived datasets and synthetic transformations. Analysis of C4 found examples from evaluation datasets inside a large web corpus.[4] Deduplication research found substantial repetition, reduced memorised emissions and reduced train-test overlap when duplicates were removed.[5] Controlled contamination studies further distinguish exposure to input text from exposure to the ground-truth answer, and show that simple n-gram definitions miss important cases.[6]

Provenance has more than one layer

Dataset documentation proposals such as Datasheets for Datasets and model documentation proposals such as Model Cards were designed to make motivation, composition, intended use and evaluation conditions visible.[7][8] The Data Provenance Initiative later audited more than 1,800 text datasets and found extensive missing or erroneous licensing metadata, illustrating how difficult lineage reconstruction can be at scale.[9]

For evaluation validity, however, provenance must go beyond legal source and licence. It needs at least five layers: source documents, transformation and filtering, training mixture and order, post-training datasets and objectives, and known or tested contact with benchmark materials. A model identifier without these layers is like a medicine label that names the bottle but omits the dose and manufacturing batch.

The benchmark has a lifecycle

Evaluation datasets are often public because openness supports inspection and reproducibility. That same openness creates time-dependent exposure. Source passages may predate the benchmark. Draft items may circulate in repositories. Evaluation prompts may later enter logs used for training. Post-training datasets may adapt benchmark formats even when exact items are absent.

A useful provenance record therefore asks when each material became available and when each model stage ended. A model trained before item creation has a stronger independence case than one trained after public release. A private test is stronger still only if its source material, annotators, generation process and access controls also resist indirect exposure.

Exposure is a distance, not a switch

“Seen” and “unseen” are too coarse for serious inference. Exact exposure means the item and answer were available together. Near-duplicate exposure preserves most wording or structure. Source exposure makes the underlying passage, table or code available without the authored question. Procedure exposure supplies worked examples that teach the same transformation while changing all surface content. These distances create different rival explanations.

Exact answer exposure makes direct reproduction plausible. Source exposure may merely remove a knowledge-retrieval burden while leaving the requested inference novel. Procedure exposure can teach a genuinely reusable skill, yet still make a benchmark a poor estimate of unaided transfer. For that reason, a contamination report should not collapse every overlap into one percentage. It should name the unit of contact, the training stage, whether answers were present, and the transformation separating training from test.

String-level novelty is only one kind of independence. When the intended claim concerns transfer, the decisive comparison is often a source-swapped or post-cutoff task that preserves the causal demand while replacing the materials that may have been learned.

One question, three instruments

Multiple-choice evaluation makes the interface problem concrete. A system can be asked to emit a letter, to generate the full answer text, or to rank option likelihoods. Those methods are not interchangeable for an instruction-tuned model.

Few-shot example order has produced large swings in earlier language-model evaluations.[10] Option order can also alter multiple-choice performance, with one study reporting sizeable gaps across models and benchmarks.[11] Yet a later analysis found that some apparent prompt sensitivity came from rigid matching and first-token likelihood scoring rather than from the model’s semantic answer.[12] Another study similarly showed that generated text can be more robust than first-token selection when the two disagree.[13]

These results do not cancel one another. They separate two mechanisms. A model may genuinely depend on superficial order. An evaluator may also mis-score a stable answer. Interface sensitivity and scoring sensitivity are different failure modes.

Construct validity comes before leaderboard validity

A benchmark name often does more rhetorical work than its items. “Reasoning”, “truthfulness”, “agency” or “medical competence” may each cover several distinct capacities. A multiple-choice item can require knowledge, language parsing, option comparison, calibration, refusal behaviour and format compliance. A single accuracy score cannot reveal which component failed.

A large systematic review of 445 language-model benchmarks found recurring gaps in construct definition, format control, contamination preparation, uncertainty reporting and error analysis.[14] HELM takes a complementary approach by evaluating multiple metrics in shared scenarios rather than treating accuracy as the only desideratum.[15]

Validity belongs to an interpretation and use, not to a dataset name. “Model α scored 82 under configuration K” is a direct observation. “Model α can reason” is an inference. The second sentence requires a bridge: task coverage, rival controls, stable performance under equivalent forms, and evidence that the bridge matters in the intended setting.

A small factorial experiment

Suppose we can train two small checkpoints with the same architecture and recipe. The clean checkpoint excludes a benchmark family. The exposed checkpoint receives paraphrases of part of that family during post-training. We then cross two prompt forms with two scorers.

Use a balanced set of two hundred items, randomise item and option order, hold decoding fixed and retain every generated answer. Estimate the exposure effect, prompt effect, scorer effect and their interactions, with uncertainty from item-level resampling. The design assumes that the checkpoints differ only in the planned post-training exposure and that human reviewers judge the paraphrases to preserve task demand and difficulty closely enough for comparison.

A large exposure-by-prompt interaction would permit the statement that the canonical advantage depends partly on benchmark form. A small interaction with stable post-cutoff performance would weaken that explanation. A null result would not prove absence of contamination, especially with low power or ineffective transformations. It would show only that this intervention failed to reveal a consequential dependency.

A realistic configured-system example

How to use a positive or negative result

A positive audit does not prove an intrinsic, context-free capability. It permits a stronger statement within the tested envelope. For example: “Model α shows stable routing performance across six instructions, two scorers and a post-cutoff domain shift.” That sentence is less dramatic than “Model α understands customer intent”, but it is actionable.

A negative audit is not a verdict that the model lacks the capability. It identifies the next discriminating test. Unknown exposure calls for a time-split or provider-side overlap report. Prompt instability calls for equivalent interfaces and error review. Weak construct coverage calls for task decomposition. Poor transport calls for shadow evaluation or a new domain slice.

The instrument cannot establish consciousness, understanding in a metaphysical sense, or a unique internal mechanism. It is a method for bounding behavioural and operational claims. It should not be used to turn missing disclosure into a numerical accusation.

from dataclasses import dataclass, asdict
from typing import Literal

Claim = Literal[
    "observation only",
    "reproducible configured performance",
    "stable benchmark-task performance",
    "construct-relevant capability evidence",
    "deployment-relevant capability evidence",
]

@dataclass(frozen=True)
class Evidence:
    construct: int      # 0..3
    exposure: int       # 0..3
    interface: int      # 0..3
    transport: int      # 0..3
    receipts: int       # 0..3
    uncertainty: int    # 0..3


def permitted_claim(e: Evidence) -> Claim:
    if e.receipts == 0:
        return "observation only"
    if min(e.receipts, e.uncertainty) < 1:
        return "reproducible configured performance"
    if min(e.exposure, e.interface, e.receipts, e.uncertainty) < 2:
        return "reproducible configured performance"
    if e.construct < 2:
        return "stable benchmark-task performance"
    if e.transport < 2:
        return "construct-relevant capability evidence"
    return "deployment-relevant capability evidence"


if __name__ == "__main__":
    evidence = Evidence(construct=2, exposure=3, interface=2,
                        transport=1, receipts=2, uncertainty=2)
    print({**asdict(evidence), "permitted_claim": permitted_claim(evidence)})

Three boundaries that prevent overcorrection

First, unknown provenance is not proof of invalidity. It is missing information. The permitted statement becomes narrower: a reproducible score under a named configuration, with exposure uncertainty disclosed. This distinction matters because accusation and scientific caution are not the same act.

Second, a hidden benchmark is not automatically valid. Hidden items can still misrepresent the construct, contain annotation errors, share public source material or reward brittle shortcuts. Secrecy protects one exposure path. It does not validate the task.

Third, an LLM judge is not a neutral oracle. Semantic grading can reduce exact-match artefacts, as some prompt-sensitivity research suggests, but the judge introduces its own prompt, model version, preferences and failure modes. Human adjudication of disagreements, rubric validation and multiple scoring paths remain necessary when the result matters.

What the envelope cannot buy

No practical audit can recover a fully opaque training history from outputs alone. Black-box contamination tests depend on assumptions about memorisation, probability access and model behaviour. Provider-side reports can also be incomplete or use incomparable overlap definitions. The best available conclusion may remain conditional.

Nor can controlled behavioural variation uniquely identify an internal algorithm. Stable performance across paraphrases supports robustness, not a specific reasoning circuit. A collapse under format change reveals dependence, but not whether the cause is tokenisation, attention allocation, post-training preference or parser behaviour. Mechanism claims require separate causal interventions inside the model.

The envelope is intentionally conservative. Its purpose is not to make every evaluation expensive. It helps match expense to consequence. A low-stakes prototype may need a reproducible configured score and error sample. A clinical, financial or safety claim may justify private time splits, overlap reporting, multi-interface testing, uncertainty and operating evidence.