TLDR

1. Instruction tuning often changes the model’s default response policy more than its underlying stock of knowledge or procedures.
2. Token-level supervision raises the probability of demonstrated responses, making instruction cues, role markers, formats and task conventions stronger behavioural attractors.
3. Evaluate a tune against both the ordinary base model and the best elicited base model, then test genuinely new compositions and protected behaviours.
4. Reorientation is not harmless: poor mixtures can teach shortcuts, overfit demonstrations, suppress useful alternatives and weaken prior safety behaviour.
5. Use SFT for stable, repeated response contracts; keep changing truth, authority, validation and outcome verification outside the model.

A capable model can still be hard to use

Pretraining teaches a model to predict continuations across a vast mixture of documents, conversations, code, instructions, mistakes and styles. That objective can support many behaviours without assigning any one of them the status of “the answer”. A question might be followed by a direct response, a debate, another question, a quotation, a refusal, a tutorial or an unfinished transcript. The model’s conditional distribution contains this ambiguity.

An application needs a narrower contract. It wants the user’s words treated as a task, the system message treated as higher-priority context, the assistant marker treated as a cue to answer, and a schema treated as a response boundary. Instruction tuning supplies examples in which those interpretations repeatedly predict the target continuation. The model becomes easier to use because the requested behaviour occupies more probability mass under familiar instruction contexts.

“Easier to use” is not a minor cosmetic property. It can turn a model whose capability appears only under hand-crafted prompts into one that works across ordinary language. FLAN, T0 and the later scaling work all emphasised generalisation beyond the exact supervised tasks, with gains depending on task diversity, model scale and training formulation.¹²³ The gain is operationally real even when the underlying computation was latent beforehand.

The theatre-lighting analogy is useful if its correspondences remain explicit. Pretraining constructs the stage, props and possible movements. A prompt aims a temporary spotlight. SFT adjusts the lighting rig so ordinary cues illuminate the useful scene. The analogy fails where parameters themselves encode a new association or procedure. Real fine-tuning changes the stage as well as the lights, which is why a diagnostic rather than a slogan is required.

The Reorientation-Acquisition-Displacement audit

Compare three systems on a fresh, task-level evaluation. Let B be the base model under the ordinary prompt intended for users. Let O be the same base model under the strongest predeclared elicitation procedure: a carefully engineered instruction, allowed few-shot demonstrations, decoding settings and tools that do not update weights. Let T be the instruction-tuned model under the ordinary prompt.

If T mostly closes the gap between B and O, reorientation is the economical explanation. If T exceeds O on held-out structures that cannot be solved by copying the demonstrations, the evidence favours capability strengthening or acquisition. It still does not prove where the information came from. The tune may have encoded narrow facts, combined latent components, or made a previously inaccessible procedure robust.

Displacement requires separate protected slices. For protected behaviour j with consequence weight w_j, define D = Σ w_j max(0, B_j − T_j). This deliberately ignores gains elsewhere. A model that becomes excellent at the target contract while losing refusal behaviour, multilingual competence or calibrated abstention has incurred a cost the aggregate target score conceals.

The oracle baseline is imperfect. Prompt search cannot establish that every latent behaviour has been found, and an extravagant prompt may violate latency or context budgets. Its purpose is causal discrimination, not metaphysical proof. Report the elicitation budget, freeze it before inspecting tuned results, and evaluate all three systems on the same fresh cases. Otherwise the oracle becomes a moving excuse for whichever story one prefers.

What the supervised loss actually pushes

Instruction tuning is usually implemented as supervised fine-tuning on prompt-response sequences. The model still performs next-token prediction. The difference is the distribution of sequences and, often, which token positions contribute to the loss. Official training documentation describes SFT as minimising the negative log-likelihood of target responses conditioned on their inputs.¹³

θ

The model parameters being updated.

N

The number of supervised conversations or prompt-completion examples.

c_i

The conditioning context: system text, roles, user request, retrieved material and control tokens.

y_i,t

The demonstrated response token at position t.

m_i,t

A mask selecting positions that contribute to the loss, commonly the assistant completion rather than padding and sometimes rather than the prompt.

When the model assigns low probability to a demonstrated token, the gradient pushes the parameters towards making that token more probable in similar contexts. Repeated across a sequence, it favours the whole response trajectory: opening directly, adopting the requested role, selecting a procedure, using a schema, expressing uncertainty in a particular way and ending at the expected boundary. When a term changes, the direction of pressure changes. More weight on rare refusal examples strengthens those continuations locally. More JSON examples strengthen schema tokens and the cues that predict them. A longer response contributes more token-level terms unless weighting compensates.

SFT does not contain a special “follow instructions” operator. It creates that macroscopic behaviour by altering millions or billions of conditional probabilities so that instruction-shaped contexts lead towards demonstrated continuations. The resulting generalisation depends on pretraining, representation reuse, shared task structure, data diversity and the geometry of the update.

Why a small tune can create a large visible change

The update can be small relative to pretraining while its visible effect is large. Imagine that several continuations are already plausible after a request: a direct answer has probability 0.26, a discursive explanation 0.29, a quotation 0.23 and a clarifying question 0.22. The model has no need to invent direct answering from scratch. A modest redistribution that moves the direct answer to 0.38 can make it the decoded default across many similar prompts.

This is a ranking effect. Generation repeatedly selects among close alternatives, so a local probability change at the opening tokens can steer the entire continuation into another region. Once the model emits an opening brace, later JSON tokens become more likely. Once it begins “I cannot help with that”, a refusal trajectory becomes easier to continue. A small parameter movement can therefore cross a behavioural decision boundary and produce a discontinuous change in what users see.

The railway-switch analogy captures this amplification. Pretraining lays several tracks; the tune moves a switch before the train reaches the junction. The analogy fails when supervision teaches a genuinely new track. A compact factual association, a specialised vocabulary item or a short algorithm may be learned during SFT, especially when repeated and representable by the model. That possibility is why the article says “usually” and why the residual test uses fresh structural cases rather than assuming all gains were latent.

Instruction tuning is one stage, not all of post-training

Supervised demonstrations specify the response that should follow a context. Preference learning compares candidates under an elicited judgement. InstructGPT, for example, first fine-tuned on labeler demonstrations and then used ranked outputs for reinforcement learning from human feedback.⁴ Collapsing those stages hides their distinct evidence and failure modes.

SFT is strongest when an expert can write the desired output directly. It is awkward when many outputs are acceptable but their relative quality matters, when a long procedure cannot be demonstrated economically, or when success is observable only after action. Preference optimisation, search and reinforcement learning can address some of those conditions, though each introduces new measurement problems. The mechanism decision should follow the available supervision, not the fashion of the training method.

Roles and chat templates are part of the learned interface

A chat API presents messages as structured dictionaries, but a causal language model ultimately receives a token sequence. Chat templates insert role markers, separators and end-of-message tokens. Different fine-tunes derived from the same base model can expect different templates, and using the wrong control tokens can sharply reduce performance.¹⁴ The template is therefore not neutral plumbing. It is part of the conditioning language on which the response policy was trained.

This matters when teams compare models or move a checkpoint between serving stacks. A purported capability regression may be a formatting mismatch. Conversely, a model may appear to obey system and user roles only because a particular template and data mixture made those tokens predictive. Statistical role sensitivity is useful. It is not a security boundary and does not itself confer authority.

From local token pressure to broad instruction behaviour

How can a token-level objective generalise to unseen instructions? The demonstrations share latent structure. Words such as “classify”, “extract” and “explain” predict different transformations. Role markers distinguish request from response. Formats share brackets, field names and ordering conventions. Many tasks reuse representations learned during pretraining. Updating shared features can therefore affect a region of prompts rather than one memorised sentence.

A NAACL study comparing pretrained and instruction-tuned models reported three model-specific internal shifts: stronger recognition of instruction portions, attention patterns involving instruction verbs, and feed-forward representations oriented towards user tasks.⁷ Those interpretability methods do not prove a universal mechanism. They do provide convergent evidence for the behavioural account: the tuned model conditions its response more strongly on the parts of the input that specify what the user wants.

A minimal worked example

Consider a synthetic model that already distinguishes answer A from answer B with perfect accuracy. It is weakly sensitive to a request for JSON, so it defaults to prose. On a balanced test of answer and format combinations, joint answer-plus-format accuracy is 50 per cent while semantic answer accuracy is 100 per cent. Multiplying the format cue through an “oracle” prompt raises joint accuracy to 100 per cent without changing weights.

Train the same model on diverse examples crossing both answers, both formats and an irrelevant marker. SFT also reaches 100 per cent under the ordinary request and preserves semantic accuracy. In the RAD audit, B = 0.50, O = 1.00 and T = 1.00. The default gain is 0.50, the elicitable span is 0.50 and the residual is zero. The simplest supported conclusion is behavioural reorientation.

This toy model does not resemble a transformer in scale or representation. It isolates the causal logic that a large-model evaluation should retain: capability, elicitation and default behaviour are different variables. The executable version appears later and includes a deliberately confounded negative case.

flowchart LR
  A["A capable model can still be<br/>hard to use"] --> B["What the supervised loss<br/>actually pushes"]
  B --> C["The dataset is the behavioural<br/>specification"]
  C --> D["Where behavioural<br/>reorientation stops helping"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#007D68,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

The dataset is the behavioural specification

The supervised objective rewards whatever the examples make predictable. A dataset full of direct, evidenced, concise answers teaches one response regime. A dataset dominated by verbose teacher outputs teaches another. If every refusal includes a distinctive phrase, the model may learn the phrase as a shortcut. If all JSON examples concern one topic, topic and format become entangled. Data collection is therefore closer to behavioural requirements engineering than to filling a generic bucket marked “quality”.

Count examples only after describing the variations they cover. Useful dimensions include task semantics, instruction phrasing, input length, response structure, language, ambiguity, evidence sufficiency, risk level, refusal reason, multi-turn state and adversarially irrelevant cues. Repeating one region reduces training loss; crossing independent dimensions reveals which relation the model must learn.

Diversity changes what can generalise

Instruction-tuning research repeatedly finds value in task or instruction diversity. The Only-IF study uses controlled string-rewrite tasks and broader experiments to examine how semantic diversity supports generalisation to unseen instructions.¹⁵ The exact optimum depends on whether the goal is a specialist or a generalist. A specialist still needs enough counterfactual variation to separate the intended rule from accidental surface cues.

The relevant denominator is not merely the number of prompts. Suppose 20,000 examples all ask for summaries with the same wording. They provide many response tokens but little evidence that paraphrased requests, contradictory evidence or a request for extraction should be handled correctly. A smaller set that varies intent, content, format and uncertainty independently can constrain the desired function more sharply.

Teacher quality must fit the student

“Use the strongest model to generate answers” sounds sensible, but a response can be excellent for a human and still be an awkward learning target for a smaller base model. It may use reasoning patterns, vocabulary or formatting far outside the student’s distribution. Recent work on GRAPE selects, among candidate high-quality responses, those that better fit the target base model’s conditional distribution and reports gains over untailored selection in its experiments.¹⁶ This is evidence for compatibility as one data-quality dimension, not a licence to prefer easy or mediocre answers.

Data mixture also matters. Experiments spanning mathematics, code and general alignment tasks found different scaling patterns and performance conflicts as composition and volume changed.⁸ A public post-training recipe such as Tülu 3 accordingly treats dataset curation, decontamination and multi-task evaluation as first-class components rather than incidental preprocessing.¹⁰

The practical implication is to version an SFT dataset as a behavioural release. Record each example’s intended behaviour, source, licence, generator, reviewer, risk class, template, language, deduplication lineage and evaluation overlap. Maintain explicit protected behaviours and known contradictions. A single global “quality score” cannot express that an example is fluent, factually sound, unsafe for one context and useful for another.

Sampling and weighting quietly set priorities

Even a well-labelled dataset becomes a different behavioural specification after sampling. Uniform sampling over rows may let a duplicated source dominate. Sampling by dataset can over-weight a tiny specialist collection. Token-level loss can give long answers more opportunities to influence the gradient than short answers. Packing, truncation and completion masking determine which parts of a conversation are actually learned.

Make these choices inspectable. Report source-level sampling weights, effective token counts, duplicate clusters, length distributions and the proportion of loss-bearing tokens by behaviour. For rare but consequential cases, add targeted evaluation before increasing their training weight. Oversampling can teach the desired response, but it can also make the model infer that the rare condition is common and trigger it too broadly.

Recent peer-reviewed work offers one example of treating training dynamics as part of data design. SFTMix identifies examples with different confidence profiles and applies a Mixup-based regulariser; its authors report consistent improvements across the studied model families, instruction datasets and healthcare-specific tasks.¹⁷ This does not establish Mixup as a universal recipe. It reinforces the narrower point that row count alone does not describe the learning pressure.

The decisive negative control

Before paying the cost and accepting the governance burden of a tune, run the same demonstrations as an inference-time baseline. Permit the base model the number of examples that the serving budget can realistically support. Freeze the prompt and decoding protocol. Then compare ordinary base, elicited base and tuned model on untouched evaluation items.

If a few-shot prompt using the same examples matches the tune, the experiment has shown amortised elicitation, latency reduction or prompt simplification rather than new competence. Those are valid reasons to tune. They should be named correctly because they predict different failure modes. Amortised elicitation may remain brittle to template changes. New competence should survive fresh compositions that the prompt baseline cannot solve.

Add a second negative control: scramble or invert the feature believed to cause the behaviour while preserving superficial cues. In the toy lab, a confounded training set always pairs answer A with JSON and answer B with prose, while an irrelevant marker also tracks format. The tuned classifier scores 100 per cent on its training pattern. On counterfactual pairs, where the requested format changes but the marker does not, joint accuracy falls to zero. The model learned a shortcut that ordinary validation would celebrate.

A release protocol that can falsify the story

Split evaluation by hypothesis rather than by one aggregate benchmark. An exact-instruction slice tests memorisation and template fit. A paraphrase slice tests surface robustness. A compositional slice crosses familiar primitives in unseen ways. A capability slice removes formatting credit and scores the substantive operation. A protected slice checks behaviours that should not move. A safety slice varies harmful intent independently of vocabulary. A current-fact slice supplies evidence at runtime and changes it after training.

For every reported gain, state which alternative explanation remains. Stronger formatting could explain a score based on exact match. Longer answers could impress a model judge. Contamination could explain an apparent new skill. Prompt-template mismatch could depress the base baseline. The RAD audit becomes useful when its residual is accompanied by tests designed to destroy these alternatives.

Where behavioural reorientation stops helping

Reorientation is powerful when the base model already supports the required transformation and the desired behaviour is stable enough to encode. It weakens as the problem shifts towards changing world state, exact computation, enforceable authority, rare edge cases or outcomes that must be verified outside language. The correct architecture starts by locating that boundary.

Style is cheaper to imitate than capability

Teacher-generated data can make a smaller model sound like a stronger one. The imitation study by Gudibande and colleagues found that such models appeared much better at following instructions to human raters while the capability gap remained substantial on tasks where the base model lacked support.⁶ Fluency, structure and confidence are highly visible. Internal competence is revealed only by discriminating tasks and counterfactuals.

This creates a dangerous evaluation asymmetry. A format or style tune can improve the surface features used by preference judges, including length, headings and confident explanation. If the metric rewards those same features, training and evaluation share a shortcut. Use answer-only scoring where possible, independent calculators for exact quantities, executable tests for code and source-level checks for factual claims.

Apparently lost capability may be suppressed, or genuinely damaged

Fine-tuning can reduce performance outside its distribution. Work on implicit task inference shows one mechanism: the tune can bias the model towards interpreting prompts as tasks seen during fine-tuning, and alternative prompting can recover some apparently forgotten capability in a simplified setting.⁹ This is behavioural displacement through task selection. Other updates may alter representations or procedures more directly. Recovery by prompting distinguishes some suppression from irreversible loss, but absence of recovery is not proof of destruction.

Protected-behaviour evaluation must accompany every target gain. Preserve the base checkpoint, run multilingual and domain-general probes, compare calibrated abstention, and test safety policies under paraphrase and adversarial context. Aggregate scores cannot tell whether one valuable route was paved by closing another.

Following instructions is not the same as respecting authority

Safety can also regress under custom fine-tuning. Red-team experiments have shown that a small adversarial set can sharply weaken safeguards, while benign datasets can cause smaller unintended degradation.¹² The exact vulnerability depends on model and tuning access, but the architectural lesson is general: controls that must hold under a hostile or damaged model should live outside the model.

A production-shaped worked scenario

Consider a synthetic policy-evidence assistant for a regulated case workflow. Its task is to read a case packet and produce a typed recommendation containing the decision question, applicable policy evidence, calculations, uncertainty, unresolved contradictions and a proposed next action. No client or deployment is implied. The scenario exposes where an SFT model belongs inside a larger decision system.

Intent comes from the user’s request and case objective. Identity binds the authenticated actor and subject. World state contains the current case values and effective policy version. A context compiler selects entitled evidence, records provenance and fits it to the model budget. The tuned model performs reasoning as a proposal: it organises the evidence into a stable contract, identifies gaps and suggests an action.

Authority remains deterministic. A typed action contract passes to a policy decision that checks permission, purpose, amount limits, segregation of duties and freshness. The action is executed only after those checks. An evidence ledger records the sources and decision receipt. The outcome is established through an effect receipt and independent readback. Release and recovery handle timeout, unknown outcome, duplicate prevention and escalation.

What should be tuned? Stable behaviours that recur across cases: recognise the decision question, separate claims from evidence, cite evidence identifiers, express uncertainty, distinguish missing from conflicting information, produce the typed schema and abstain when the evidence contract is incomplete. What should not be stored as behavioural memory? Current thresholds, customer state, permissions, outstanding balances, legal effective dates and whether an action actually succeeded.

Treat the tune as a behavioural release

A tuned checkpoint should enter service through the same discipline as a consequential software change. Preserve the base and previous tuned versions. Run slice-level regression tests, canary the new model on bounded traffic, attach the dataset and template lineage to the release, and define rollback triggers before exposure. Since stochastic outputs vary, rollback criteria should use distributions and failure rates rather than exact string equality.

Monitor the contract the tune was meant to change: schema validity, instruction adherence, evidence use, abstention and protected behaviours. Also monitor second-order effects such as response length, tool-call rate and escalation volume. A tune that improves a benchmark while doubling unnecessary actions has changed the operating system, not merely the model score.

Choose the mechanism that changes the right variable

Teams often reach for SFT because it feels like the most model-native intervention. A better question is which variable must change. If the model can perform the task under a stable prompt, prompt engineering may suffice. If truth changes, retrieve it. If exactness matters, calculate or validate. If the desired answer is comparative rather than demonstrable, preference learning may fit. If success can be scored after exploration, reinforcement learning may supply a more appropriate signal. If an action must be authorised, use a deterministic control regardless of training method.

stateDiagram-v2
  [*] --> Claim
  state "The four doors" as Claim
  state "The…" as Test
  state "Why a small tune can create a<br/>large visible change" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Executable artefact: a causal SFT laboratory

The following NumPy programme implements the minimal example. It uses a four-class softmax model whose outputs combine a semantic answer (A or B) and a format (JSON or prose). The base weights encode the semantic distinction strongly, prefer prose by default and respond weakly to the format instruction.

The positive training set crosses semantic answer, requested format and an irrelevant marker. The negative set confounds them. The programme is deterministic, uses synthetic data, and contains both positive and negative cases. Its purpose is to expose evaluation logic, not to approximate transformer learning dynamics.

Base ordinary        joint=50%  semantic=100%  format=50%
Base oracle cue      joint=100%  semantic=100%  format=100%
Diverse SFT          joint=100%  semantic=100%  format=100%
RAD residual         T-O=0 percentage points
Confounded train     joint=100%  semantic=100%  format=100%
Confounded counter   joint=0%  semantic=100%  format=0%
Protected displacement in semantic accuracy: 0 percentage points

from __future__ import annotations

from dataclasses import dataclass
from typing import Iterable
import numpy as np

# Each output class is (semantic answer, response format).
# +1 means A or JSON; -1 means B or prose.
CLASSES = [(1, 1), (1, -1), (-1, 1), (-1, -1)]


@dataclass(frozen=True)
class Case:
    answer: int
    requested_format: int
    marker: float = 0.0


def class_index(answer: int, requested_format: int) -> int:
    return CLASSES.index((answer, requested_format))


def features(case: Case, oracle: bool = False) -> np.ndarray:
    # The oracle prompt makes the genuine format cue easier to read.
    format_strength = 8.0 if oracle else 1.0
    return np.array([
        1.0,
        float(case.answer),
        format_strength * float(case.requested_format),
        float(case.marker),
    ])


def initial_weights() -> np.ndarray:
    weights = np.zeros((4, 4))
    for row, (class_answer, class_format) in enumerate(CLASSES):
        weights[row, 1] = 2.2 * class_answer       # strong semantic ability
        weights[row, 2] = 0.08 * class_format      # weak instruction cue
        weights[row, 0] = 0.55 if class_format == -1 else 0.0  # prose default
    return weights


def softmax(logits: np.ndarray) -> np.ndarray:
    shifted = logits - logits.max(axis=1, keepdims=True)
    exp = np.exp(shifted)
    return exp / exp.sum(axis=1, keepdims=True)


def train(
    weights: np.ndarray,
    cases: Iterable[Case],
    *,
    learning_rate: float = 0.08,
    steps: int = 400,
    l2: float = 0.002,
) -> np.ndarray:
    examples = list(cases)
    x = np.stack([features(case) for case in examples])
    y = np.array([class_index(case.answer, case.requested_format)
                  for case in examples])
    result = weights.copy()

    for _ in range(steps):
        probabilities = softmax(x @ result.T)
        targets = np.eye(len(CLASSES))[y]
        gradient = ((probabilities - targets).T @ x) / len(y) + l2 * result
        result -= learning_rate * gradient
    return result


def evaluate(
    weights: np.ndarray,
    cases: Iterable[Case],
    *,
    oracle: bool = False,
) -> tuple[float, float, float]:
    examples = list(cases)
    x = np.stack([features(case, oracle=oracle) for case in examples])
    predicted = np.argmax(x @ weights.T, axis=1)
    expected = np.array([class_index(case.answer, case.requested_format)
                         for case in examples])

    joint = float(np.mean(predicted == expected))
    semantic = float(np.mean([
        CLASSES[prediction][0] == case.answer
        for prediction, case in zip(predicted, examples)
    ]))
    output_format = float(np.mean([
        CLASSES[prediction][1] == case.requested_format
        for prediction, case in zip(predicted, examples)
    ]))
    return joint, semantic, output_format


def report(label: str, metrics: tuple[float, float, float]) -> None:
    joint, semantic, output_format = metrics
    print(
        f"{label:<20} joint={joint:.0%}  "
        f"semantic={semantic:.0%}  format={output_format:.0%}"
    )


def main() -> None:
    base = initial_weights()

    # Balanced evaluation: every answer-format combination, with the marker
    # varied independently. This prevents the marker from predicting format.
    balanced = [
        Case(answer, output_format, marker)
        for answer in (1, -1)
        for output_format in (1, -1)
        for marker in (-3.0, 3.0)
    ]

    # Positive SFT: cross all causal factors and balance the irrelevant marker.
    diverse_training = balanced * 8
    diverse_tuned = train(base, diverse_training)

    # Negative SFT: marker, answer and format are perfectly confounded.
    confounded_training = (
        [Case(1, 1, 3.0), Case(-1, -1, -3.0)] * 32
    )
    confounded_tuned = train(base, confounded_training)

    # Counterfactual: keep the familiar answer-marker relation but reverse
    # the requested format. A shortcut learner should fail here.
    counterfactual = (
        [Case(1, -1, 3.0), Case(-1, 1, -3.0)] * 16
    )

    base_standard = evaluate(base, balanced)
    base_oracle = evaluate(base, balanced, oracle=True)
    tuned_standard = evaluate(diverse_tuned, balanced)

    report("Base ordinary", base_standard)
    report("Base oracle cue", base_oracle)
    report("Diverse SFT", tuned_standard)
    residual = tuned_standard[0] - base_oracle[0]
    print(f"RAD residual         T-O={100 * residual:.0f} percentage points")
    report("Confounded train", evaluate(confounded_tuned, confounded_training))
    report("Confounded counter", evaluate(confounded_tuned, counterfactual))
    displacement = base_standard[1] - tuned_standard[1]
    print(
        "Protected displacement in semantic accuracy: "
        f"{100 * displacement:.0f} percentage points"
    )


if __name__ == "__main__":
    main()

Compact glossary

Instruction tuning

Supervised adaptation on examples phrased as instructions, conversations or prompt-completion pairs, intended to improve task following and response behaviour.

Behavioural reorientation

A change in which supported behaviour is selected by default under ordinary requests.

Capability acquisition

A durable improvement on a substantive operation that exceeds the strongest frozen elicitation baseline and survives fresh structural variation.

Behavioural displacement

A regression in a protected behaviour caused or revealed by the update.

Oracle elicitation

A strong, predeclared inference-time protocol used to estimate what the unchanged base model can already express.

Shortcut

A predictive feature that succeeds in the training distribution without representing the intended causal relation.

References and source notes

1. Wei et al., “Finetuned Language Models Are Zero-Shot Learners”. Primary research. Paper and versions.
2. Sanh et al., “Multitask Prompted Training Enables Zero-Shot Task Generalization”. Primary research. Paper and versions.
3. Chung et al., “Scaling Instruction-Finetuned Language Models”. Primary research. Paper and versions.
4. Ouyang et al., “Training language models to follow instructions with human feedback”. Primary research separating supervised demonstrations from later preference optimisation. Paper.
5. Zhou et al., “LIMA: Less Is More for Alignment”. Primary research. Paper.
6. Gudibande et al., “The False Promise of Imitating Proprietary LLMs”. Negative and limitation evidence. Paper.
7. Wu et al., “From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning”. Peer-reviewed interpretability study. ACL Anthology record.
8. Dong et al., “How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition”. Peer-reviewed mixture study. ACL Anthology record.
9. Kotha, Springer and Raghunathan, “Understanding Catastrophic Forgetting in Language Models via Implicit Inference”. Primary research. Paper and versions.
10. Lambert et al., “Tülu 3: Pushing Frontiers in Open Language Model Post-Training”. Open post-training recipe and evaluation suite. Paper and versions.
11. Chu et al., “SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training”. Primary, task-bounded comparison. Paper and versions.
12. Qi et al., “Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!”. Security and safety limitation evidence. Paper.
13. Hugging Face TRL, “SFT Trainer”. Official implementation documentation for data formats and SFT loss. Documentation.
14. Hugging Face Transformers, “Chat templates”. Official documentation for role formatting and control tokens. Documentation.
15. Zhang, Wang and Charton, “Only-IF: Revealing the Decisive Effect of Instruction Diversity on Generalization”. Primary research. Paper and versions.
16. Zhang et al., “The Best Instruction-Tuning Data are Those That Fit”. Recent primary research on target-model-compatible response selection. Paper and versions.
17. Xiao et al., “SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe”. Peer-reviewed 2026 work on confidence-varying examples and regularisation. ACL Anthology record.