What has actually drifted?

Classical concept drift concerns a changing relation between input data and a target over time. Dataset-shift methods ask whether a current sample differs from a reference sample, often through two-sample testing or a domain discriminator. Those are important instruments, especially when a predictive model depends on training-time assumptions.[1][2] They do not, by themselves, identify the object that a compound decision system must keep stable.

Consider a decision service at time t as a mapping:

A decision system has materially drifted when this mapping changes outside its validated envelope for a consequential slice. That definition does not require the entire system to remain numerically identical. A seasonal increase in applications may be expected. A planned policy change may deliberately alter decisions. A tool upgrade may be safe if its semantics, tests and readback remain equivalent. Material drift begins when an assumption, conditional behaviour or control changes without the corresponding evidence, authorisation or revalidation.

Drift, change and failure are not synonyms

An authorised policy revision can intentionally move the review rate. That is change, and the sentinel should confirm that the observed movement agrees with the policy impact tested before release. It becomes drift when the movement appears in an unapproved slice, exceeds the validated envelope, starts before the effective time, continues after withdrawal, or cannot be reconstructed from the change record. A drift signal becomes a failure only when it breaks a required property or creates unacceptable consequence.

This separation changes alert language. “Review rate increased” is an observation. “Population mix explains the increase” is a decomposition. “Policy version 12 caused the within-cohort increase” is a causal diagnosis that requires replay or another intervention. “Customers were wrongly delayed” is an outcome claim that requires case evidence. A useful operating record preserves these levels rather than allowing an early statistical alert to harden into an incident narrative.

The validated envelope should therefore contain more than upper and lower metric bounds. It should state permitted policy versions, tool-contract ranges, required workflow invariants, authorised decision rights, expected case slices and known seasonal regimes. Stability means conformance to an explicit envelope, not immobility. A system may adapt safely inside that envelope and fail while every aggregate chart appears stationary.

Thought experiment one: freeze the model

Clone the entire system, including model weights and prompts. In the clone, replace only the policy table with an obsolete version. Give both systems the same cases and evidence. If their decisions differ, the causal change is real even though every model statistic is identical. A model monitor will correctly report no model drift and still fail to protect the decision.

Thought experiment two: freeze conditional behaviour

Now keep every component fixed, but send twice as many high-risk cases. The aggregate review rate rises because the population mixture changed, not because the system treats any case differently. A monitor that equates an output-rate change with behavioural drift will trigger the wrong response, perhaps rolling back a healthy release during a genuine demand shift.

The radar makes a disciplined point: several sensors can describe one decision, but they do not become interchangeable. Evidence freshness is a time-and-provenance property. Population shift is a distribution property. Policy conformance is a version-and-applicability property. Workflow drift concerns traces and transitions. Model behaviour concerns conditional outputs. Outcome drift may arrive much later. Do not compress unlike signals until each has first been interpreted in its own unit.

Decompose movement before diagnosis

Suppose the monitored decision is referral to human review. Let c index a case cohort, such as standard or vulnerable customers. Let p_t(c) be the share of current cases in cohort c, and let q_t(c) be the current review probability within that cohort. The aggregate review rate is:

Minimal worked example

In the reference period, 80 per cent of cases are standard and 20 per cent are vulnerable. The system reviews 10 per cent of standard cases and 30 per cent of vulnerable cases, producing an aggregate review rate of 14 per cent. In the current period, the within-cohort rates remain exactly 10 and 30 per cent, but the population becomes 50 per cent standard and 50 per cent vulnerable. The aggregate rises to 20 per cent.

A raw monitor sees a six-point increase. A directly standardised comparison holds the reference mixture fixed and asks what the current conditional rates would produce under that mixture. It returns 14 per cent. The entire raw movement is population composition, not changed treatment within either cohort. That is a negative control for behavioural drift: the correct action may be to update capacity planning, not roll back the decision service.

The seven signals answer different questions

Distribution shift detection can tell us that two samples differ, and uncertainty under dataset shift can reveal when confidence becomes unreliable.[2][3] Neither result establishes that the model caused the operational change. ML systems accumulate risk through data dependencies, feedback loops, interfaces and system-level interactions, which is why a decision-centred monitor must reach beyond model weights.[4] Process-mining work similarly treats control flow and activity relationships as changing objects that can be detected and localised from event data.[5]

Time is part of the evidence

Drift analysis is easily corrupted by a single timestamp. A source document has an observation time and may describe a fact effective at another time. A policy has publication, approval, effective and withdrawal times. A tool contract has deployment time, while a cached workflow may begin using it later. The decision and its external effect have separate timestamps, and the outcome may mature weeks afterwards.

A “current policy” field overwritten in place destroys the ability to ask what was applicable when a decision occurred. So does attaching today's customer record to yesterday's case. The event ledger should preserve both transaction time, when the system learned or recorded something, and valid time, when it was true or applicable. A policy sentinel can then test applied_version = expected_version(case, decision_time); an evidence sentinel can test whether a source was fresh and authorised at that same moment.

This temporal discipline prevents a subtle false diagnosis. Replaying an old case against today's data can make a correct historical decision look defective, while replaying it against an old policy but a newly corrected document can make a real defect disappear. A defensible replay reconstructs the knowable and applicable state, not merely the latest state.

The monitoring sequence is therefore movement, decomposition, localisation, consequence and response. Skipping decomposition confuses demand with behaviour. Skipping localisation encourages blunt rollback. Skipping consequence turns every statistically detectable deviation into an incident. Skipping response leaves a mathematically elegant dashboard that changes nothing.

flowchart TD
  A["The dashboard that stayed green"] --> B["What has actually drifted?"]
  B --> C["Decompose movement before<br/>diagnosis"]
  C --> D["Build sentinels as falsifiable<br/>claims"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#B96D00,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Build sentinels as falsifiable claims

A sentinel is not a chart or a generic anomaly detector. It is a predeclared claim that a particular observable change is evidence for a bounded failure hypothesis, on a defined population, with an owned response. NIST's AI Risk Management Framework calls for monitoring the functionality and behaviour of systems and components in production, linking measurement to deployment context and documenting responses to emergent risks.[6] A later NIST monitoring report emphasises that post-deployment practice remains fragmented, with barriers including drift detection, distributed logging and policy complexity.[7] The practical answer is not one universal metric. It is a small portfolio of explicit sentinel contracts.

Instrument typed events, not private reasoning

The trace should make the decision reconstructable without asking a model to reveal hidden chain-of-thought. OpenTelemetry distinguishes operations with duration, represented as spans, from named events that mark checkpoints, state changes and outcomes.[8] That distinction maps well to a decision system. Retrieval, calculation and tool calls are spans. Policy selection, evidence acceptance, human approval, effect confirmation and exception creation are events. Each carries stable semantic attributes.

A minimal decision receipt should include a case identifier or privacy-preserving reference, intent and cohort, model and workflow versions, evidence references with observed and effective times, expected and applied policy versions, tool name and contract version, proposed and approved action, authority identity, effect identifier, readback result and final disposition. The trace should store exposed rationale or structured checks only when required. Monitoring needs the evidence of control flow, not an invented transcript of cognition.

Use controlled comparisons to localise

Dashboards reveal association. Diagnosis needs an intervention or a comparison that holds plausible causes fixed. A small replay matrix is often more useful than a larger catalogue of drift metrics.

Reference replay asks whether a known set of cases changes under the current build. A policy swap tests whether the observed decision change follows the intended rule. A contract probe checks schema and semantic edge cases without waiting for live failures. Standardisation removes case-mix composition. A stratified outcome sample tells the team whether an early control signal corresponds to harm, customer friction or merely harmless implementation variation.

Design the measurement before selecting the threshold

Statistical significance is only one gate. A sentinel also needs an effect-size floor, a minimum denominator, a consequence model, correction for repeated or multiple tests, and an alert budget. Several tests over the same events are correlated evidence, not independent votes. A tiny but precisely measured change may be operationally irrelevant; a large change in a rare, irreversible action may require containment before a conventional significance threshold is crossed.

The current and reference windows should match the failure's timescale. A short window catches release shocks but overreacts to weekday or campaign effects. A long window smooths noise but can hide a sharp break. One practical design runs a release-aligned sentinel, a seasonal sentinel and a slow-trend sentinel as separate tests with separate hypotheses. Their outputs should not be averaged. A release marker can explain why a change began; a seasonal reference can test whether it is expected; a cumulative trend test can expose baseline chasing.

Baselines should be approved snapshots or versioned seasonal references, not silently rolling windows that absorb the behaviour under examination. Reference refresh is a governed change: identify the candidate period, exclude known incidents, test subgroup performance, record applicable policy and tool versions, obtain an owner decision, then preserve the retired baseline for replay. Release markers, policy effective times and contract versions must remain queryable because an unexplained time series is a weak diagnostic instrument.

Validate the sentinel as an instrument

A sentinel should pass its own evaluation before it is allowed to page an operator or narrow authority. Construct a fault library with at least one episode for each claimed failure: stale evidence, shifted cohort mix, wrong policy version, schema-semantic change, omitted workflow step, changed conditional model behaviour and delayed outcome deterioration. Inject one fault at a time first, then combinations that can cancel in the aggregate. Include negative controls such as planned seasonality, authorised policy changes and harmless latency variation.

Measure detection probability, false-alert rate, time-to-detection distribution, localisation accuracy and the cost of the prescribed response. Lead time is meaningful only relative to the first moment the fault was observable and the moment a consequential outcome could have been prevented. A sentinel that alerts early but routinely points to the wrong component may increase recovery time. A sentinel with perfect historical recall may be unusable if it pages every night.

Run the library against shadow traces before release, replay it after telemetry changes, and retain missed episodes as regression tests. Independent human review should label whether the alert corresponded to a control deviation and whether that deviation was material. The sentinel is a fallible measurement system, so its calibration, blind spots and retirement criteria belong in the assurance case.

Run a minimal sentinel suite

The following standard-library Python programme creates a reference population and three current populations. The negative case changes only cohort mix. The hidden-system case keeps both the mix and decision rate stable while injecting stale policy, a skipped verification step and failed readback. The third case changes within-cohort decision behaviour. It reports population total variation, raw and standardised review-rate change, workflow Jensen-Shannon divergence, stale-policy rate and readback failure.

Assumptions are deliberately visible: events are deterministic synthetic records; cohort labels are known; the reference mixture is approved; thresholds are illustrative; and the classifier is a decision rule, not a learned detector. Expected output is shown after the code. The artefact's purpose is to reveal why one aggregate metric cannot identify the failure class.

from __future__ import annotations

from collections import Counter, defaultdict
from dataclasses import dataclass
from math import log2
from typing import Callable, Iterable, Sequence


@dataclass(frozen=True)
class Event:
    cohort: str
    decision: str
    policy_applied: str
    expected_policy: str
    workflow: tuple[str, ...]
    readback_ok: bool


def build_events(
    cohort_counts: dict[str, int],
    review_rates: dict[str, float],
    *,
    expected_policy: str,
    stale_policy_fraction: float = 0.0,
    omit_verify_fraction: float = 0.0,
    readback_failure_fraction: float = 0.0,
) -> list[Event]:
    """Create repeatable synthetic decision events."""
    events: list[Event] = []
    total = sum(cohort_counts.values())
    stale_cutoff = round(total * stale_policy_fraction)
    omit_cutoff = round(total * omit_verify_fraction)
    readback_cutoff = round(total * readback_failure_fraction)
    global_index = 0

    for cohort, count in cohort_counts.items():
        reviews = round(count * review_rates[cohort])
        for local_index in range(count):
            stale = global_index < stale_cutoff
            omit = global_index < omit_cutoff
            events.append(Event(
                cohort=cohort,
                decision="review" if local_index < reviews else "proceed",
                policy_applied="v1" if stale else expected_policy,
                expected_policy=expected_policy,
                workflow=("retrieve", "decide", "record") if omit else
                         ("retrieve", "verify", "decide", "record"),
                readback_ok=global_index >= readback_cutoff,
            ))
            global_index += 1
    return events


def distribution(events: Sequence[Event], key: Callable[[Event], object]):
    counts = Counter(key(event) for event in events)
    total = sum(counts.values())
    return {item: count / total for item, count in counts.items()}


def total_variation(p, q) -> float:
    keys = set(p) | set(q)
    return 0.5 * sum(abs(p.get(k, 0.0) - q.get(k, 0.0)) for k in keys)


def js_divergence(p, q) -> float:
    keys = set(p) | set(q)
    midpoint = {k: 0.5 * (p.get(k, 0.0) + q.get(k, 0.0)) for k in keys}

    def kl(a, b):
        return sum(v * log2(v / b[k]) for k, v in a.items() if v > 0.0)

    return 0.5 * kl(p, midpoint) + 0.5 * kl(q, midpoint)


def decision_rate(events: Iterable[Event]) -> float:
    rows = list(events)
    return sum(event.decision == "review" for event in rows) / len(rows)


def standardised_rate(events, baseline_mix) -> float:
    by_cohort = defaultdict(list)
    for event in events:
        by_cohort[event.cohort].append(event)
    return sum(baseline_mix[c] * decision_rate(by_cohort[c]) for c in baseline_mix)


def sentinel_report(baseline, current):
    baseline_mix = distribution(baseline, lambda e: e.cohort)
    current_mix = distribution(current, lambda e: e.cohort)
    baseline_flow = distribution(baseline, lambda e: e.workflow)
    current_flow = distribution(current, lambda e: e.workflow)

    raw_delta = decision_rate(current) - decision_rate(baseline)
    standardised_delta = (
        standardised_rate(current, baseline_mix)
        - standardised_rate(baseline, baseline_mix)
    )
    stale_policy = sum(e.policy_applied != e.expected_policy for e in current) / len(current)
    omitted_verify = sum("verify" not in e.workflow for e in current) / len(current)
    readback_failure = sum(not e.readback_ok for e in current) / len(current)

    if stale_policy > 0.01 or omitted_verify > 0.05 or readback_failure > 0.02:
        classification = "decision-system drift"
    elif abs(standardised_delta) > 0.03:
        classification = "conditional decision drift"
    elif total_variation(baseline_mix, current_mix) > 0.10:
        classification = "population shift"
    else:
        classification = "no material signal"

    return {
        "population_tv": total_variation(baseline_mix, current_mix),
        "raw_review_delta": raw_delta,
        "standardised_behaviour_delta": standardised_delta,
        "workflow_js": js_divergence(baseline_flow, current_flow),
        "stale_policy_rate": stale_policy,
        "readback_failure_rate": readback_failure,
        "classification": classification,
    }


def show(name, report):
    print(name)
    for key, value in report.items():
        print(f"  {key}: {value:.3f}" if isinstance(value, float)
              else f"  {key}: {value}")


baseline = build_events(
    {"standard": 800, "vulnerable": 200},
    {"standard": 0.10, "vulnerable": 0.30}, expected_policy="v1")

population_only = build_events(
    {"standard": 500, "vulnerable": 500},
    {"standard": 0.10, "vulnerable": 0.30}, expected_policy="v1")

hidden_system_drift = build_events(
    {"standard": 800, "vulnerable": 200},
    {"standard": 0.10, "vulnerable": 0.30}, expected_policy="v2",
    stale_policy_fraction=0.30, omit_verify_fraction=0.25,
    readback_failure_fraction=0.12)

conditional_decision_drift = build_events(
    {"standard": 800, "vulnerable": 200},
    {"standard": 0.16, "vulnerable": 0.36}, expected_policy="v1")

for name, current in (
    ("population_only", population_only),
    ("hidden_system_drift", hidden_system_drift),
    ("conditional_decision_drift", conditional_decision_drift),
):
    show(name, sentinel_report(baseline, current))

population_only
  population_tv: 0.300
  raw_review_delta: 0.060
  standardised_behaviour_delta: 0.000
  workflow_js: 0.000
  stale_policy_rate: 0.000
  readback_failure_rate: 0.000
  classification: population shift
hidden_system_drift
  population_tv: 0.000
  raw_review_delta: 0.000
  standardised_behaviour_delta: 0.000
  workflow_js: 0.138
  stale_policy_rate: 0.300
  readback_failure_rate: 0.120
  classification: decision-system drift
conditional_decision_drift
  population_tv: 0.000
  raw_review_delta: 0.060
  standardised_behaviour_delta: 0.060
  workflow_js: 0.000
  stale_policy_rate: 0.000
  readback_failure_rate: 0.000
  classification: conditional decision drift

A production-shaped incident

A regulated lender uses a decision-support service to prepare affordability evidence and recommend either normal processing or specialist review. A human remains accountable for adverse or exceptional decisions. The service has five relevant components: a document-evidence connector, a deterministic affordability calculator, a policy service, a language model that drafts the case rationale, and a workflow that verifies required evidence before recommendation.

On release day, the document connector moves from contract 3.8 to 4.0. A previously explicit unknown value becomes a missing field. The connector still returns HTTP success, so availability and latency remain healthy. Two days later, policy version 12 becomes effective. It requires specialist review when income evidence is missing for a vulnerable-customer flag. One workflow route, however, caches policy version 11 for seven days.

The incoming population also becomes safer because a marketing campaign attracts more salaried applicants. That mix change lowers the expected review rate. The stale policy and omitted verification raise it. The two effects almost cancel. Model score distributions, aggregate review rate and human override rate remain near their baselines. There are not yet enough matured cases for outcome analysis.

What the sentinel portfolio sees

The evidence sentinel reports a rise in cases with missing rather than explicit unknown income status, concentrated immediately after contract 4.0. The policy sentinel reports that 30 per cent of cases on one route applied version 11 after version 12 became effective. The workflow sentinel detects that the required verify event is absent from 25 per cent of vulnerable-customer traces. The tool sentinel's edge-case probe reproduces the semantic change, and readback shows that some recorded evidence states do not match the connector's raw response.

The population sentinel separately reports the safer applicant mix. Direct standardisation shows no change in within-cohort review rates, so the stable aggregate rate is recognised as cancellation rather than reassurance. The model sentinel remains quiet because the model is not the changed component. Outcome analysis is still pending. This is the important operating pattern: early sentinels localise a control deviation; delayed outcomes establish consequence.

Human review can otherwise mask the incident. A reviewer may notice the missing evidence, retrieve it manually and still approve the case. Final accuracy then looks healthy because a person repaired the trajectory. The repair is operational evidence, not noise. The case record should preserve the model proposal, the pre-review evidence state, the human correction, the final decision and the reason for change. Rising correction effort can be an early burden signal even when customer outcomes remain protected.

The inverse is also possible. Override rates can stay flat because reviewers accept a fluent rationale without noticing that its policy citation is stale. Monitoring only the human's final choice would then reproduce the same blind spot. Stratified review must inspect the supporting evidence and policy applicability, not merely agreement between human and system.

Contain before completing the investigation

The response does not require shutting down the entire service. The authority kernel can route the affected slice to specialist review, pin the connector to the last known equivalent contract or introduce an explicit adapter, invalidate the stale policy cache, and require a readback before a recommendation is accepted. A replay over recent affected cases estimates scope. A stratified human sample tests materiality. Every changed decision is linked to the evidence and policy version that produced it so that correction is possible.

Current US interagency model-risk guidance for quantitative models emphasises ongoing monitoring, outcome analysis, limitations and use context; it also warns that a sound model can still create risk when misapplied or misused.[10] A compound decision service needs that discipline plus explicit controls for deterministic policy, tools and workflow, because those elements may fall outside a narrow model definition. Where the EU AI Act applies, Article 72 establishes post-market monitoring duties for providers of high-risk AI systems.[11] These sources support ongoing, contextual monitoring. They do not prescribe the synthetic thresholds or architecture used here.

stateDiagram-v2
  [*] --> Claim
  state "Drift, change and failure are<br/>not synonyms" as Claim
  state "Thought experiment one, freeze<br/>the model" as Test
  state "Thought experiment two: freeze<br/>conditional behaviour" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Where sentinels fail

Typed sentinels make a compound system more observable, but observability is not omniscience. A sentinel can detect only what the trace represents. If evidence provenance is dropped before logging, policy versions are not immutable, human corrections happen outside the case record, or a tool reports success without a verifiable postcondition, the monitor inherits those blind spots. More events do not repair false semantics.

A second boundary is the reference itself. An approved baseline may already contain discriminatory treatment, a hidden workaround or an unrecognised control failure. Monitoring deviation from it can preserve the wrong behaviour with great precision. The baseline therefore needs substantive validation, periodic challenge and explicit expiry. Production-readiness work has long argued for tests and monitoring that cover data, features, models and infrastructure rather than relying on one offline score.[12]

Third, high-dimensional slicing creates sparse counts and alert inflation. A system with many intents, jurisdictions, policies and tools can generate thousands of plausible comparisons. Hierarchical monitoring can begin with high-consequence slices, then drill down after an aggregate control signal. Sequential methods, confidence intervals and false-discovery control may help, but no statistical correction chooses the right harm threshold. That remains a governance and domain decision.

Fairness monitoring sharpens this tension. Aggregate standardisation can remove a broad population-mix effect while concealing a changed decision rule for a small intersectional group. Yet unrestricted intersection search can manufacture unstable findings from tiny denominators. Predeclare protected and high-vulnerability slices, report uncertainty, combine quantitative alerts with case review, and maintain a route for affected people to contest outcomes. Absence of a statistically stable estimate is not evidence of absence of harm; it may instead require narrower authority or deliberate evidence collection.

Fourth, adaptive systems can change their own routing in response to monitoring, and adversaries can target the monitored surface while shifting harm elsewhere. Randomised audits, hidden canary cases, independent outcome sampling and failure injection reduce that risk. They do not eliminate it. Fifth, a stable decision ontology may not exist in exploratory or creative systems. When the system's purpose, actions and success criteria are still changing, sentinels should monitor hard safety and authority invariants while treating softer behavioural movement as research data rather than a release incident.

The serious competing baseline

For a simple, single-model classifier with stable preprocessing, immediate labels, no external tools, no policy branching and a reversible low-consequence action, conventional input, calibration and outcome monitoring may be enough. Adding seven sentinel families would create cost without useful localisation. The decision-centred approach earns its complexity only when multiple changing components can independently alter treatment, labels are delayed, or the consequence of a silent control failure justifies earlier containment.

This negative control matters because the recommendation is not “instrument everything”. It is instrument every independent source of consequential change that cannot be inferred reliably from the final outcome. The unit of monitoring should expand only as the unit of causation and accountability expands.

The drift decision record

Before adding a sentinel or responding to an alert, complete the following record. A blank field is itself a finding.

The record prevents a common failure: teams begin with an available metric, discover a threshold from historical noise, and only then invent a story about what the alert means. The order should run the other way. Start with a consequential failure, identify its earliest observable evidence, define a discriminating test, and attach a response that is proportionate and reversible.

sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: The dashboard that stayed green
  M->>E: Drift, change and failure are not synonyms
  E-->>M: compare rival explanations
  E->>D: Monitor what you must be prepared to defend
  D-->>Q: revise scope or act

Monitor what you must be prepared to defend

A model can remain numerically stable while the evidence it sees becomes stale, the policy it applies becomes obsolete, the tool it calls changes meaning, or the workflow skips a required check. An aggregate output can also move because the population changed while every conditional decision remained stable. Both errors arise from choosing the wrong monitoring unit.

The defensible unit is the decision chain from authorised case and evidence to approved action and verified outcome. Model, population, policy, tool, workflow and outcome sentinels each observe a different surface of that chain. Standardisation separates mixture from treatment. Typed events make the path reconstructable. Replay and probes localise plausible causes. Outcome sampling determines whether an early control deviation mattered.

This changes the architecture decision. Monitoring is no longer a dashboard attached after deployment. It becomes part of the decision contract: versioned semantics, effective-time policy, traceable authority, testable postconditions, approved references and predeclared response. The goal is not to stop all change. It is to distinguish intended adaptation from unexamined movement early enough to preserve evidence, limit harm and recover deliberately.