The strong rivalSpeed is a product capability

Behavioural data can reveal friction that interviews miss. Online experiments can distinguish two interface choices under real conditions. Frequent releases can repair search, retrieval and ranking defects before a quarterly review meets. A governance design that treats every event as sensitive training material, every change as permanent and every expert opinion as final can make a product slower and less responsive without making it safer.

The answer is consequence-sensitive routing. A spelling suggestion can often learn within bounded traffic and automatic rollback. A credit decision cannot infer the quality of rejected applicants from repayment records that exist only for approved applicants. A support assistant may use an uncertain thumbs-down to send a conversation for review, but should not immediately train on it as a correctness label. The same event can be useful for one purpose and invalid for another.

Fast loops therefore need explicit limits: a stated product decision, an eligible population, a versioned exposure policy, a valid outcome, a short blast radius, hard release gates and a reversal mechanism tested before launch. Slower loops need protected outcome windows, adjudication, affected-person rights and evidence outside the deployed policy. Learning cadence is an output of this design, not an aspiration imposed on every signal.

Original constructThe eight-stage Feedback Provenance Loop

The Feedback Provenance Loop is this article's synthesis. It joins the event that could have happened to the evidence used to change the product. A trace is complete only when each stage can be reconstructed for the relevant decision class. The stages are necessary distinctions, not a maturity score.

Signal semanticsSix lanes, not one feedback lake

Feedback becomes safer when signal families stay separate until a recorded interpretation decision connects them. The separation prevents a convenient event from inheriting a stronger meaning than it has. It also lets privacy, retention, access and challenge rules follow the actual relationship rather than a generic telemetry category.

A signal may move between lanes only through an interpretation record. For example, an accepted draft begins as behaviour. A later verified correction can create expert evidence. A resolved customer impact can create an outcome. The original acceptance does not retroactively become a correct label, and the later correction does not authorise every future use.

Negative spaceWhat the policy prevents you from seeing

Selective labels arise when an outcome is observed only after an earlier decision. A lender sees repayment for approved applicants, not the repayment that rejected applicants would have produced under credit. Treating rejection as a bad outcome bakes the earlier policy into the target. Deleting rejected cases also hides the population for which model quality remains unknown.

The same structure appears elsewhere. A screening product sees job performance only among selected candidates. A fraud system sees investigated cases more clearly than ignored ones. A recommendation engine sees responses to displayed items. Predictive policing sends attention to selected places, generating records that can justify more attention. Deployed predictions can also change behaviour and conditions, so the future population is partly an effect of the model.

The trace must therefore include the eligible denominator, exposure propensity, selection mechanism and missing outcome. Credible remedies depend on context. Randomised exploration may be appropriate for low-consequence ranking within hard safeguards. Natural or quasi-random variation can sometimes support policy evaluation, but only under defensible assumptions. Targeted follow-up can reduce censoring. In consequential settings, the honest result may be to collect independent evidence or decline the update.

Time scaleFast signals can outrun slow truth

A weekly update may optimise immediate conversion while returns, defaults, service failures or conduct consequences arrive months later. If the product repeatedly changes before the consequential outcome window closes, attribution becomes weak and guardrails can never catch up. The team sees a dense stream of proximal wins and a censored stream of later costs.

Define an outcome clock for each product promise. A search ranking may have useful short-window measures, but repeated use, abandonment and complaint can still matter. A lending decision requires the relevant repayment horizon and adverse outcomes. A drafting assistant may need later correction, customer response and incident data. Early signals can support monitoring or triage while the outcome remains pending; they should not be promoted into final training truth because they arrive first.

Version persistence is essential. Preserve the model, data, prompt, retrieval, interface and policy versions that produced each exposure. When an outcome arrives, join it to the exact action and contemporaneous context rather than the current product. Set a freeze or cohort boundary when updates would make delayed comparison unintelligible. Speed without temporal identity creates a stream that no longer explains its own consequences.

Learning envelopeDesign the experiment before collecting the signal

A bounded online loop begins with an estimand expressed in product language. State which change, for which eligible population, is expected to improve which outcome over what period. Then name the interference that could make the comparison misleading: ranking affects later choice, one user's action changes another's inventory, a support intervention changes future contact, or a reviewer learns from both variants. Without this statement, random assignment can produce a precise answer to the wrong product question.

Set the unit of exposure to match the mechanism. Randomising individual impressions can be inappropriate when memory, repeated use or shared resources carry treatment across events. A user, account, location, reviewer, time block or operational cohort may be the relevant unit. Preserve assignment, eligibility and non-exposure records even when telemetry is lost later. Analyse departures from assignment separately from outcome so a product defect that suppresses delivery is not interpreted as user preference.

Predefine the evidence hierarchy. The primary outcome should represent the product promise as closely as the observation window allows. Proximal behaviour can help diagnose mechanism, but it should remain secondary when it does not establish the promised consequence. Hard guardrails cover outcomes that an average gain cannot compensate for, including serious safety, rights, conduct or distributional failures. Decide what pauses exposure, what triggers automatic rollback and what requires human adjudication before looking at variant results.

Preserve negative and neutral results. A stopped test, an unchanged outcome or a rollback contains information about the operating envelope and should remain linked to the versions and population tested. Deleting unsuccessful experiments lets teams repeat hazards and creates publication bias inside the product organisation. A decision not to update can be the correct learning result when uncertainty remains high, the observed benefit is only a proxy or a gate fails.

Plan for novelty and gaming. Users may discover the experiment, creators may optimise content for the ranking signal, reviewers may adapt to prompts and operators may route difficult cases away from measurement. Record these responses as changes to the environment rather than noise to clean away. Use qualitative review, sampled traces and adversarial cases alongside aggregate results. If the intervention changes who remains eligible or willing to participate, recalculate the denominator and reopen the causal claim.

The experiment receipt should survive release. It links the hypothesis, eligibility rule, exposure unit, versions, signal lanes, outcome clock, guardrails, analysis decision, affected-party considerations, owner and rollback state. Later outcome joins and appeals append to the receipt rather than overwriting it. When the product changes enough that the old assignment or holdout no longer represents current use, retire the evidence explicitly. Provenance is valuable because it lets the organisation know where learning stops.

Failure mapFive ways a product trains on itself

These failures can coexist. A ranking policy changes exposure, incentives favour acceptance, outcome delay hides later repairs and the same stream evaluates the update. One feedback-quality score would conceal the chain. A release decision needs named contamination risks and evidence actions at the failing stage.

Feedback rightsA provenance ledger for every signal family

The feedback rights and provenance ledger is this article's second synthesis. For each signal family, record the eligible and exposed denominator; presentation and system version; user role, affected party and power relationship; signal type and purpose; consent or other applicable lawful basis; minimisation and retention; interpretation limits; delay, censoring and selection; verification and disagreement; update destination; protected evaluation; rollback trigger; owner; contest route; and retirement rule.

This ledger is not permission to collect every field. It is a decision record that can show when collection is unnecessary or disproportionate. Where personal-data processing is in scope, apply the relevant lawfulness, fairness, transparency, purpose-limitation, minimisation, accuracy, storage and rights duties. Product usefulness does not create a legal basis. Consent is not automatically valid in a power-imbalanced setting, and a broad service notice does not make an unrelated training purpose specific.

Users and affected people are not interchangeable. A caseworker may be the user, while an applicant bears the consequence. A customer's appeal can correct a product record, while an employee's acceptance behaviour reflects workplace incentives. Record who provided a signal, who is affected by reuse and who can contest. Product teams should not improve a model from an appeal while leaving the original decision, downstream record or remedy unresolved.

Retirement matters because feedback semantics decay. A changed interface can alter click meaning. A new incentive can alter acceptance. A policy update can change eligibility. A model can learn the challenge set. Owners should state when a signal definition, cohort or holdout expires and what evidence is required before reuse. A permanently open stream with a permanently fixed label is usually a provenance failure.

Worked controlRepair the case before learning from it

An applicant appeals a decision and supplies evidence that a source record was wrong. The operational path first pauses any dependent action, verifies the correction, repairs the affected decision and propagates the change to downstream records. The person receives the applicable explanation, outcome and remedy. None of these duties should wait for a model-training cycle.

The learning path begins separately. The event enters the incident or appeal lane with restricted access, a stated purpose and an adjudication record. The team asks whether the failure is individual, systemic or still uncertain. It may repair a data-quality rule, retrieval source, interface or policy without placing personal details into training. If a generalisable failure hypothesis remains, a minimised case pattern can inform a challenge set under appropriate controls.

Release evaluation then tests the proposed change against the protected challenge, broader operating envelope and distributional gates. Passing the corrected case is necessary but not sufficient; a patch can overfit the appeal or damage other populations. The release receipt links affected-record repair, learning decision, test evidence, owner and rollback. The appeal is evidence of a product obligation, not free labelling labour.

Interactive instrumentFeedback Loop Governor

The Governor routes a proposed feedback use. Set the actual operating conditions, not the launch aspiration. Its output is one of five actions with a contamination risk and evidence action. It does not calculate a quality or maturity score.

Five routesWhat each decision permits

Learn within a bounded online loop

Use this only when consequence is low and reversible; eligibility and exposure are recorded; the outcome window fits the cadence; independent evidence is protected; traffic and authority are bounded; and hard gates can trigger tested rollback. The route authorises the stated update destination, not general reuse of the stream.

Use the signal for evaluation or triage, not training

A signal can identify cases to inspect or hypotheses to test without becoming a target label. Use this route for ambiguous preferences, proximal behaviour, stable products that do not need online learning, and repaired incidents whose general meaning remains uncertain. Keep evaluation claims proportional to the sampling design.

Collect independent or counterfactual evidence first

Choose this when exposure, denominator, selection, censoring, delayed outcome or policy-shaped data prevents a valid inference. Preserve the current cohort, design an ethical evidence source and state the assumptions it can support. Do not fill the missing population with model-generated labels and call them independent.

Quarantine the signal and investigate corruption

Use quarantine when incentives are gaming the measure, purpose has drifted, rights or version provenance fail, contest is unavailable, or interpretation is disputed. Stop movement into update destinations, preserve lineage, repair access and adjudicate meaning. Quarantine is a control state with an owner and exit test, not an indefinite data lake.

Stop or roll back the feedback-driven update

Apply this when severe or irreversible consequence combines with absent gates, rollback, repair or ownership, or when the same stream trains and claims to independently evaluate a broad release. Contain exposure, restore the last evidenced version, repair affected records and rebuild the evaluation boundary before considering another update.

Route testsTen cases the Governor must distinguish

Safe fast loop: a reversible search-ranking change uses bounded traffic, random holdout, verified task outcome, hard guardrails and automatic rollback. It may learn online within that envelope.

Click corruption: ranking exposure is incomplete and behaviour is treated as preference. The team needs counterfactual impressions or bounded exploration before training.

Selective outcome: repayment is observed only after approval. Rejected applicants are an unobserved counterfactual population, so independent evidence must precede an update.

Agreement trap: acceptance occurs under throughput pressure and correctness is not verified. Quarantine the inferred labels and investigate workflow incentives and downstream repair.

Sycophancy: explicit preference rewards agreement while factual evaluation shares the same stream. Quarantine the signal, separate style from truth and build an independent challenge.

Complaint silence: contest is difficult, denominator is missing and outcomes are censored by abandonment. Collect sampled affected-person and outcome evidence rather than interpreting silence.

Delayed harm: weekly conversion optimisation outruns monthly returns. Hold the cohort and collect the consequential outcome before allowing the signal to train the promise.

Evaluation leak: production feedback trains and evaluates a broad model release with weak gates. Stop or roll back, reconstruct lineage and replace the claimed test.

Appeal repair: the affected record and remedy are complete, but the general lesson remains uncertain. Use the event for evaluation or triage and move only a minimised, adjudicated pattern through a separate decision.

Stable product: outcomes remain acceptable and available feedback cannot distinguish quality from exposure. Do not learn online merely to appear adaptive. Monitor and evaluate until better evidence supports a change.

Operating proofReceipts, holdouts and rollback

A feedback provenance receipt should identify the product decision, eligible population, exposure mechanism, signal lane, action, outcome window, interpretation, destination, versions, lawful or consent basis where relevant, access, owner, evidence boundary and rollback. It should also state what the signal cannot establish. A concise limitation is more useful than a long generic disclaimer.

Protect holdouts organisationally as well as technically. Restrict who can inspect labels, prevent feature engineering against repeated failures, rotate or replenish tests under a documented protocol and maintain separate challenge sets for prohibited behaviour. Synthetic or model-generated feedback may broaden scenario coverage, but it is not independent evidence about the same model family unless validated against an external criterion.

Exploration needs an ethical envelope. Randomise only where alternatives satisfy safety, service and rights constraints; cap traffic and duration; predefine stop rules; and analyse non-participation. Consequential decisions may make random exploration inappropriate. The Governor's request for counterfactual evidence is not permission to expose people to unmanaged harm.

Rollback must restore more than model weights. A feedback-driven update may change cached outputs, retrieval indexes, user profiles, rules, queues and downstream records. The rollback plan names each state, its owner and reconciliation procedure. A tested indicator must connect a hard-gate breach to containment within the required time. Manual rollback can be adequate for a slower, reversible workflow if it is staffed and rehearsed.

Measure the loop by link, not by one score. Track missing eligibility and exposure records, signals moved between lanes without interpretation, outcomes arriving after cohort closure, holdout overlap, unresolved appeals, failed repairs, purpose changes, access exceptions and rollback drills. Preserve rates by population, consequence and version. An aggregate green indicator must not compensate for a failed rights or severe-harm gate.

Review incentives directly. Ask why a user clicked, accepted, stayed silent or appealed; compare stated reasons with observed context; and sample people who left. Examine whether reviewers are rewarded for throughput, agreement or low incident counts. Product teams should expect behaviour to adapt when a measure becomes consequential. The response is to improve evidence diversity and control, not to accuse users of corrupting clean data.

Independent re-evaluation closes the loop but does not end responsibility. Monitor the released update in its new environment because deployment can change the population and response it encounters. Reopen the route when exposure, incentive, population, outcome delay, interface, model family, legal purpose or rollback capacity changes. Retirement is a valid outcome when the evidence can no longer support safe learning.

Evidence boundaryWhat the sources do and do not establish

Performative-prediction and recommender studies establish that deployed systems can shape later data. Selective-label research explains missing outcomes after prior decisions, with methods tied to particular assumptions and settings. Goodhart variants organise ways a proxy can fail when optimised. None provides a universal correction formula for every product.

Datasheets, model cards and production-ML work support documentation of collection, intended use, evaluation and cross-component dependencies. The NIST AI RMF Core supports continuous lifecycle governance, production monitoring, independent review, feedback and risk response. AI RMF 1.0 remains the published framework while a revision is in progress. These are governance resources, not evidence that a particular loop is lawful or effective.

GDPR principles apply where the processing is in scope, with duties depending on facts, role and lawful basis. The EU AI Act's data-governance, logging, post-market monitoring, incident and corrective-action provisions depend on system classification, role, provision and application timeline. OECD principles were updated in 2024 and operate at principle level. Product teams still need jurisdiction-specific advice and evidence for the concrete use.

The eight-stage loop, six lanes, feedback rights and provenance ledger, and Governor are this article's synthesis. They separate user response, inferred label, causal outcome, product objective, privacy purpose and independent evaluation. Their value must be tested through complete receipts, case reconstruction, protected evaluation, affected-record repair and successful rollback under realistic operating conditions. Review those controls after every material change to exposure, incentive, population, purpose or outcome timing.