At the quarterly gate, every slide described what had already happened. The team had built a retrieval layer, cleaned records, trained reviewers, integrated a sandbox and delivered an impressive demonstration. The next request was larger than the first. Yet nobody could name the operating decision that another tranche would resolve, the uncertainty it would reduce or the threshold that would cause funding to stop.

The room was being asked to reimburse history. The artefacts were real and some might be reusable, but neither fact made the prospective expenditure valuable. The relevant comparison began today: what can the next tranche teach or enable, what will it cost in full, what exposure and delay accompany it, and what better use exists for the same capital and attention?

A disappointing result does not always demand termination. A failed test may reveal that the chosen workflow was wrong, a metric did not represent the consequential outcome or a bounded technical defect is repairable. Conversely, a polished result may deserve no continuation when it repeats a known demonstration, leaves integration economics untouched or depends on a value theory already contradicted by evidence.

The funding decision is sequential. Each gate observes what the last tranche revealed and decides whether scale, one bounded test, a narrower route, termination or a different intervention now dominates. Wald’s work on sequential tests supplies the discipline of evidence boundaries, but a socio-technical pilot is not reduced to one universal statistical formula.[1] Business value, operational authority and risk remain part of the gate.

Prospective logicAsk what the next tranche can change

The deferred decision must be consequential and specific. It might be whether to integrate into the production workflow, extend to a new population, automate a bounded class of cases, contract with a provider or terminate the capability. “Learn more about AI” is too broad. A decision owner should be able to describe what will happen under a favourable, adverse or ambiguous result.

Remaining uncertainty is valuable to reduce only when it is material to that decision. If leadership will scale regardless of the answer, the test is theatre. If no result could overcome a legal prohibition, capability shortfall or dominant non-AI alternative, another experiment cannot create an option. If a narrow result would change the population, control or investment size, the evidence retains decision value.

Reducibility asks whether the proposed design can actually address the uncertainty. A selected sample cannot establish a population outcome. Another prompt demonstration cannot resolve workflow adoption. A model-quality test cannot determine whether released time is captured. Repeating the same weak evidence with more polish does not turn ambiguity into information.

The full prospective cost includes data preparation, integration, evaluation, control work, user time, vendor fees, security review, incident readiness, delay and the opportunity cost of displaced initiatives. Prior spend is excluded from this comparison except where it changes future options. Salvage from prior work is recorded separately whether the pilot continues or stops.

This is not a claim that evidence value can always be reduced to one precise number. The contract can use bounded qualitative judgments, explicit ranges and dominance. A cheap, safe test that could reverse a large decision may justify itself without elaborate monetisation. An expensive repeat that cannot change any boundary fails even if its dashboard is detailed.

Irreversibility sharpens the comparison. Waiting can preserve value when scale is difficult to unwind, but waiting also has cost when a market or regulatory window closes. Pindyck’s investment logic treats uncertainty and irreversibility as reasons to value timing rather than rush through a point estimate.[12] The gate makes that timing exposure visible instead of assuming delay is free.

A gate packet should show the decision tree before it shows the latest metric. For each plausible result, it states the route, owner, funding consequence and operating action. If every branch still says “continue”, the test is not connected to a decision. If an adverse result triggers a narrower population while a favourable result triggers production integration, the evidence has a defined use.

Management attention is a real constraint. A small pilot can absorb senior review, risk interpretation, procurement effort and scarce engineering slots far beyond its vendor invoice. The contract caps those resources as explicitly as cash. A proposal that requires constant executive rescue may be exposing an immature operating model, even when its direct spend remains below budget.

Benefits and harms should share the same horizon. A pilot should not count immediate time saved while postponing appeals, rework, customer attrition or control remediation beyond the measurement window. Nor should an early nuisance cost automatically outweigh a durable improvement. The next test sets a horizon appropriate to the causal mechanism and records unresolved tail effects for the scale gate.

Before repairClassify the failure you observed

A repair tranche should match the failure class. Otherwise the team can keep changing visible parts while the binding problem remains untouched. Seven classes cover the most common operating case without pretending they are mutually exclusive.

Invalid value theory is the hardest failure because better implementation cannot rescue it. If faster drafting does not affect customer resolution, more accurate drafting remains economically secondary. If released time is not removed, redeployed or converted into throughput, technical productivity is not captured value. The pilot can still yield reusable evaluation assets, but those are salvage rather than prospective benefit.

Measurement failure is different. Suppose a support pilot misses an average handling-time target because the new policy deliberately spends longer on complex cases while reducing repeat contact. If repeat contact is the consequential outcome, the original metric was incomplete. The team may correct the outcome definition, but it must disclose that correction and avoid moving the finish line simply because the original result was unfavourable.

Technical failure can be repairable when the mechanism is known. A bounded retrieval gap caused by one source and one access rule may warrant a focused test. A diffuse reliability problem across populations, languages and changing providers may not. “The model will improve” is not a repair hypothesis unless the next tranche names the change, comparator, boundary and decision rule.

Gate architectureThe pilot continuation contract

The contract is written before the next tranche and remains attached to the funding record. It names the deferred operating decision, the hypotheses still capable of changing it, current evidence, disconfirming evidence, the next test, eligible population, comparator, measurement window and the five possible routes. It then caps new spend, elapsed time, exposure and management attention.

Continuation, narrow, redirect, scale and stop thresholds should be operational rather than rhetorical. A scale boundary can require evidence across the relevant population, control readiness and a credible path to captured value. A narrow boundary can restrict task, population or authority. Redirect can move investment to the binding workflow constraint. Stop can be triggered by unacceptable consequences, non-reducible uncertainty or a dominated operating case.

The funding ceiling includes the integration and control work needed to interpret the result. A cheap isolated test that proves nothing about the operating path is not necessarily efficient. Conversely, the gate should not demand full-scale integration merely to learn whether a material hypothesis is false. The test is the smallest safe tranche that can change the decision.

The owner of evidence should not be the sole owner of the gate. Independent finance, risk, portfolio or transformation authority must be able to stop despite sponsor commitment. Staw’s experimental work and later meta-analysis show how responsibility, justification and project factors can escalate commitment after negative information.[2][3] Independence creates a practical counterweight to that pressure.

Ambiguity does not default to continuation. The contract states what an inconclusive result means. If the test had adequate power and faithful execution, ambiguity may cross a stop or narrow boundary. If the metric, comparator or instrumentation failed, a measurement repair may be justified, but only once and under a revised, disclosed limit. Endless underpowered samples are not a learning strategy.

The contract also carries an expiry. Market conditions, data permissions, providers, policies and model versions change. Evidence that once supported the next tranche can decay. An expired pilot either re-enters discovery with a new decision and boundary or closes. It cannot preserve priority simply by keeping a demo and a small team alive.

Gate minutes record dissent and boundary changes. A sponsor may argue that an unexpected result exposes a valuable new use case. That claim enters a fresh discovery record with its own population, economics and authority; it does not overwrite the failed hypothesis. Keeping the histories separate allows later reviewers to distinguish genuine redirection from narrative repair.

The counter-case is intentionally small. One material uncertainty remains: whether assisted review changes resolution quality for the eligible production population under the approved human authority. A short comparison uses a credible baseline, complete outcome window and predeclared threshold. Crossing the favourable boundary triggers controlled scale; an uneven result narrows the population; failure triggers shutdown. The evaluation harness and labelled test set enter the shared library regardless of the route.

Scale is also a gate outcome, not permission to stop learning. Production monitoring tests drift, consequence, adoption and capture against operating thresholds. New populations or larger authority return to a fresh decision gate. The pilot contract ends when its specific deferred decision is taken; it does not become an evergreen mandate for the capability.

Negative controlsFour ways activity captures the gate

The moving finish line

The pilot misses its stated outcome, so usage, positive comments and demonstration applause replace it after the result. These signals may explain the failure, but they cannot become success retroactively. The gate retains the original outcome, records the miss and decides whether a disclosed measurement or workflow repair could change the operating decision.

The integration ransom

Sponsors argue that stopping would waste prior engineering and integration. Yet the next tranche recreates much of the scale cost without addressing value or capture. This is escalation logic: previous commitment makes further commitment feel necessary. The prospective gate ignores unrecoverable spend, values salvage independently and compares the next tranche with current alternatives.

Perpetual ambiguity

A tiny, selected sample cannot confirm or disconfirm the case. The next request proposes another sample with the same selection, missing comparator and weak outcome. More observations do not repair a biased design. The pilot stops or changes its evidence architecture. A bounded shadow or randomised window may be a genuine redesign; an equally weak repeat is not.

The wrong intervention

A pilot accelerates an upstream assessment, but a fixed downstream committee still meets weekly and determines throughput. A deterministic workflow change or rule-based prioritisation would remove more delay at lower cost and consequence. The correct gate outcome is replacement, even if the model performs well on its local task.

These controls do not imply that every missed metric or operational constraint kills the work. They force the team to name what failed and why the proposed response differs. McGrath’s real-options reasoning treats entrepreneurial failure as a way to learn and redirect, while Adner and Levinthal warn that an option needs defined boundaries and abandonment discipline.[4][5]

Strong rivalDo stop rules kill exploration too early?

Strict gates can destroy learning when capability, data quality and user proficiency improve through iteration. Exploration necessarily produces failure, and an organisation that funds only predictable success will underinvest in discovery. March’s account of exploration and exploitation explains why both modes matter and why each competes for scarce attention.[6]

The rival is accepted when the next tranche targets a named uncertainty, the uncertainty remains material, the proposed evidence can reduce it and exposure is bounded. The decision rule must genuinely change with the result. A poor first attempt may justify a second when it identified a repairable data boundary and the new design tests that boundary directly. This is disciplined exploration, not automatic continuation.

The rival fails when success definitions move after every result, repair classes change without diagnosis or the next tranche simply buys more time. An option is not indefinite permission. It has an exercise decision, an expiry and a maximum loss. If leaders cannot describe what evidence would make them stop, the pilot is already operating as an unacknowledged commitment.

Technology readiness evidence can help separate a laboratory achievement from an integrated operating capability. The US Government Accountability Office’s guide treats readiness assessment as evidence-based and tied to defined environments, integration and decision needs.[7] A readiness label should not substitute for the business, control or adoption case; it answers one part of the path.

NIST’s Core asks whether an AI system achieves intended purposes and whether development or deployment should proceed. Its Manage guidance includes monitoring, response, disengagement and deactivation.[8][9][10] The continuation gate turns those governance expectations into a tranche decision with an owner and exit.

Exploration can be protected with a portfolio envelope rather than by weakening every gate. Leaders may reserve capital for uncertain experiments, accept a high stop rate and evaluate whether the programme produces useful decisions over time. Each pilot still needs its own ceiling and exit. This separates a healthy appetite for exploration from attachment to one sponsored project.

Failure tolerance is therefore compatible with evidence discipline. Teams should not be punished for crossing a precommitted stop boundary honestly. Incentives can reward early disclosure, reusable evaluation work and clean shutdown. If careers depend on continuation, evidence owners will rationally avoid decisive tests and prefer metrics that remain promising but inconclusive.

Interactive instrumentPilot Continuation Gate

The Gate treats every recommendation as terminal or bounded. “Fund one tranche” carries a ceiling and expiry. It never returns open-ended continuation. Inputs cover the deferred decision, uncertainty, evidence, cost, exposure, repair class, salvage, alternatives, authority and threshold stability.

After the decisionShutdown hygiene and salvage receipts

Stopping funding is not the end of the operating obligation. Live data, credentials, model endpoints, scheduled jobs, vendor commitments, user promises and customer-facing interfaces must be found and disposed of. Owners verify deletion or retention bases, revoke access, archive or destroy models, close incident channels and communicate the change to affected teams.

Salvage is recorded in a separate receipt. Reusable datasets may have permission limits. Evaluation harnesses, test cases, control patterns, integration components and workflow findings can return to a governed portfolio library. Negative evidence about a population or provider can prevent duplicated cost elsewhere. None of these assets needs the failed pilot to remain funded in order to be useful.

The learning receipt names the hypothesis, evidence, failure class, boundary crossed, decision taken and reuse destination. It does not relabel every failed pilot as a success. Learning is valuable when it changes a later decision, avoids exposure or improves another evidence path. The receipt gives that change an owner.

Cleanup evidence is retained at the minimum level needed for accountability, audit and future learning. It records what was deleted, what was archived, the legal basis, the responsible owner and verification date. Raw customer or worker data do not need to remain accessible merely because a lessons-learned document cites them. The salvage plan keeps derived tests and metadata only where permission and provenance allow.

External commitments require special attention. A pilot may have created sandbox accounts, provider retention settings, temporary data transfers, licences, evaluation contractors or communications that imply future service. Procurement, legal, security and business owners reconcile these obligations. Closing the internal budget without closing them leaves cost and exposure behind.

Generative AI pilots may leave prompts, retrieved content, fine-tuning data, evaluation traces and third-party accounts across several services. NIST’s Generative AI Profile reinforces lifecycle risk management for these assets and dependencies.[11] ISO/IEC 42001 likewise frames AI management as an organisational system with responsibilities and continual control rather than a one-off technical artefact.[14]

Portfolio recycling considers the best next use of released people, attention and infrastructure. A stopped pilot may free a reviewer cohort for a stronger comparison elsewhere, release a scarce security review slot or make an evaluation dataset available to a deterministic solution. Redeployment is part of the economic outcome, not an admission that earlier work must be forgotten.

Firm adoption evidence shows that skills, data, complementary technologies and organisational support shape whether AI produces value.[13] A stop decision can therefore target the pilot while preserving a broader capability-building investment, provided leaders name that investment and evaluate it on its own terms.

ConclusionA gate is a promise about the next decision

At every tranche, separate past spend from prospective value. Name the operating decision being deferred, the uncertainty that still matters and the smallest safe test capable of reducing it. Count integration, controls, delay, exposure and opportunity cost. Compare the pilot with the best workflow, deterministic or vendor route available now.

Diagnose the observed failure before funding repair. Preserve original thresholds and disconfirming evidence. Give an independent owner authority to scale, fund once, narrow, redirect, stop or replace. Ambiguity is an outcome to interpret under the contract, not an automatic claim on more budget.

When the pilot stops, close live exposure and commitments, preserve governed reusable assets and issue a learning receipt tied to a changed decision. The discipline does not punish experimentation. It protects exploration from becoming indefinite activity and makes every new tranche earn its place.

A credible gate leaves an auditable answer to four questions: what evidence arrived, which boundary it crossed, who owned the resulting action and what happened to the remaining obligations. That record lets a later portfolio review distinguish a wise stop from neglect, a justified tranche from escalation and a genuine scale decision from momentum.

Evidence spineReferences

1. Wald, A. (1945), ‘Sequential Tests of Statistical Hypotheses’, The Annals of Mathematical Statistics, 16(2), 117–186. doi:10.1214/aoms/1177731118.
2. Staw, B. M. (1976), ‘Knee-Deep in the Big Muddy: A Study of Escalating Commitment to a Chosen Course of Action’, Organizational Behavior and Human Performance, 16(1), 27–44. doi:10.1016/0030-5073(76)90005-2.
3. Sleesman, D. J., Conlon, D. E., McNamara, G. and Miles, J. E. (2012), ‘Cleaning Up the Big Muddy: A Meta-Analytic Review of the Determinants of Escalation of Commitment’, Academy of Management Journal, 55(3), 541–562. doi:10.5465/amj.2010.0696.
4. McGrath, R. G. (1999), ‘Falling Forward: Real Options Reasoning and Entrepreneurial Failure’, Academy of Management Review, 24(1), 13–30. doi:10.5465/amr.1999.1580438.
5. Adner, R. and Levinthal, D. A. (2004), ‘What Is Not a Real Option: Considering Boundaries for the Application of Real Options to Business Strategy’, Academy of Management Review, 29(1), 74–85. doi:10.5465/amr.2004.11851715.
6. March, J. G. (1991), ‘Exploration and Exploitation in Organizational Learning’, Organization Science, 2(1), 71–87. doi:10.1287/orsc.2.1.71.
7. US Government Accountability Office (2020), Technology Readiness Assessment Guide: Best Practices for Evaluating the Readiness of Technology for Use in Acquisition Programs and Projects, GAO-20-48G. GAO guide.
8. National Institute of Standards and Technology (2023), ‘AI Risk Management Framework: Core’. NIST AIRC Core.
9. National Institute of Standards and Technology (2023), ‘NIST AI RMF Playbook: Manage’. NIST AIRC Playbook.
10. Tabassi, E. (2023), Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1. doi:10.6028/NIST.AI.100-1.
11. Autio, C. et al. (2024), Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1. doi:10.6028/NIST.AI.600-1.
12. Pindyck, R. S. (1990), ‘Irreversibility, Uncertainty, and Investment’, NBER Working Paper 3307; later published in the Journal of Economic Literature, 29(3), 1110–1148. doi:10.3386/w3307.
13. OECD, Boston Consulting Group and INSEAD (2025), The Adoption of Artificial Intelligence in Firms: New Evidence for Policymaking. doi:10.1787/f9ef33c3-en.
14. International Organization for Standardization and International Electrotechnical Commission (2023), ‘ISO/IEC 42001:2023, Information technology, Artificial intelligence, Management system’. ISO standard record.