TLDR
Executive briefAvailability can be green while the service is failing
- Define the AI service promise for a named population, workflow, configuration and time window, not for an abstract model.
- Measure eight dimensions covering coverage, outcome, uncertainty, escalation, distribution tails, controls, recovery and change evidence.
- Give every indicator a denominator, ground-truth owner, window, interval, threshold, breach action and bound configuration.
- Keep severe harm, ordinary error, confidence, review capacity, latency, cost, control and change budgets separate because they cannot trade freely.
- Select automation on the risk-coverage frontier with finite human capacity, then move breaches through explicit constraint, degradation, rollback and restoration states.
A customer-advice service meets its uptime commitment and returns quickly. It also produces unsupported recommendations for a material group of customers. The model endpoint is available, the median latency is attractive and the aggregate quality score looks acceptable. Reviewers are correcting the affected cases manually, so the defect is visible only as a growing queue and later complaints. By its conventional service agreement, the system is healthy. By its business effect, it is failing.
Probabilistic systems add variation in meaning, confidence, abstention and outcome to familiar variation in time and load. The same configuration can perform differently across languages, evidence conditions and consequence classes. A threshold can send confidently wrong cases past review. A safety improvement can raise abstention until the human queue becomes the dominant source of harm. A model update can improve average task quality while breaking a non-negotiable control.
The service level must therefore describe a bounded distribution of acceptable behaviour. It connects the model, retrieval, tools, policy, people and operations that deliver the result. It states what is measured, for whom, under which configuration, with what uncertainty and what happens on breach. Commercial credits can compensate for disruption. They cannot make an unsafe service acceptable.
From SRE to AI serviceKeep the mechanics and enlarge the promise
Google's site reliability material defines an indicator as a carefully specified measure, an objective as its target or range, and an agreement as the commitment and consequence around that target. It recommends user-centred indicators, explicit windows, distributions and percentiles because averages conceal tail behaviour.12 Those mechanics remain useful. The object being governed is wider.
A request can succeed technically while the work event fails. The response may be unsupported, incomplete, unfairly routed, unauthorised or impossible to act upon. The model can abstain safely but overwhelm a review function. Ground truth can arrive months later. Generative work can have several acceptable answers and no useful exact match. A service indicator must follow the end-to-end work event and preserve the consequence class, population slice and missing outcome.
Probabilistic Service EnvelopeEight dimensions, one bounded operating promise
Scope and coverage
Name eligible tasks, users, populations, conditions, exclusions and the share handled automatically. Coverage is not a virtue by itself. It defines where the service accepts responsibility.
Outcome quality
Measure task correctness, usefulness, completion, error severity and later business result. Preserve claim-level or adjudicated evidence when many outputs can be acceptable.
Uncertainty and calibration
Test whether stated confidence matches observed correctness. Where appropriate, specify prediction-set coverage and the assumptions behind uncertainty evidence.
Abstention and escalation
Observe safe refusal, deferral, override, escalation precision, queue arrival, disposition and final outcome. Abstention merely moves responsibility unless the next route works.
Time and cost distribution
Track median and tail latency, throughput, timeout, review time, tool consumption and total outcome cost. The unit is the completed work event, not just model response.
Control performance
Test authority, privacy, security, fairness, provenance, audit, reversibility and prohibited-action prevention. A control failure cannot be averaged away by fluent output.
Continuity and recovery
Specify fallback, degraded mode, detection, containment, rollback, replay and safe return to service. Recovery ends when the work and affected people are reconciled.
Change and evidence
Bind model, prompt, retrieval, tools, policy and human roles to a monitoring window. Name drift, notice, revalidation and decision-record triggers.
| Dimension | Misleading shortcut | Operating receipt |
|---|---|---|
| Scope and outcome | One accuracy result for all requests | Eligible denominator, slice results, consequence classes and later reconciliation |
| Uncertainty and review | Confidence threshold plus unlimited escalation | Calibration evidence, review arrival, service time, capacity and disposition |
| Time and cost | Median response time and token charge | End-to-end percentiles, queue age, tools, human correction and total outcome cost |
| Control and recovery | Policy statement and service credit | Blocked prohibited actions, incident evidence, rollback, replay and safe restoration |
| Change | Stable API name | Bound configuration, notice, targeted regression and acceptance decision |
Service Evidence ContractMake each indicator reproducible and governable
Every indicator needs ten fields: user or affected population; decision or work event; numerator, denominator and inclusion rules; error or consequence classes; data source and ground-truth owner; sampling and delayed-outcome method; measurement and comparison window; threshold plus confidence or uncertainty interval; action on breach and accountable owner; and configuration plus evidence version.
The denominator is a governance decision. Excluding cases with missing outcomes can make a service appear better precisely where follow-up is hardest. Counting an abstention as success hides whether the reviewer corrected it, abandoned it or acted too late. Sampling only completed cases can remove the people most harmed by delay. The evidence contract states these missingness rules before results are seen.
Ground truth can be deterministic, adjudicated or delayed. A prohibited transfer can be checked from an external system receipt. A generative recommendation may need claim-level source support and specialist judgement. A customer outcome may arrive after a quarter. Use leading indicators only with an explicit bridge to that later outcome, then reconcile when the outcome arrives. NIST's metrology and TEVV material emphasises evaluation objectives and measurement in the intended context, while its 2026 statistical-model report highlights assumptions and uncertainty in evaluation.589
| Field | Customer-advice example |
|---|---|
| Population and event | Eligible advice requests in three supported languages, from intake through accepted disposition |
| Numerator and denominator | Material unsupported recommendations divided by every eligible completed, escalated or timed-out event |
| Consequence | Severe customer detriment, material correction, ordinary omission and harmless style variation remain separate |
| Truth and missingness | Independent specialist adjudication, customer outcome reconciliation and an explicit unresolved-outcome class |
| Window and interval | Rolling service window with slice-specific uncertainty reported beside observed values |
| Threshold and action | Severe breach suspends the affected route; ordinary burn narrows coverage and opens remediation |
| Version | Named model, prompt, retrieval snapshot, tools, policy, reviewer protocol and evidence release |
Non-Fungible Reliability BudgetDo not let cheap success buy expensive harm
Conventional error budgets help decide when reliability work should outrank feature change.3 A probabilistic service needs separate ledgers because its failures have different consequences and remedies. Track severe consequence, ordinary outcome error, uncalibrated confidence, abstention and reviewer capacity, latency and queue, cost and tool consumption, control failure and evidence gaps, and change plus recovery.
Balances inform release pace but do not exchange freely. A severe prohibited action consumes the stop budget even when every availability and ordinary-quality objective is green. An exhausted reviewer-capacity budget forces lower coverage, different routing or suspension. It cannot be repaired by cheaper inference. A control gap requires a control remedy, not extra correct answers.
| Ledger | Early burn action | Exhaustion action |
|---|---|---|
| Severe consequence or prohibited control | Contain the affected path and investigate immediately | Suspend or roll back regardless of other results |
| Ordinary error or calibration | Watch slices, tune routing and refresh evidence | Constrain eligible work or route uncertain cases |
| Review, latency and queue | Add capacity only if safe and temporary; reduce arrival | Lower automation, split population or degrade service |
| Cost and tool use | Find causal retries, rework and exception drivers | Redesign or narrow rather than hide human cost |
| Change and recovery | Pause expansion and increase targeted regression | Freeze release, roll back or switch configuration |
Risk-Coverage FrontierAutomation stops where consequence and capacity say it should
Selective classification research describes the relationship between the error among handled cases and the share handled, often called the risk-coverage trade-off.17 Confidence must first be calibrated for the intended population. Modern neural networks can be overconfident, and calibration evidence can change after deployment or a slice shift.15 Conformal prediction can provide set-coverage guarantees under stated assumptions, but the guarantee and its conditions must match the service claim.16
Plot observed severe and ordinary error among handled cases against automation coverage. Overlay reviewer arrival, service time, available capacity and total outcome cost. The operating point is not the point with maximum automation. It is the highest coverage that keeps non-negotiable harm, evidence and queue conditions inside the envelope. Comparison methods must avoid selective-classification flaws that make one system appear better merely because thresholds or coverage ranges differ.18
Service-Level State MachineTurn a breach into an operating decision
The state machine begins healthy within the envelope. Rising burn moves it to watch. A likely breach narrows eligibility or automation in constrained state. A material breach activates fallback and mandatory review in degraded state. A severe or unmeasurable condition suspends the service or rolls back the configuration. Restoration follows targeted revalidation, bounded replay and accountable acceptance.
State changes need observable triggers and executable authority. The system that detects the breach should be able to restrict the affected route. The owner who accepts residual risk must be named before an incident. Recovery should reconcile delayed or disputed outcomes, not merely restore the endpoint. OMB M-25-22 emphasises ongoing monitoring, performance requirements, unacceptable behaviour, version standards and rollback in AI acquisition.4 NIST's AI RMF and Playbook connect governance, context, measurement and response across the lifecycle.67
| State | Operating posture | Release and remedy rule |
|---|---|---|
| Healthy | Bounded automation inside every hard gate | Release may continue with routine evidence refresh |
| Watch | Rising burn or uncertainty, no hard breach | Pause expansion, investigate and increase targeted sampling |
| Constrained | Narrower population, task or automation level | Protect capacity while closing the causal gap |
| Degraded | Fallback route and mandatory review | Prioritise affected people, queue safety and recovery evidence |
| Suspended | Stop, rollback or switch configuration | Contain severe breach and preserve evidence; credits are secondary |
| Revalidated | Bounded replay and staged restoration | Return only after causal and neighbouring tests pass |
The Probabilistic SLA
The commercial agreement names reporting access, independent test rights, provider evidence, bound configurations, notice, remediation, rollback, support and termination. It assigns who can sample, adjudicate, inspect slices and retain evidence. The provider cannot define the test data, metric and conclusion alone. A buyer should be able to reproduce the decision or commission independent challenge.
Service credits remain a financial remedy for failure to deliver. The primary remedy for unsafe or materially wrong behaviour is constraint, correction, rollback, affected-person support and safe restoration.
The Canadian Directive on Automated Decision-Making provides a useful public-sector example of impact-scaled testing, monitoring, human intervention, recourse and peer review.11 AISI's evaluation approach and Inspect Evals illustrate explicit task construction, limitations and reusable independent tooling.1213 MLPerf shows how system-level inference measurement can separate scenarios for throughput and latency, although such benchmarks do not define the buyer's outcome envelope.14
Measurement and managementDesign a promise that survives contact with work
Start with the work event, not the easiest telemetry
Measure from an eligible request entering the workflow to a final disposition that matters to the user.
A server response is an intermediate event. The completed event may include retrieval, model reasoning, tool calls, a reviewer decision, an external-system effect and later reconciliation. Give the event a stable identity so retries, deferrals and replays do not become extra successes or disappear from the denominator. When a task is abandoned, timed out or unresolved, retain that state. Excluding it would reward the service for losing difficult work.
Separate measurement for diagnosis from measurement for commitment. Token counts, logits, retrieval hit rates and tool latency help engineers locate causes. Completion, supported claims, prohibited effects, reviewer delay and customer outcomes describe the service people receive. Both sets matter, but only the second can define the business promise. Link them through a trace so an envelope breach can be attributed to model, data, tool, policy, human capacity or downstream operation without confusing internal activity with achieved service.
Preserve slices and consequence before aggregation
A population slice belongs in the contract when its context can change performance or harm: language, product, geography, accessibility need, evidence quality, workflow route or consequence class. Do not create endless dashboards for every available attribute. Choose slices from the service claim, causal understanding, affected-party evidence and observed incidents. Report the eligible count, missing outcomes and interval for each material slice. When samples are too small for a stable rate, use targeted challenge cases, qualitative review and a conservative operating boundary instead of manufacturing precision.
Consequence classes need observable definitions. A harmless stylistic difference, a correctable omission, a material unsupported recommendation and a prohibited external action should not share a numerator. Domain owners define what changes the user's position, what can be remedied and what cannot be tolerated. The operating system then attaches a response to each class. This keeps hard controls enforceable while allowing ordinary variation and learning where perfection would be unnecessary or impossible.
Treat automated judges as measurement instruments
An automated evaluator can support scale, consistency and rapid regression, but it is not self-validating. Bind its model, prompt, rubric, reference context and decision threshold. Compare it with expert adjudication on the intended population, inspect disagreement by slice and recheck it after changes. Use deterministic checks for claims that can be verified directly, such as schema, citation presence, authority or external postcondition. When judge drift could change the reported service level, the judge belongs inside the bound evidence configuration.
Human adjudication has uncertainty too. Define assessor qualifications, information access, rubric, repeat review and a method for resolving consequential disagreement. Do not turn consensus into truth without explaining the decision rule. A service can report a range or unresolved category where judgement legitimately differs. The evidence contract becomes stronger by exposing ambiguity, not by forcing every complex output into a binary label.
Plan reviewer capacity as a queue, not a promise
Forecast review arrival from workload, coverage, confidence and abstention. Measure service time by case class, not only an overall average, because the hardest cases often take longest. Include working hours, specialist availability, rework, interruption and surge. Queue age should trigger scope reduction before reviewers become overloaded. Adding people can be a temporary remedy, but a permanently rising arrival rate signals that the automation point or workflow design is wrong.
Reviewer quality also changes with volume and presentation. If the interface hides evidence, compresses uncertainty or encourages automatic approval, formal human involvement may add little control. Observe override, agreement, correction, time and downstream outcome without assuming that agreement proves the model was right. Protect reviewers from being used as an invisible liability transfer. Their authority, information and capacity are part of the supplied service.
Govern threshold changes as service changes
A threshold is policy encoded as a number. Lowering it can increase coverage and expose more people to error. Raising it can reduce handled-case risk while flooding review. Changing a quality rubric, denominator or missing-outcome rule can move the reported result without changing the service. Require a decision record that states the evidence, affected population, predicted queue and cost effect, owner, expiry and rollback condition. Silent metric substitution is a configuration change and should reopen acceptance.
Thresholds should mature with evidence. Early service may use conservative coverage, wider uncertainty and more adjudication. Later evidence can justify a different point, but improvement is not automatic. New populations, changed policies or different consequence can require a narrower envelope again. The discipline is not permanent numbers. It is a stable method for changing them with evidence while keeping prohibited behaviours and accountable authority visible.
Ten worked casesCharge the right budget and change the state
1. Green uptime, unsupported advice
The service promise covers eligible customer recommendations, evidence support, completion, latency and prohibited actions. The evidence contract includes all completed, escalated and abandoned cases, with independent claim adjudication by population. A material cohort shows unsupported recommendations despite healthy uptime. Charge ordinary and severe outcome ledgers, not availability. Move from healthy to constrained, route the uncertain tail to review, and lower coverage until the affected slice is measured. Decision: split the service by population and consequence rather than accepting the aggregate.
2. One language slice carries severe error
Aggregate correctness is strong, but one supported language has more severe mistakes and escalations. The envelope promised that language, so excluding it after observing results would corrupt the denominator. The evidence contract reports adjudicator agreement, missing outcomes and the exact configuration. Charge the severe, ordinary and review ledgers for that slice. Place separate operating points on the frontier. The well-performing languages can remain healthy while the affected language becomes constrained or degraded. Decision: split the service by population and consequence, with a lower automation point for the affected language.
3. Confidence is overstated
A classifier sends high-confidence cases past review, yet calibration analysis shows confidently wrong decisions in the material class. The task score has not fallen, but the uncertainty dimension and confidence budget have breached. The evidence contract binds the calibration sample, population, interval and threshold. Recalibration and selective evaluation must precede release, and the state moves to watch or constrained. Decision: route the uncertain tail to human review using a tested threshold, while monitoring the resulting queue rather than treating confidence as truth.
4. Safer abstention overwhelms review
Raising abstention reduces error among handled cases. It also sends work to a team whose arrival rate exceeds service capacity. Queue age grows and time-sensitive outcomes deteriorate. The service envelope joins handled-case risk, escalation precision, review time and final disposition. Charge the review, latency and outcome ledgers. The frontier shows that maximum apparent safety at the model boundary is not minimum harm end to end. Decision: release intentionally below maximum automation only at a point the reviewers can sustain, then narrow eligible scope when arrival rises.
5. Median latency hides a damaging retry tail
Median model response is acceptable, while tool retries create a long end-to-end tail and backlog. The evidence contract measures the complete work event, timeout semantics, percentile distribution and queue age. Charge latency, change and cost budgets because retries consume tools and human reconciliation. The service moves from watch to constrained as the tail approaches the limit. Decision: roll back or switch the model configuration if retry ownership and postcondition checks cannot restore the tail.
6. Token cost is green while correction cost rises
Inference remains within budget, yet reviewers correct more cases, exceptions take longer and audit evidence is reconstructed manually. The total outcome denominator includes model, tool, review, rework, incident and audit work. Charge the cost and review ledgers rather than celebrating cheap tokens. Quality can remain statistically stable while usefulness falls. Decision: constrain release and redesign the handoff, then return within the envelope only when observed total outcome cost fits.
7. Multiple valid answers defeat exact match
A drafting task has several defensible outputs. A single golden answer would reward imitation and penalise legitimate variation. The evidence contract uses claim-level source support, prohibited assertions, completeness, usefulness and specialist adjudication with assessor uncertainty. Deterministic controls block disallowed actions. Charge ordinary outcome and evidence-gap budgets, not a fabricated exact-match failure. Decision: release within the bounded service envelope when the claim controls and adjudicated evidence pass.
8. Outcome truth arrives months later
The service produces a recommendation today and the business outcome appears after a quarter. The evidence contract defines leading indicators, a representative sample, unresolved-outcome handling, the later reconciliation date and the ground-truth owner. The current state stays watch during evidence lag. Coverage expands only through governed release cohorts. Decision: route uncertain cases to review and keep a bounded release; stop if no credible bridge between the proxy and later outcome can be measured.
9. Average quality rises but a control fails
A provider update improves average usefulness but allows an action outside approved authority. The control ledger has a hard breach even though ordinary quality and latency are green. Because the evidence contract binds model, prompt, tools and policy, the provider cannot carry forward the old acceptance. The state moves directly to suspended. Decision: roll back or switch the model configuration, preserve the breach evidence and revalidate neighbouring controls before restoration.
10. One prohibited action consumes the stop budget
A service performs thousands of ordinary cases acceptably and remains available. One case executes a prohibited external commitment. The severe and control budgets do not trade with the volume of correct work. Containment traces the affected transaction, authority and downstream effect. The service stops or the action path is disabled, even if a financial credit is available. Decision: suspend the service path, reconcile the affected person and restore only after causal repair and replay evidence.
The strongest rival
Detailed AI service levels can create false precision, optimise proxies, invite gaming and impose measurement cost before ground truth exists. Expert labels can disagree, several answers can be useful and rigid thresholds can reward conservative abstention. A team may spend more effort measuring than improving the service.
The answer is typed evidence and progressive maturity. Use deterministic tests for prohibited actions, statistical intervals where sampling supports them, adjudicated cases for complex quality, proxies only with a stated bridge, and qualitative incident evidence when a percentage would mislead. Begin with a small measurable envelope. Record uncertainty. Improve thresholds through governed review, never through silent metric substitution.
Interactive instrumentProbabilistic Service-Level Board
Select the conditions for one bounded service. The board returns an operating posture, not a universal reliability score. Its decision preserves human capacity, hard gates and configuration evidence.
Operating disciplineLet the promise improve without moving silently
At service formation, the business owner names the outcome, affected population, consequence and exclusions. Product and evaluation define the envelope and evidence contracts. Operations and human-service owners validate tails, queue capacity, fallback and recovery. Procurement gives the buyer reporting, independent test, notice, remediation, rollback and termination rights. Finance includes tools, correction, adjudication, audit and incident work in total outcome cost.
During operation, monitors enforce this promise rather than invent it. This keeps Paper 126 distinct from Paper 095, which selects behavioural sentinels, and Paper 075, which builds the evaluation product specification. Here the selected measures become persistent commitments, budget ledgers, breach states and remedies. Paper 121 defines what capability was bought; this paper says when that capability remains acceptable in service.
At each material model, prompt, retrieval, tool, policy, population or reviewer change, bind a new evidence version. Reuse evidence only where the causal claim still holds. Run targeted regression and neighbouring controls, then record whether the service returns healthy, remains constrained or stops. An API alias is not a configuration record.
A credible service level makes uncertainty operational. It does not promise perfect prediction or bury variation inside a composite score. It states what the service may handle, what evidence supports that boundary, how much ordinary variation can be tolerated, what can never be traded and who changes the state when reality moves.
Evidence spineSources checked 3 September 2026
- Google, Site Reliability Engineering: Service Level Objectives.
- Google, The Site Reliability Workbook: Implementing SLOs.
- Google, The Site Reliability Workbook: Error Budget Policy.
- US Office of Management and Budget, M-25-22.
- NIST, AI Metrology Center.
- NIST, Artificial Intelligence Risk Management Framework.
- NIST, AI RMF Playbook.
- NIST, The TEVV-Athlon Framework for Evaluating AI Systems.
- NIST, Expanding the AI Evaluation Toolbox with Statistical Models.
- NIST, Generative Artificial Intelligence Profile.
- Treasury Board of Canada Secretariat, Directive on Automated Decision-Making.
- UK AI Security Institute, Our Approach to Evaluations.
- UK AI Security Institute, Inspect Evals.
- MLCommons, MLPerf Inference: Datacenter.
- Guo et al., On Calibration of Modern Neural Networks.
- Angelopoulos and Bates, A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification.
- Geifman and El-Yaniv, Selective Classification for Deep Neural Networks.
- Ding et al., Overcoming Common Flaws in the Evaluation of Selective Classification Systems.
About this paperReliability is a governed distribution
The Probabilistic Service Envelope, Service Evidence Contract, Non-Fungible Reliability Budget, Risk-Coverage Frontier, Service-Level State Machine and Probabilistic Service-Level Board are this article's synthesis. They extend service-level mechanics to probabilistic outcome, uncertainty, human capacity, control and change without treating uptime, average accuracy, abstention, benchmarks, credits or human review as complete reliability proof.