The queue trains the answer

The intuitive case for human approval is simple: machines make mistakes, people can notice context, so consequential actions should stop until a person agrees. The hidden assumption is that each approval request receives an independent act of judgement. In a real queue, decisions are coupled through time.

This capacity problem is not new. Bainbridge’s classic analysis of automation observed that machines often remove routine work while leaving people responsible for abnormal conditions, precisely when their skill and situational understanding have been least exercised.[1] Research on out-of-the-loop performance similarly found that higher automation can reduce situation awareness and slow recovery when the automated aid fails.[2] Approval queues add a modern twist: the human is technically inside the transaction path yet cognitively outside the causal work that produced the proposal.

flowchart LR
  A["The one hundred and<br/>seventeenth approval"] --> B["The queue trains the answer"]
  B --> C["Why the human becomes the<br/>weakest sensor"]
  C --> D["Replace the universal gate<br/>with selective control"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#007D68,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Why the human becomes the weakest sensor

An approval queue does more than consume attention. It shapes what evidence is sampled, which actions feel normal and how costly dissent becomes. Three forces combine: repeated reliability, low information value and organisational asymmetry.

Reliability creates trust, then hides the moment trust is wrong

Automation bias includes commission errors, in which people follow incorrect advice, and omission errors, in which they fail to act because the system did not signal a problem. Parasuraman and Manzey’s review argues that automation complacency and automation bias overlap through attentional allocation. They occur in novices and experts, under multiple-task load, and are not removed by simple instruction or practice.[4] Skitka and colleagues likewise showed that decision aids can bias human judgement when the aid is imperfect.[5]

This does not mean humans irrationally worship machines. Often the locally rational strategy is to rely. If the proposal has been correct for the last 600 cases, independently reconstructing case 601 is expensive. The reviewer learns the empirical regularity that effort rarely changes the result. A high-reliability system can therefore make its rare failure harder to detect by teaching the reviewer that checking is unproductive.

Uninformative work consumes the same channel as important work

Clinical alert systems offer a close analogue. Ancker and colleagues distinguished simple desensitisation from cognitive overload and work complexity. In their data, clinicians were less likely to accept reminders as more reminders arrived in an encounter and as repeated reminders increased. They did not find a simple general decline over time for newly introduced alerts.[8] That nuance matters. “Fatigue” is not always a smooth biological depletion. It can be an adaptation to low-value interruptions.

An approval request is also an interruption. If it repeats information the system already knows, asks the reviewer to confirm a deterministic policy result or supplies no actionable alternative, then it uses human attention without adding human discrimination. The correct design response is not to make the card brighter. It is to remove low-information requests from the queue.

The cost of saying no is usually higher than the cost of saying yes

Approving often closes the card. Rejecting may require a reason, supporting evidence, a call to the requester, a manager’s attention and ownership of any delay. The harm from an incorrect approval may appear later and be distributed across teams. The cost of an escalation appears now and belongs to the reviewer. This asymmetry moves the criterion even when the interface has no explicit default.

Organisations sometimes respond by mandating longer comments for approvals too. That creates documentation, but it can also generate templated text. A reason field is useful only when its content can change the action, support an audit or reveal misunderstanding. Friction should be placed where deliberation has decision value, not spread evenly as compliance theatre.

The audit trail can optimise the wrong behaviour

Approval systems are often managed through queue age, completion time and the proportion of cases resolved before a service deadline. These measures are easy to count. The counterfactual that matters is harder: which harmful action would have occurred without this reviewer? When only throughput is visible, an unresolved case becomes an immediate performance failure while an unsafe approval may remain invisible until much later. The least costly way to appear compliant is therefore to keep the queue moving.

Compare two teams. Team Fast approves 99.8 per cent of requests and rarely delays execution. Team Discriminating escalates 3 per cent, requests missing evidence and sometimes breaches the queue target. A conventional dashboard praises Team Fast. Yet without known challenge cases, independent readback or downstream outcome sampling, the organisation cannot tell whether Team Fast received cleaner work, exercised better judgement or simply stopped contesting the generator. An approval rate is a health signal only when reviewers had a realistic opportunity to disagree and the system can show that material disagreement changes outcomes.

Ceremonial approval also creates what we might call approval debt. It produces false assurance for auditors, weakens the reviewer’s practised ability to reconstruct unusual cases and may pollute learning data when accepted actions are later treated as verified labels. In that feedback loop, habitual assent makes the model and reviewer appear increasingly aligned while a shared blind spot survives. This is a systems inference, not a claim that every approval system retrains on reviewer decisions. It identifies a failure mode to test.

The corrective is to pair throughput measures with discrimination evidence: catch rate on seeded hazards, false-escalation cost, queue age by risk class, review-time distributions, use of independent evidence, reversals after verified readback and reviewer reports that a decision request was futile. No single measure establishes good oversight. Together they stop the audit trail from confusing activity with intervention.

↔ Scroll within the table to inspect all columns.

Replace the universal gate with selective control

The remedy is not to remove humans from consequential decisions. It is to stop spending human judgement on decisions that do not require it. Oversight should separate permission, evidence production, judgement, execution and recovery.

Authority is not the same thing as judgement

A system can enforce identity, role, purpose, limits, idempotency and policy deterministically. Those controls decide whether an action is permitted to proceed to the next stage. A model can propose an action and summarise evidence. A human can resolve normative ambiguity, conflicting evidence or a high-impact exception. Parasuraman, Sheridan and Wickens distinguish information acquisition, information analysis, decision selection and action implementation as separate automation functions; assigning one level of automation to the whole chain hides consequential differences.[3] Conflating these functions creates a queue in which humans repeatedly confirm facts that software can check more reliably.

This distinction matters for applied AI. Keep intent, identity, world state, permitted context, model proposal, authority, typed action, evidence, verified outcome and recovery separate. The human should not be asked to infer authority from a persuasive narrative. The authority kernel should supply a clear policy decision. The human should decide only the residual question that policy and deterministic evidence cannot settle.

Route by the decision’s structure

Two variables are especially useful before adding more elaborate scoring: reversibility and unresolved uncertainty. A low-cost, reversible action with complete evidence rarely merits synchronous approval. An irreversible action with disputed identity or ambiguous harm does. Expected impact adds a third dimension, but the first two already expose why a universal gate is incoherent.

Design the review packet for disagreement

A review packet should not be a polished argument for approval. It should make disagreement cheap and diagnostic. At minimum, show the requested effect, the person or entity affected, why the action is permitted, what evidence is independent of the model, what evidence conflicts, what would make the action unsafe, whether it can be reversed, and what happens if the reviewer abstains.

Do not force the reviewer to read an entire activity history when a difference view can show what changed. Do not bury the source record under a generated summary. Do not use colour as the only risk signal. Do not expose hidden model reasoning as if it were causal evidence. A concise rationale may aid navigation, but the reviewer needs source-linked facts and a rival explanation.

Where dual approval is justified

Dual control is strongest when collusion, individual error or value conflict has material consequences and when the two reviewers can inspect sufficiently independent evidence. It is weak when reviewer B merely confirms reviewer A’s summary, when both face the same deadline, or when rejecting creates career risk. The consolidated EU AI Act requires high-risk AI oversight measures that enable people to understand limitations, remain aware of automation bias, interpret outputs and disregard, override or stop the system.[13] Those capabilities are necessary. They do not by themselves guarantee that the surrounding workload makes their use feasible.

stateDiagram-v2
  [*] --> Claim
  state "Keep the reviewer, interface<br/>and cases constant; change<br/>only volume" as Claim
  state "Criterion collapse, not mere<br/>tiredness" as Test
  state "Keep the cards identical;<br/>change only how often a real<br/>defect appears" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Test the work system, not the button

An oversight design is credible only if it can be evaluated under realistic volume, rarity, interruption and failure. “A human can override” is a feature claim. “Humans detect and contain specified hazards under operating load” is a system claim.

The minimum arithmetic of review capacity

Let N be the number of routed requests in a shift, T the available review seconds and r the seconds needed for a competent review of one case. The deliberation coverage ratio is:

D = T / (N × r)

If D is below one, the system cannot complete the declared review procedure without overtime, queue growth, bypass or compressed inspection. This equation is not a psychological law. It is an accounting identity. It exposes impossible operating claims before subtler human-factors effects are considered.

Test failure modes that make approval look stronger than it is

Normal-case user acceptance testing will almost always show that approvers can click approve and reject. It rarely shows whether they notice a defect after 300 ordinary requests, whether a persuasive explanation suppresses independent checking, whether a queue deadline changes the criterion, or whether the reject path actually prevents execution.

Use failure injection. Seed matched cases with stale evidence, changed permissions, duplicated actions, hidden conflicts, model disagreement and post-approval execution errors. Vary prevalence and queue length. Record not just the final click but which evidence was opened, how long the review took, whether the reviewer requested more information and whether the effect was contained. Do not secretly punish individuals for missing planted cases. The object of study is the work system.

Measure oversight as operating evidence

A dashboard that shows 100 per cent human approval coverage can conceal total control failure. At minimum, track the measures below by action class, risk tier, reviewer cohort and time window. Use planted challenges sparingly and ethically to estimate detection where natural hazards are too rare. Pair them with real incident review so the test does not become a game.

↔ Scroll within the table to inspect all columns.

The NIST AI Risk Management Framework treats AI risk management as a lifecycle activity across governance, mapping, measurement and management rather than a single control.[11] Its generative AI profile similarly emphasises governance, testing and incident disclosure.[12] ISO 9241-210 places human-centred design across the life cycle of interactive systems.[14] A recent cross-disciplinary framework likewise treats effective human oversight as an architecture of roles and processes, not an isolated moment of human assent.[15] The practical consequence is straightforward: human oversight must be designed, staffed, tested and revised as a socio-technical subsystem.

sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: The one hundred and seventeenth approval
  M->>E: Keep the reviewer, interface and cases constant; change only volume
  E-->>M: compare rival explanations
  E->>D: Protect the judgement, remove the ceremonial click
  D-->>Q: revise scope or act

Protect the judgement, remove the ceremonial click

The conventional governance question is: Where should we put a human approval? That question produces gates, queues and signatures. Replace it with a harder question: Which consequential uncertainty requires human judgement, and what work design preserves the reviewer’s ability and authority to intervene?

This shift changes architecture. Low-risk, reversible and policy-complete actions move through deterministic controls with sampling and readback. Ambiguous or high-impact actions receive protected review time, source-linked contrast and a real veto. Irreversible actions with weak evidence may be refused or redesigned rather than decorated with more approvers. Dual control is reserved for cases where the second path is genuinely independent.

It also changes management. Review capacity becomes a safety budget, not free labour. Queue age, end-of-shift acceleration and intervention yield become governance signals. A growing business volume cannot silently dilute the same reviewers across ten times more actions. When volume rises, the organisation must improve routing, add qualified capacity, narrow autonomy or accept fewer actions.

Finally, it changes accountability. A human click is no longer treated as proof that a human controlled the outcome. The assurance claim must show that the reviewer received decision-sufficient evidence, had realistic time, could disagree without procedural penalty, exercised an effective stop or hold, and that the executed result was independently verified.

The human factors of oversight do not argue against human responsibility. They make responsibility operational. A responsible system does not ask a person to bless an impossible workload and then cite the blessing as safety evidence. It arranges authority, evidence, time and recovery so that saying no remains a real act.