Rajesh Mahapatra · Applied AI strategyWorkforce and capability

Incentives That Do Not Reward Blind Use

A governing architecture for making AI-assisted judgement economically honest: reward evidence, appropriate challenge and verified outcomes, while keeping use, modification, override, escalation, abstention and manual work neutral in themselves.

TLDR

  1. The performance unit is a bounded decision opportunity, not a login, prompt, accepted recommendation or override.
  2. Equivalent evidence, consequence and outcomes should earn equivalent treatment regardless of the chosen reliance mode.
  3. Every target needs an intended-action trace and a rational-gaming trace before money, rank or status is attached.
  4. Human review is real only when time, independent evidence, authority, protection and a viable fallback make disagreement usable.
  5. Defer material reward until downstream rework, harm and affected-party outcomes can be seen, or remove the incentive.

The acceptance bonus that made service worse

A service centre promised a monthly bonus when agents accepted an AI answer often enough. The dashboard climbed. Agents stopped opening source records, awkward cases were pushed into a manual queue, and repeat contacts appeared after the reward period. Leaders had paid for agreement while hoping for resolution. The model might have been useful, but the incentive made its usefulness impossible to distinguish from obedience.

Remove the bonus and hold the case, evidence, review effort and downstream result constant. Should an agent who corrects the answer earn less than one who accepts it? Should a careful manual resolution lose to a fast AI-assisted mistake? If the reward changes only because of the route, accountability is theatre. The organisation has priced independent judgement out of the job.

Home · Writing · Research Essays

Research Essays

Incentives That Do Not Reward Blind Use

Rajesh Ranjan Mahapatra

A reliance-neutral architecture for rewarding decision quality, challenge and downstream outcomes without turning AI use into obedience.

Start reading
Research EssaysEssay

© 2026 Rajesh Ranjan Mahapatra. 22 min read

Incentives That Do Not Reward Blind UseAll Research Essays

01 · The governing principleMake reliance neutral before measuring performance

Reliance neutrality means that, with case, evidence, review effort, consequence, downstream result and accountable scope held constant, reward does not change merely because a person accepted, modified, overrode, escalated, abstained from or bypassed AI.

This is not indifference to performance. It is the condition under which performance can be seen. An action earns credit because it is well evidenced, authorised and produces the right result for the affected party. A correct acceptance may be excellent. So may a justified override. An unnecessary override can be wasteful, just as an unexamined acceptance can be dangerous. Reliance mode is descriptive evidence about how work was completed, not an achievement by itself.

The unit of analysis is a decision opportunity: a defined work event in which AI advice may legitimately be considered under a particular capability boundary and consequence tier. That unit contains the case, evidence available at the time, permitted actions, chosen action, handoff, outcome and accountable owner. Counting seats or prompts mixes people who never encountered an eligible case with people handling difficult exceptions. Counting all transactions treats prohibited and manual-only work as failed adoption. Both denominators invite false comparisons.

Steven Kerr's classic warning about rewarding one behaviour while hoping for another explains the organisational mechanism. Holmström and Milgrom's multitask analysis adds a sharper point: strong incentives on what is easy to measure can draw effort away from important, weakly measured work. Applied here, an acceptance target prices the click but leaves evidence checking, challenge, explanation and downstream repair outside the reward. The score changes the allocation of attention before it changes any formal policy.

The leaderboard bends judgement An acceptance leaderboard pulls six decision routes towards accept, then dissolves into a level field where equal-quality routes receive equal treatment. When the target prices agreement ACCEPTANCE RANK bonus · status · queue ACCEPT NEUTRAL FIELD accept modify override escalate abstain manual Remove route-based payoff; reward evidence and outcome
Figure 1. Illustrative mechanism. A use target can bend every permitted route towards acceptance; neutrality restores the choice set without declaring any route superior.

Thought experiment: the identical result

Two underwriters receive the same difficult case and the same model output. Both inspect identical evidence, document the same uncertainty and reach the same sound decision. One modifies the recommendation; the other completes the calculation manually. A use bonus pays only the first. Nothing about value or care differs, yet the reward manufactures a preferred route. That is a failed counterfactual.

Negative control: authorised automation

A deterministic, validated calculation is authorised to run automatically for a bounded case class. No human agreement is required, so there is no reliance choice to reward. Measure service, exceptions and control performance. Reliance neutrality does not insert discretionary review where automation authority has already been properly granted.

The distinction between required process and required agreement matters. An organisation may require an eligible case to be logged, specified evidence to be assembled, or a validation check to be completed. Those are auditable duties. It should not require acceptance of the recommendation unless the system itself is authorised to decide at that consequence level. Obligation to consider evidence is not obligation to obey advice.

Before approval, run one deliberately awkward case in which the model is wrong but confident, one in which the model is correct and the person is doubtful, and one in which neither route has enough evidence to act. The incentive should support correction, justified acceptance and abstention respectively. If the same behavioural target pushes all three towards one answer, the target is measuring allegiance rather than performance. Repeat the exercise under peak workload because a review route that works only in a quiet demonstration is not an operating control.

02 · Original synthesisThe Reliance-Neutral Incentive Architecture

The architecture follows an opportunity from eligibility to reward correction. It keeps the model's output, the person's action, the work system and the affected-party result separate long enough to assign responsibility fairly.

The first four layers describe the conditions shaping choice. The next two capture the action and its consequences. The last two decide whether reward survives later evidence and whether the arrangement may continue. This sequence blocks the common shortcut of starting with an adoption percentage and inventing a performance story after it moves.

Reliance-Neutral Incentive Architecture Eight connected layers carry a decision opportunity through action, evidence, perceived payoff, handoff, outcome, reward correction and learning. Eight layers, one accountable opportunity DECISION OPPORTUNITY ACTION SET EVIDENCE & REVIEW PERCEIVED PAYOFF DECISION & HANDOFF OUTCOME & EFFECT REWARD & CORRECTION LEARN & REAUTHORISE Learning can change eligibility, boundaries, targets and ownership
Figure 2. This article's synthesis. The perceived payoff sits before the choice because formal and informal incentives alter conduct even when policy says the human is responsible.
Layer Decision question Evidence Characteristic hazard Owner Control
Decision opportunity Which cases may enter? Eligibility, exclusions, case mix, consequence False denominator Business owner Published opportunity rules
Action set What may a person do? Accept, modify, override, abstain, escalate, defer, manual Non-use becomes failure Process owner Protected legitimate routes
Evidence and review Can advice be challenged? Independent facts, limits, time, competence Ceremonial confirmation Domain and product owners Review independence test
Perceived payoff What does each route cost? Pay, rank, queue, status, workload, discipline Acceptance becomes rational Workforce owner Full payoff inventory
Decision and handoff Who acts and receives it? Reason, authority, exception, recipient Risk exported downstream Operations owner Traceable handoff
Outcome and consequence What changed for whom? Quality, safety, delay, rework, value, distribution Local speed masks harm Outcome owner End-to-end measures
Reward and correction When is credit earned? Attribution, adjustment, deferral, appeal Premature bonus Reward owner Deferral and reversal
Learning and reauthorisation May the incentive remain? Gaming, drift, incidents, voice, changed context Metric becomes permanent Independent assurance Expiry and fresh decision

The accompanying Reliance-Neutral Incentive Contract records the opportunity and exclusions; operating boundary and consequence; eligible actions; evidence required; immediate and downstream measures; case-mix, workload and uncertainty adjustments; every financial, operational, status and social payoff; prohibited measures and inferences; business, workforce, product, risk and assurance owners; worker notice, representation, appeal and protected reporting; reward timing, reversal and recovery; and expiry or change triggers. It states in plain words that AI use, acceptance, override and abstention are not performance outcomes by themselves.

03 · Denominator before rateBuild the Appropriate Reliance Opportunity Set

An acceptance rate has no stable meaning until every case has been classified. A low rate may be correct when few eligible opportunities occur. A high rate may be unsafe when cases lie outside capability or people cannot challenge.

The Appropriate Reliance Opportunity Set is this article's eight-class gate. Required evidence assembly means the tool can structure facts while its conclusion remains contestable. Permitted advice allows consideration inside a specified boundary. Optional use keeps manual completion equally legitimate. Required human confirmation demands independent checking of named evidence or consequence. Override or escalation eligibility opens a protected challenge route for novelty, uncertainty, missing facts, protected characteristics or high consequence. Abstain eligibility recognises that withholding action can be correct. Manual-only cases prohibit AI reliance. Excluded or unobservable cases never enter the rate.

The denominator is the number of eligible decision opportunities in the relevant class. It is not headcount, licences, prompts or all transactions. Exclusions, classification changes and case mix must travel with any comparison. A team serving unusual or consequential cases should not look resistant beside a team processing routine cases. Nor should managers earn a better score by routing difficult work away from the model.

Appropriate Reliance Opportunity Set Cases pass through eight coloured classification channels before joining valid denominators, protected challenge routes or exclusions. Classify the opportunity before calculating a rate Required evidence assembly Permitted advice Optional use Human confirmation Override or escalate Abstain eligible MANUAL-ONLY EXCLUDED VALID CLASS RATE Publish exclusions and changes; never smuggle them into the denominator
Figure 3. This article's synthesis. Classification protects both appropriate use and appropriate non-use while separating prohibited and unobservable cases.
Case class Permitted actions Denominator treatment Evidence duty Reward rule Escalation
Evidence assembly Use tool, correct packet, reject recommendation Count completed eligible packets Source completeness and provenance Reward sound evidence, not agreement Missing or conflicting records
Permitted advice Accept, modify, override, defer Count comparable in-boundary cases Review capability and limits Outcome-neutral across modes Novelty or material uncertainty
Optional use AI or manual Include both routes Equivalent service proof No route premium Workflow or access barrier
Human confirmation Confirm, challenge, abstain Count confirmed opportunities Specified independent facts Reward verification quality Evidence cannot be established
Override or escalate Override, escalate, second review Preserve in relevant case class Reason and contemporaneous facts No disagreement penalty Protected consequential route
Abstain eligible Withhold recommendation or action Count as legitimate disposition Uncertainty and missing evidence No completion penalty Named owner decides next step
Manual-only Manual action only Exclude from AI reliance rates Prohibition and control basis Measure manual outcome AI contact becomes incident
Excluded or unobservable No performance inference Remove and disclose Reason for exclusion No reward calculation Repair observation design

Thought experiment: the improving rate

An underwriting team begins sending simple cases through AI and difficult cases to a manual queue. The AI success rate improves each month. Nothing about model quality changes; only the denominator does. If leaders reward that rate, a rational team protects the score by exporting complexity. Recombine the queues, classify opportunity and adjust for case mix. The apparent improvement disappears while the downstream workload becomes visible. This is not worker dishonesty alone. The metric taught the routing behaviour.

04 · Test the economics of choiceTrace the intended route and the rational gaming route

The Incentive-to-Action Chain runs from target to perceived payoff, action choice, review effort, handoff, downstream action, outcome and eventual correction. Trace it twice before approving a measure.

The first trace states what leaders intend. An AI-assisted throughput target might be expected to increase familiarity, remove delay and improve service. The second asks how a rational person can maximise the measure while weakening the result. The worker may spend less time checking, select easy cases, suppress uncertainty, move complex work to another queue or postpone a report until the bonus is secured. Include pay, promotion, manager approval, public rank, schedule preference, queue pressure, workload relief and vendor fees. Informal status can bend choice as strongly as money.

Incentive-to-Action Chain An intended route and a gaming route travel from target through action and outcome, with hidden rework and harm returning to correct the reward. Run the chain twice TARGET PAYOFF ACTION REVIEW HANDOFF OUTCOME SCORE SHORTCUT CHERRY-PICK COMPRESS EXPORT HIDE Delayed rework, harm and reporting return to the reward INTENDED BEHAVIOUR RATIONAL GAMING
Figure 4. The gaming trace is not an accusation. It is a design test asking what the measure makes locally rational under real workload and career pressure.

The Blind-Use Hazard Test asks twelve direct questions. Does the measure favour AI use over an equally good manual or abstain decision? Can someone improve the score by checking less? Does it punish uncertainty or escalation? Does it assign responsibility without time, evidence or authority? Can easy cases be selected? Can local speed export rework? Does zero-incident pay suppress reports? Does a low-override goal equate disagreement with failure? Can use move beyond the validated boundary? Does a vendor benefit from volume rather than outcome? Do results appear after the reward period? Would workers or affected parties challenge the complete chain? Any yes demands redesign, separation or removal. Training cannot repair a payoff that makes unsafe conduct rational.

Blind-Use Hazard Test Six hazard detectors surround an incentive and send positive findings to redesign, separation or removal. A single positive hazard blocks the shortcut PROPOSED INCENTIVE ACCEPTANCE BIAS REVIEW COMPRESSION ESCALATION PENALTY CHERRY- PICKING REPORT SUPPRESSION VENDOR CONFLICT
Figure 5. This article's synthesis. The full twelve-question test expands these six families to include denominator, authority, timing, affected-party and downstream risks.
Proposed measure Rational gaming response Downstream effect Disconfirming evidence Repair Owner
Minimum acceptance Approve borderline advice Error and appeal increase Correctness by agreement state Remove target; sample decisions Business and risk
AI-assisted throughput Route complex work elsewhere Manual queue and delay grow Unified case-mix view Rebuild denominator Operations
Maximum override Hide disagreement as modification Weak model signal disappears Seeded-error detection Assess evidence quality Product and assurance
Zero incidents Delay or recode reports Harm persists without learning Near miss and downstream evidence Protect candour; remove zero goal Safety and workforce
Vendor active seats Expand access and prompts Cost and exposure rise Eligible outcomes and harm Outcome-linked terms Commercial owner
Raw speed bonus Compress review and hand off repair Repeat work shifts teams End-to-end resolution time Include rework and consequence Service owner

A Reliance-Neutral Counterfactual Test follows. Hold the case, evidence quality, review effort, consequence, downstream result and responsible scope constant; compare reward across accept, modify, override, escalate, abstain and manual handling. A difference caused only by route fails neutrality. Then reverse the test: hold the route constant and vary evidence discipline, decision quality, downstream outcome and candid reporting. A sound arrangement should respond to those differences. It may reward a careful acceptance over a careless acceptance, and a sound override over an arbitrary one, without treating acceptance or override as inherently good.

05 · Human oversight with economic room to actTest independence, not reviewer presence

A person cannot provide meaningful oversight when every practical condition favours agreement. Review independence requires time, independent evidence, real authority, protected challenge, consequential escalation and a functioning manual fallback.

The Review Independence Test examines ten conditions. Review time must remain feasible at realistic volume. The person needs facts beyond the model's output and a clear view of limitations. Override, abstention and escalation authority must work in the system, not merely appear in policy. Pay, rank, queue and manager expectations cannot prefer acceptance. Overrides must be judged for evidence quality rather than counted as failure. High-consequence disagreement needs a second route. Recurring weakness can be reported without retaliation. The adoption owner cannot be the sole judge of challenge. Seeded errors and difficult cases must test performance. Manual fallback must retain people, access and capacity.

Consider reviewers given ten seconds to confirm a complex recommendation while their throughput is ranked. Adding a free-text explanation does not produce reflection; it creates fast ritual typing. If an explanation is required only for disagreement, the interface also taxes challenge. The correct control is to redesign time, evidence, route symmetry and workload, then test whether reviewers detect known weak cases. If these conditions cannot be supplied at a high consequence tier, restrict automation rather than relabel the click as accountability.

Review Independence Test Six gates for time, independent evidence, authority, protection, second review and fallback open around a human decision route. Independent review is a route with six working gates TIME EVIDENCE AUTHORITY PROTECTION 2ND ROUTE FALLBACK A closed gate means the review claim fails at this consequence tier Test with seeded errors, difficult cases and downstream outcomes
Figure 6. This article's synthesis. The review route stays open only when each gate is operational under representative workload.
Consequence Review time Independent evidence Authority Outcome horizon Deferral Reversal Appeal
Low, reversible Sample-calibrated Source access on demand Modify or manual Immediate plus short follow-up Small or none For systematic hidden rework Manager-independent route
Material service Based on case complexity Required source fields Override, defer, escalate Resolution and repeat contact Meaningful portion When local gain exported cost Worker and customer routes
High consequence No throughput compression Independent record and uncertainty Stop plus second review Disposition, harm and remedy Most material reward For concealed or invalid evidence Independent consequential review
Small sample Representative case review Direct contextual evidence Team-level disposition Longer observation window No individual incentive Correct process, not volatile pay External or board challenge

Deferral does not authorise arbitrary remote pay. The Reward Deferral and Reversal Rule identifies the earliest moment the claimed outcome can be observed, separates immediate operating feedback from material reward, protects incident reporting during the wait, and holds enough reward back when rework or harm appears later. Reversal applies when a measured gain came from inappropriate reliance, hidden work, unreported harm or a manipulated denominator. It does not punish a good-faith decision that was reasonable inside the approved boundary merely because later events were unfavourable. Model failure, workflow failure and individual misconduct require different remedies.

06 · Evidence without an obedience rankingUse a Reliance Outcome Ledger

The ledger evaluates the incentive and work system across comparable opportunities. It does not become an individual compliance table.

For each representative opportunity, record eligibility and case class; model version, capability boundary and uncertainty; evidence available to the person and system; chosen reliance mode and reason; review effort, authority and escalation; immediate quality and service effect; downstream outcome, rework, delay and cost; safety incidents and near misses; affected-party and distributional effects; value captured and displaced; reward, deferral, reversal and appeal; and missing evidence, rival explanations and learning action.

Analyse comparable cases over appropriate windows, using protected cohorts or samples when distribution matters. Separate three causal families. A model cause concerns recommendation quality, calibration or boundary. A workflow cause concerns time, evidence, interface, routing, staffing or target design. Conduct concerns arise when a person knowingly acts outside a workable rule. Jumping straight from a poor outcome to individual blame will hide product and workflow defects. Jumping from every error to system blame will erase accountability. The ledger preserves the distinction.

Reliance Outcome Ledger and delayed reward An opportunity ledger separates evidence, action, immediate outcome, downstream result, affected party and deferred reward, with correction returning after later evidence. Let the outcome mature before material reward OPPORTUNITY & EVIDENCE ACTION IMMEDIATE EFFECT DOWNSTREAM OUTCOME AFFECTED PARTY & DISTRIBUTION DEFERRED REWARD WINDOW attribute · adjust · appeal · correct Learning changes boundary and incentive
Figure 7. This article's synthesis. Immediate feedback can remain fast while material reward waits for the outcome horizon it claims to represent.

The Reliance Incentive Docket

The decision record brings together the claimed business value; opportunity denominator and exclusions; capability boundary and action set; proposed target and reward; intended and gaming chains; hazard, counterfactual and independence findings; immediate and downstream measures; workload, distribution and worker voice; management and vendor conflicts; deferral, reversal, appeal and reporting protection; repair owners and deadlines; and a final decision with expiry and reauthorisation trigger. The associated panel has seven views: opportunity set; action mix without a preferred rate; evidence and review independence; outcome, harm and distribution; gaming signals; deferred, reversed and appealed rewards; and open repairs with owners and expiry. It contains no enthusiasm score, obedience rank, minimum acceptance line or zero-override league table.

07 · Worked decisionsTen cases where the route is not the result

Each case defines the opportunity and action set, tests the payoff chain, names downstream effects and assigns a correction. Together they include proper use, non-use, override, escalation, gaming, vendor conflict, delayed harm and small-company limits.

1. The service acceptance bonus

Opportunity: eligible answerable contacts, adjusted for topic and complexity. Distortion: acceptance pays; verification consumes time. Repeat contacts and correction burden appear later. Decision: remove acceptance from pay, sample evidence quality and reward case-adjusted durable resolution. Protect manual and abstain routes. The service owner closes only after downstream outcome improves.

2. Throughput-ranked underwriters

Opportunity: in-boundary files across a published risk mix. Distortion: complex work moves to a manual queue and adopters appear faster. Decision: unify the denominator, include queue transfer and decision quality, and keep manual handling legitimate. Operations and underwriting risk own the repair; a case-mix change triggers fresh authorisation.

3. The clinician's override

Opportunity: a high-consequence case with missing contextual evidence. Distortion: override loses an adoption point. Decision: the counterfactual exposes a pure route penalty. Remove it, provide independent records and second review, and judge contemporaneous evidence plus patient outcome. Clinical safety owns escalation and appeal.

4. Fraud escalation versus speed

Opportunity: uncertain alerts separated from routine dispositions. Distortion: escalation harms personal throughput, encouraging premature closure. Decision: create a protected consequential route, adjust workload and defer material reward until disposition, false-positive burden and loss evidence mature. Fraud risk, service and assurance share ownership.

5. Vendor paid for active seats

Opportunity: validated use opportunities, not licence holders. Distortion: the supplier and product team gain from activation and accepted recommendations even when value is absent. Decision: contract for verified service outcomes, bounded harm, reversibility and learning; separate vendor evidence from buyer assurance. Commercial ownership cannot replace independent review.

6. Zero overrides and zero incidents

Opportunity: all reviewed cases and protected reports. Distortion: managers earn more when disagreement and incidents disappear from records. Decision: remove both zero targets, restore confidential reporting, inspect near misses and test seeded errors. Candour and correction count as evidence; raw report reduction does not count as safety.

7. Easy cases feed the model

Opportunity: the complete eligible population with routing history. Distortion: teams preserve an impressive AI success rate by diverting difficult cases. Decision: the opportunity audit reveals selection. Recalculate by case class, include manual queue outcomes and prevent route shopping. The product claim is reopened rather than blaming the manual team.

8. Ten-second confirmation

Opportunity: complex recommendations requiring independent verification. Distortion: throughput pay and queue pressure close the time and evidence gates. Decision: classify confirmation as ceremonial, stop high-consequence use and redesign workload. Only representative seeded-error tests can reopen the route; a longer explanation box is insufficient.

9. Bonus before rework

Opportunity: cases whose errors emerge through later appeal or repair. Distortion: the reward arrives before those signals. Decision: pay immediate operating feedback separately, defer a material portion, reverse gains built from hidden rework and protect reasonable good-faith decisions. Affected-party remedy remains independent of employee pay review.

10. A small firm with few cases

Opportunity: a sparse, varied monthly set. Distortion: individual rates are volatile and identifiable. Decision: use team-level representative review, qualitative evidence and a longer window. Do not attach reward to AI use. An external adviser or board member supplies challenge; complexity follows consequence rather than company size.

Strongest rival: sometimes use must be required

An organisation may reasonably require a new tool to be considered so people build familiarity, gather evidence, standardise a regulated step or stop ignoring a demonstrably superior aid. Outcome measures can be noisy, delayed and influenced by factors beyond one worker. Manual discretion can also be gamed; over-override and automation disuse are real risks. Small samples weaken comparison.

The answer is not unconditional neutrality about duties. Require evidence assembly, a defined check or case logging where justified, then audit the quality of that process. Do not convert consideration into required agreement. Use representative sampling, case-mix adjustment, team learning and independent review when individual outcomes are noisy. Parasuraman and Riley distinguish use, misuse, disuse and abuse; experimental work on algorithm aversion and algorithm appreciation shows that reliance can move too low or too high under bounded conditions. Governance must therefore protect justified acceptance and justified challenge alike.

Failure boundary. Do not use individual outcome incentives where the result cannot be attributed fairly, the sample is too small, the outcome arrives outside a reasonable horizon, or affected-party harm cannot be observed. Do not call a reviewer independent when their time, evidence, authority or pay closes the challenge route. In those conditions, use team learning and system controls, or remove the incentive entirely.

08 · Executable instrumentThe Appropriate-Reliance Incentive Designer

Use the designer before a target, bonus, rank, queue rule or vendor fee is authorised. It does not calculate a trust score. It assembles the opportunity, payoff, independence, outcome and correction evidence needed to choose one of five actions.

Start with a concrete business decision and its eligible denominator. Choose the consequence and capability boundary, then declare every legitimate action. Describe evidence, review time and authority as they operate at representative volume. Map the direct and indirect payoffs for the worker, manager, product team and supplier. Finally, identify the outcome horizon, gaming route, reporting protection and owners. The result is a draft docket, not proof that the selected incentive works. A positive route permits a bounded proposal to proceed to representative testing. A negative route requires redesign, deferral or removal.

Appropriate-Reliance Incentive Designer Seven evidence streams enter a decision loom and leave through five incentive routes. From opportunity evidence to one of five routes OPPORTUNITY PAYOFF REVIEW OUTCOME HARM OWNERS DEFERRAL RELIANCE DOCKET QUALITY CHALLENGE REBALANCE DEFER REMOVE
Figure 8. This article's executable synthesis. Seven evidence streams populate a reviewable docket and five materially distinct routes.

How to govern the output

Record the result in the Reliance Incentive Docket, attach the two action-chain traces and name evidence that could disconfirm the proposal. Consult workers before introducing or changing a material formula, particularly where monitoring, pay, discipline or workload is involved. Keep safety incidents in a protected route while allowing relevant aggregate learning to reach the incentive review. Set an expiry. Reauthorise after material changes to the model, boundary, population, action set, workflow, staffing, case mix, target, reward, vendor terms, monitoring, law or affected population, and after gaming, falling report rates, compressed review, unexplained action shifts or rising downstream rework.

09 · The executive ruleDo not make independent judgement economically irrational

An organisation cannot claim that people remain accountable while paying them to agree with a model. Rights to override, abstain and escalate are real only when using them does not create an unrelated penalty.

Boards and executives should ask for one bounded decision opportunity, its complete action set and the counterfactual reward across those actions. They should then ask how a rational worker could win the measure while weakening the outcome, how review independence was tested, when affected-party effects become visible, and who can reverse or retire the incentive. These questions turn a reassuring adoption chart into an operating decision.

The Reliance-Neutral Incentive Architecture is deliberately stricter than a usage dashboard and narrower than a general culture programme. It does not promise a universal formula. It creates a falsifiable condition: equal-quality decisions should not receive different treatment solely because of reliance mode, while materially different evidence and outcomes should change the response. Where that condition cannot be observed or enforced, no personal reliance incentive is justified.

Reward the quality of the decision, the discipline of the evidence, the appropriateness of challenge, the integrity of reporting and the consequence that survives downstream. Require process steps when they are necessary. Govern disuse as seriously as misuse. But never pay for the mere appearance of AI agreement and call the result human judgement.

Evidence and limitsPrimary and empirical sources

Sources were checked on 3 September 2026. NIST AI RMF 1.0 is the operative version cited; NIST states that a revision is in progress. UK guidance applies to UK government context. Regulation (EU) 2024/1689 applies within EU jurisdiction according to its phased timetable. OECD findings are bounded to the reported seven-country manufacturing and finance surveys. Experimental and meta-analytic evidence is used within its original designs; it does not establish one reward formula for every occupation.

  1. National Institute of Standards and Technology. AI Risk Management Framework Core, version 1.0.
  2. National Institute of Standards and Technology. AI Risk Management and Human-AI Interaction, AI RMF 1.0 Appendix C.
  3. UK Government. Artificial Intelligence Playbook for the UK Government.
  4. Organisation for Economic Co-operation and Development. The Impact of AI on the Workplace.
  5. International Labour Organization. Global Case Studies of Social Dialogue on AI and Algorithmic Management.
  6. European Union. Regulation (EU) 2024/1689.
  7. Kerr, S. On the Folly of Rewarding A, While Hoping for B. Academy of Management Journal, 1975.
  8. Holmström, B. and Milgrom, P. Multitask Principal-Agent Analyses. Journal of Law, Economics, and Organization, 1991.
  9. Ordóñez, L., Schweitzer, M., Galinsky, A. and Bazerman, M. Goals Gone Wild. Harvard Business School working paper, 2009.
  10. Parasuraman, R. and Riley, V. Humans and Automation: Use, Misuse, Disuse, Abuse. Human Factors, 1997.
  11. Lee, J. and See, K. Trust in Automation: Designing for Appropriate Reliance. Human Factors, 2004.
  12. Skitka, L., Mosier, K. and Burdick, M. Does Automation Bias Decision-Making? International Journal of Human-Computer Studies, 1999.
  13. Dzindolet, M., Pierce, L., Beck, H. and Dawe, L. The Perceived Utility of Human and Automated Aids in a Visual Detection Task. Human Factors, 2002.
  14. Dietvorst, B., Simmons, J. and Massey, C. Algorithm Aversion. Journal of Experimental Psychology: General, 2015.
  15. Logg, J., Minson, J. and Moore, D. Algorithm Appreciation. Organizational Behavior and Human Decision Processes, 2019.
  16. Kluger, A. and DeNisi, A. The Effects of Feedback Interventions on Performance. Psychological Bulletin, 1996.
  17. Colquitt, J., Conlon, D., Wesson, M., Porter, C. and Ng, K. Justice at the Millennium. Journal of Applied Psychology, 2001.
  18. Klein, K. and Sorra, J. The Challenge of Innovation Implementation. Academy of Management Review, 1996.