Decision Rights as the Hidden Architecture of AI Strategy
“Human in the loop” names a presence, not an authority. An operating strategy must specify who may frame, inform, recommend, decide, execute, override, validate and change each consequential AI-assisted decision.
TLDR
- The unit of governance is a bounded decision, not an AI product, committee or job title. One decision contains several rights that may need different holders.
- Recommendation, decision and execution are not synonyms. Combining them in a model or agent silently moves authority even when a nominal human remains nearby.
- Assign rights towards specific knowledge, but add independent challenge, explicit mandates and technical constraints as consequence and irreversibility rise.
- Compile the allocation into identities, permissions, interfaces, logs, time limits, escalation routes and change controls. A RACI row alone cannot stop an unauthorised action.
- The Decision-Rights Compiler rejects self-approval, conflicted validation, missing override, unowned change and residual-risk acceptance without mandate.
A retailer deploys an AI service to handle delivery refunds below £40. The steering paper says that customer-service agents remain “in the loop”. In one region, the service drafts a recommendation and a person approves every payment. In another, it triggers the refund unless a person objects within 20 seconds. In a third, it calls a payment tool directly and sends a completed-case notice. The organisation chart, supplier and policy label are identical. The authority architectures are not.
Now imagine an incorrect refund caused by a forged photograph. Who decided? “The business” is not an answer. The model ranked the evidence, a workflow set the threshold, a product manager enabled automatic execution, a service identity called the payment system and an operations lead accepted the residual fraud exposure. If the incident record captures only model output and human presence, it conceals the decisions that made the loss possible.
This is why RACI exercises often disappoint in AI programmes. They allocate broad responsibility around an artefact, while the operating system allocates narrow powers through code. A person can be accountable on a slide yet unable to halt a transaction. A developer can lack formal approval authority yet alter a prompt that changes thousands of outcomes. A reviewer can click approve without the evidence, time or mandate required to challenge.
01 · Strategic claimCompile authority before scaling intelligence
AI strategy becomes operational only when decision rights are compiled into the system. Compilation means translating a governance intent into named human and technical identities, admissible evidence, permissions, thresholds, time limits, challenge duties, logs and recovery routes. The output is an executable authority design. “Human oversight” without those properties is an aspiration.
The organisational problem predates AI. Jensen and Meckling distinguish general knowledge, which transfers cheaply, from specific knowledge, which is costly to move. Using specific knowledge often requires decentralising decision rights, but delegation creates a control problem.1 AI sharpens both sides. Models can move generalised analysis towards the front line, while the decisive context may remain with a nurse, engineer, caseworker or customer. At the same time, software can exercise delegated power at a scale and speed that make weak controls contagious.
The solution is not centralise every decision. A distant committee may lack the knowledge and latency to act well. Nor is it delegate everything to the person nearest the case. Local actors may lack a legal mandate, a portfolio view or independence from the outcome. Rights should move towards the cheapest reliable source of specific knowledge, while consequence, conflicts and reversibility determine the controls around that right.
02 · Rights grammarNine verbs reveal the hidden operating model
Start by separating nine rights. Frame defines the decision, population, objective and exclusions. Inform supplies admissible facts and policy. Recommend proposes a non-binding action. Decide chooses the outcome within a mandate. Execute changes the official world. Override halts, reverses or replaces the path. Validate independently challenges design and evidence. Change alters the model, prompt, data, threshold, tool or policy. Accept residual risk approves the exposure that remains after challenge.
These rights need not belong to nine people. In low-consequence work, one trained operator may hold several. The critical question is which combinations are unsafe or misleading. The same identity should not develop a material decision rule, validate it and accept its residual risk without an independent challenge. A recommender should not inherit execution rights merely because the tool interface makes a call convenient. A validator should be able to cause change, not merely submit a comment.
NIST's AI Risk Management Framework calls for executive responsibility for AI risk decisions and for policies that differentiate human-AI roles and oversight responsibilities.2 Its actor-task appendix spans development, deployment, operation, evaluation, domain expertise and governance.3 The useful inference is that “the owner” cannot carry every kind of knowledge and control. Accountability needs a connected set of authorities, not one ceremonial name.
Organisational research reinforces the need to separate stages. Joseph and Gaba's review of structure, information processing and decision-making notes uneven treatment of different decision stages and distinguishes perspectives that aggregate information from those that constrain action.13 An AI system can change both at once: it aggregates evidence and makes some options easier, faster or invisible. A rights map should therefore record not only who receives information, but who controls the option set, default, timing and binding act.
Create a rights ledger beside the process map. Each row names one right, its human role, technical identity, mandate source, required evidence, time window, conflict restriction, log event and fallback. Then test the row in the running system. Can the validator actually block release? Can the override holder revoke the agent token? Does the executor reject an unsigned decision? If the documented answer and observed behaviour differ, the software is the real constitution.
Frame and change rights deserve special attention because they operate upstream of case review. A product lead who changes the refund threshold may affect more customers than any case approver. A retrieval engineer who removes a policy source can alter the evidence presented to every reviewer. A compliance lead who narrows the eligible population changes what the system is permitted to see. These are strategic decisions even when no model weight changes.
“Human in the loop” fails the grammar test
A human label does not answer which person, for which decision, using what evidence, under which mandate, within what time and with what duty to challenge. NIST's human-AI interaction appendix explicitly distinguishes people who use, interact with, manage and oversee AI systems, and notes that some systems may not require human oversight at all.4 Presence should follow a control purpose. Otherwise the reviewer becomes delay, theatre or a source of automation bias.
Human-factors research separates automation of information acquisition, analysis, decision selection and action implementation.5 That distinction maps closely to inform, recommend, decide and execute. It also shows why a system can be highly automated in evidence assembly while preserving a human decision right, or allow automated bounded decisions while requiring a separate technical permission to execute.
Nominal approval can make control worse. In an experiment with 199 participants, Buçinca and colleagues found that cognitive forcing designs reduced overreliance on incorrect AI advice compared with simpler explanation designs, although participants liked the more effective interventions less.6 An approver who sees the recommendation first, has seconds to respond and faces a queue penalty for disagreement is not an independent decision-maker. The interface has already assigned practical authority to the recommendation.
Interaction can also redistribute errors across groups. Green and Chen's algorithm-in-the-loop analysis shows why system performance cannot be inferred from the model alone: human responses to risk scores can create disparate effects even when the score is formally advisory.12 The relevant control is therefore not “a person made the final choice”. It is whether the combined recommendation, interface, discretion and execution process was evaluated for the population in which it operates.
03 · AllocationMatch authority to consequence, knowledge and latency
For one decision, score consequence, reversibility, location of specific knowledge, required response time, legal or fiduciary duty and conflict risk. These dimensions shape the operating pattern. A low-value, reversible action with a clear rule may support bounded delegated execution. A material and irreversible action drawing on distributed expertise may require AI evidence assembly followed by a human decision. A high-speed material action may need dual control, pre-authorised envelopes and an immediate stop route rather than impossible case-by-case review.
Worked example one is the delivery-refund service. The frame right belongs to the commerce policy owner, who defines eligible markets, value ceiling and exclusions. The AI may inform and recommend. For refunds below £15 with matched carrier evidence, a service identity may decide and execute within a daily exposure cap. Between £15 and £40, a trained agent decides; the payment identity executes only after receiving that signed decision. Fraud operations can override the route, independent assurance validates the rule and a product-risk executive accepts residual exposure.
This design is not automatically correct. It is inspectable. A forged-photo incident can be traced to a frame, evidence source, recommendation, decision credential, payment call and threshold version. If fraud rises, the override right stops execution without waiting for a model release. If false rejections rise, the policy owner can narrow the frame while validation investigates. Rights convert a vague accountability question into testable controls.
Worked example two is an employment-screening assistant. The AI assembles evidence against published criteria and highlights missing information. It does not decide rejection. A trained hiring manager holds the decision right within a structured process; a separate people-policy function frames permissible criteria; qualified assessors validate adverse-impact evidence; candidates retain a challenge and correction route; only the applicant-tracking system identity can execute a status change. The roles differ because consequence, duties and power asymmetry differ.
The European Commission's current AI Act guidance distinguishes provider and deployer obligations and says deployers of high-risk systems must assign human oversight to people sufficiently equipped and enabled to exercise it.7 It also ties high-risk status to intended purpose and modalities of use. The strategic lesson is not that every employment tool has the same design. It is that authority must be grounded in the actual decision and applicable obligations, not inherited from a generic platform label.
Compile rights into controls, not just documents
Each right needs a technical expression. Frame becomes a versioned policy object. Inform becomes an allow-list of evidence sources with provenance. Recommend becomes a non-binding output type. Decide becomes a signed assertion from an authorised identity. Execute becomes a narrow tool permission with value, population and rate limits. Override becomes a tested stop, reversal and fallback. Validate becomes independent access to evidence and the power to require remediation. Change becomes a controlled release permission. Residual-risk acceptance becomes a dated record tied to mandate and expiry.
Logs should record the rights exercised, not only the conversation. Capture policy and system versions, evidence references, recommendation, decision identity, execution call, override opportunity, actual outcome and later changes. The record must allow a reviewer to reconstruct why the action was permitted at that moment. A perfect transcript without authority metadata cannot answer that question.
A useful token carries less authority than its holder's job title. The caseworker may be generally authorised to approve benefits, but the workflow token should cover only the open case, permitted action and decision window. The AI agent may read approved evidence yet lack the credential to alter case status. The execution service should verify purpose, amount, population and a signed decision rather than trust a natural-language instruction. Narrow tokens make accidental delegation visible and limit the damage of prompt injection or tool misuse.
NIST's current digital-identity guidance supplies a concrete scoped example: support personnel must be available to intervene in and override issue-adjudication outputs generated with algorithmic support, and findings from issue handling should feed continual improvement.11 That rule is not a universal template for all AI. It illustrates the completeness of a real control: a named human support capability, intervention power, a redress process and a route from incidents to system change.
Time changes authority. A 30-minute review window is meaningful for a weekly pricing update and absurd for an emergency shutdown. Pre-authorised envelopes can preserve speed: the system may act within tested parameters, while crossing a threshold invokes dual control or a safe state. The person on call needs the competence and system access to intervene. Escalation that reaches an inbox after execution is notification, not oversight.
04 · Independence and changeSeparate challenge from delivery, then give challenge teeth
Independence is not achieved by renaming the same team. A validator needs appropriate expertise, access to evidence, organisational standing and freedom from delivery incentives. The US banking agencies' 2026 model-risk guidance describes effective challenge in similar terms and emphasises conflicts between development and validation.8 Its scope boundary matters: generative and agentic AI models are explicitly excluded, although the guidance says governance practices should inform tools outside the document. It is precedent for control design, not a direct rule for a generative agent.
Validation must reach the change right. If independent reviewers can identify a defect but only the delivery owner can choose whether to act, challenge is advisory. Define which findings block release, reduce an envelope, force an expiry or require residual-risk acceptance by a separate mandate holder. Record disagreements and their resolution. Independence without an effect path creates reports, not control.
Residual-risk acceptance is itself a decision, not a signature at the end of a meeting. The holder needs authority over the exposed business, visibility of validation findings, an estimate of affected populations, proposed mitigations, expiry conditions and the ability to fund or stop the activity. Acceptance should name what remains uncertain and what observation reopens the decision. An executive cannot validly accept another person's statutory duty, an affected person's rights or an exposure outside the executive's mandate.
Separate acceptance from exception. A temporary exception permits a defined departure under compensating controls and an expiry date. Residual-risk acceptance acknowledges exposure after required controls are in place. Blending the two lets repeated exceptions become an undeclared operating model. The compiler should reject an expired exception, prevent it from silently renewing and route any extension back through validation and the correct mandate holder.
Change rights are often more consequential than case rights. Separate authority to change the base model, retrieval corpus, prompt, tool list, threshold, policy and interface. A model update can alter capabilities; a prompt change can alter judgement style; a tool permission can convert advice into action; a threshold can expand exposure; an interface can suppress challenge. Each component needs an owner, validation trigger, rollback unit and compatibility rule.
ISO/IEC 42001 describes an AI management system as interrelated elements that establish policies, objectives and processes for responsible development, provision or use, with continual improvement.9 A management-system standard can provide organisational scaffolding. It does not prove that a particular rights allocation is lawful, safe or effective. The decision compiler supplies the missing case-level question: who can do what, to whom, through which identity, under which tested limit?
Worked failure: the reviewer who cannot refuse
A benefits agency pilots an assistant that recommends whether uploaded evidence is sufficient. Caseworkers formally decide. The system pre-selects “sufficient”, displays the supporting summary before the source, allows eight seconds before the queue target turns red and requires a long free-text reason for disagreement. Managers monitor override rates as a productivity problem. The policy states human decision-making; the interface assigns recommendation and practical decision power to the model.
Repair requires more than training. Show source evidence before or alongside the recommendation, preserve an unassisted view for selected cases, make challenge symmetric, remove punitive override metrics, sample disagreements for learning and ensure the caseworker's credential is the only decision token. High-consequence cases or missing evidence route to a second authority. The redesign changes information order, incentives, permissions and escalation, which are all components of the rights architecture.
05 · Decision instrumentThe Decision-Rights Compiler
Run the compiler for one decision and population. Choose the most consequential ordinary case, not a reassuring average. The result proposes an authority pattern. It does not determine legality, validate the model, identify every conflict or replace domain-specific process design. Every non-zero answer needs a source, owner and expiry trigger.
Decision-Rights Compiler
The compiler combines consequence, knowledge, latency and control conditions, then rejects authority arrangements that cannot produce accountable action.
Assumptions, rejection rules and limits
- Assumptions: a single decision and population are named; the role and technical-identity inventory is current; evidence sources are admissible; and execution semantics are known.
- Advisory test: low consequence, no binding tool access and ample time should keep the AI in evidence assembly or recommendation.
- Bounded-action test: low-to-moderate consequence, reversible effects, tested override and narrow permissions may support delegated action inside a pre-authorised envelope.
- Dual-control test: material fast action with distributed knowledge should require two independent conditions or authorities before execution.
- Human-decision test: persistent high consequence, professional duty or contested knowledge should preserve the decision right for a qualified human while AI assembles evidence.
- Rejection rules: broad execution plus absent override, required authorisation without a mandate holder, or high self-approval risk produces redesign regardless of speed or model quality.
- Limits: ordinal scores can be gamed. The compiler cannot legalise prohibited activity, create missing expertise, validate a model, establish fairness or repair an unusable interface.
06 · Rival and boundaryRights are necessary for accountable action, not sufficient for value
The strongest rival explanation for failure is capability. A clear authority map cannot rescue a weak model, irrelevant data, broken workflow, poor incentives or an unusable interface. A well-governed refund assistant can still cost more than it saves. A validated screening model can still address the wrong business problem. Decision rights make the action attributable and challengeable; they do not make the action intelligent.
Test the rival directly. Hold the rights architecture constant while improving data or model quality. If outcomes recover, capability was the binding constraint. Hold model performance constant while changing evidence order, permissions or approval timing. If error and override behaviour change, authority design mattered. Both mechanisms may interact, but treating every failure as a governance issue wastes effort and hides technical defects.
Proportionality matters. A deterministic, reversible formatting tool that removes extra spaces has no material decision effect. It needs ordinary software ownership, security and quality controls, not a nine-rights council. NIST notes that some systems, such as video-compression models, may not require human oversight.4 Elaborate control around trivial consequence creates queues and teaches teams to evade governance.
The opposite boundary is non-negotiable. Rights mapping cannot authorise a prohibited use, create a missing professional qualification, turn a proxy into valid evidence or make an irreversible harm recoverable. A workflow cannot delegate authority that the organisation itself does not hold. A technical token cannot expand a person's legal mandate. When a required decision-maker cannot receive usable evidence in time, the answer is redesign or stop, not a faster approval button.
Formal structure also needs lived power. The assigned reviewer must be equipped, enabled and protected when exercising oversight. The EU guidance uses precisely the language of sufficiently equipped and enabled human oversight for deployers of high-risk systems.7 NIST similarly notes that empowerment and incentives to challenge AI output require attention.4 A compiler should therefore test workload, interface and consequences for disagreement, not only names in a policy.
Internal override is not the same as external challenge. A customer, worker or citizen may need to contest input facts, correct a record, understand the material basis of a decision or obtain human reconsideration. Map that route as part of the decision architecture: who receives the challenge, which execution can be paused, which evidence can be corrected, who re-decides and how the system learns without treating every complaint as a label. A complete authority design covers both power to act and power to seek redress.
07 · ConclusionAuthority is part of the product
AI strategy does not end when leaders choose a model, supplier or use case. It becomes operational when the organisation specifies which consequential decisions exist and compiles their rights into the system. Frame, inform, recommend, decide, execute, override, validate, change and accept residual risk are different verbs because they create different powers and failure modes.
The design principle is balanced. Move authority towards the people and systems holding decision-specific knowledge. Add independent challenge, mandate checks, narrow technical permissions and tested override as consequence rises. Preserve speed through pre-authorised envelopes and typed tools rather than pretending a human can review an impossible queue. Treat prompt, tool, threshold and policy changes as authority events, not minor configuration.
The practical test is simple to state and demanding to pass. For one decision, can the organisation identify who frames it, what evidence is admissible, whether the AI recommends or decides, which credential permits execution, who can stop it, who challenges the design, who can change each component and who has the mandate to accept what remains? If any answer is “the system”, “the business” or “a human”, the architecture is still hidden.
Making it visible does not guarantee a good model or a valuable service. It does make delegation intentional, action reconstructable and challenge capable of changing the system. That is the minimum operating architecture for scaling AI without allowing convenience to become authority by accident.
SourcesReferences
- Jensen, M. C. and Meckling, W. H. (1995). Specific and General Knowledge, and Organizational Structure. Journal of Applied Corporate Finance, 8(2), 4–18.
- NIST (2023). Artificial Intelligence Risk Management Framework Core. Used for executive responsibility, differentiated roles, oversight, override and change management.
- NIST (2023). AI RMF Appendix A: Descriptions of AI Actor Tasks.
- NIST (2023). AI RMF Appendix C: AI Risk Management and Human-AI Interaction.
- Parasuraman, R., Sheridan, T. B. and Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics Part A, 30(3), 286–297.
- Buçinca, Z., Malaya, M. B. and Gajos, K. Z. (2021). To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-Assisted Decision-Making. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), Article 188.
- European Commission (current 2026 guidance). Navigating the AI Act. Used for role-specific obligations, intended purpose and enabled human oversight; not as a substitute for the regulation or legal advice.
- Board of Governors of the Federal Reserve System, OCC and FDIC (2026). Supervisory Guidance on Model Risk Management, SR 26-2. Scope excludes generative and agentic AI models; cited as control-design precedent with that boundary.
- ISO (2023). ISO/IEC 42001:2023, Artificial intelligence management system. Used for management-system lifecycle context, not as evidence that a particular system is certified, safe or lawful.
- European Union (2024). Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence. The applicable text and implementation status should be checked for each use.
- NIST (2025). Special Publication 800-63-4, Digital Identity Guidelines. Used for the concrete requirement that support personnel can intervene in algorithm-supported issue adjudication within its digital-identity scope.
- Green, B. and Chen, Y. (2019). Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments. FAT* 2019.
- Joseph, J. and Gaba, V. (2020). Organizational Structure, Information Processing, and Decision-Making: A Retrospective and Road Map for Research. Academy of Management Annals, 14(1), 267–302.