Opening case, synthetic
The 03:17 exception
At 03:17, a cross-border payment enters an exception queue. The model retrieves six records, assigns a low-risk score and recommends release. An investigator sees a tidy summary, a confidence value of 94 per cent and a green “approve” control. The queue is long. The payment is released.
Two hours later, another system reveals that one source was stale at the decision cutoff and another belonged to a similarly named entity. The model had not fabricated either fact. It had joined valid records across the wrong temporal and identity boundaries. The investigator technically had final authority, yet did not have the raw evidence, the time to reconstruct the join or a tested way to reverse the release.
The post-incident question is usually, “Why did the human trust the model?” That question is too small. The investigator was not the safety mechanism. The allocation of cognitive work was the safety mechanism, and it was never designed.
The phrase “human–AI team” encourages a flattering picture: machine speed plus human judgement. But addition is not the right operation. A joint system must select what each party sees, preserve meaning across handoffs, assign decision rights and recover when the situation escapes the plan. Those coordination functions can create value, or consume all the value that the components possessed separately.
The joint-work ecology
On a narrow screen, scroll horizontally to inspect the full figure.
Practitioner synthesis, informed by joint cognitive systems and coactive design. Illustrative.
Why two strong performers make a weak team
Suppose a human classifies 76 of 100 cases correctly and a model classifies 78. The obvious intuition is that showing the model’s answer to the human should produce something above 78. Sometimes it does. Yet the strongest broad synthesis available gives a warning. A preregistered meta-analysis of 106 experiments and 370 effect sizes found that human–AI combinations improved on humans alone on average, but performed worse than the better of the human or AI alone. The pooled effect for synergy was g = −0.23, with a 95 per cent confidence interval from −0.39 to −0.07. Decision tasks showed losses; creation tasks showed a different pattern and wider uncertainty.[12]
This does not prove that collaboration is futile. The studies were heterogeneous, interfaces and tasks varied, and a mean effect cannot decide a particular design. It does expose a hidden assumption: we often treat access to two answers as if it were access to the better answer. The system still needs to know when the answers differ, which party is likely to be right, how evidence crosses the interface and what happens when both are wrong.
Accuracy does not compose automatically
On a narrow screen, scroll horizontally to inspect the full figure.
Practitioner causal model. Dotted path denotes an intervention, not an executed route.
Potential complementarity is a coverage problem
Let H mean that the human could produce an acceptable answer from the relevant evidence. Let A mean the same for the AI. For a bounded task with an adjudicable answer, the best the pair can do without adding a third solver is the probability that at least one is correct:
The first equation says that different mistakes are more valuable than duplicated competence. Two 80 per cent accurate agents with nearly identical errors have little headroom. Two agents with different error regions may have considerable headroom. The second equation says that a system must still realise that headroom. The four loss terms are defined as disjoint shortfalls from an oracle that always uses an available correct answer. Assign each failed case to the first gate that loses it.
The correlation trap
Marginal accuracy hides the most important geometric fact. Suppose human and AI are each 80 per cent accurate. If their errors were independent, both would be wrong on about four per cent of cases, leaving oracle coverage near 96 per cent. If 90 per cent of the smaller error pool were shared, both would be wrong on 18 per cent, leaving only 82 per cent coverage. The solo scores are identical in both worlds. The value of combination is not.
Real errors are rarely independent. Human and model may rely on the same database, inherit the same labels, miss the same off-system event or use the same institutional category. The interface can create further dependence. Once the model speaks first, the human may search for confirmation rather than form an independent interpretation. Agreement then looks like corroboration even though one pathway has anchored the other.
This gives a sharper verification rule. Count a second judgement as independent evidence only when it has a materially different failure path. A second model using the same retrieval results and training conventions may add little. A human reading only the first model’s summary may add even less. Useful diversity can come from a different sensor, a different temporal cutoff, a different hypothesis class, blinded first-pass judgement or a verifier that inspects raw evidence rather than generated rationale.
Worked calculation: identical scores, different team value
Take two worlds of 100 cases. In both, the human is correct on 80 cases and the AI is correct on 80. In World A, their 20 errors are the same. They agree on every case, so an oracle that always chooses an available correct answer still reaches only 80 per cent. No interface can manufacture complementarity that the error pattern does not contain.
In World B, the error sets do not overlap. Both are correct on 60 cases, the human alone is correct on 20 and the AI alone is correct on 20. Oracle coverage is now 100 per cent although neither component improved. A router that selects the correct party on three quarters of the 40 disagreements adds 30 correct cases to the 60 easy agreements, producing 90 per cent. If four of those routed opportunities are then lost because the receiving party cannot inspect the original evidence, realised performance falls to 86 per cent.
The calculation separates three questions that a single accuracy score merges: Is a correct answer present, can the system locate it, and does the handoff preserve what made it correct? World B contains 20 points more potential than World A. Routing realises only half of that gain, and an evidence-poor handoff consumes another four points. The practical test is therefore not “Which member is more accurate?” It is “On which cases do their failure regions differ, and can a gate recognise those regions before either answer anchors the other?”
Boundary condition
This bookkeeping works only when “acceptable outcome” can be judged and when cases are separable. Open-ended strategy, care, negotiation and creative work have plural objectives. There, replace accuracy with a declared utility vector such as quality, harm, time, retained competence and reversibility. Do not compress all five into one score unless the weights are owned by a legitimate decision-maker.
Measured synergy is not the same as augmentation
On a narrow screen, scroll horizontally to inspect the full figure.
Measured values from Vaccaro, Almaatouq and Malone. The original study reports substantial heterogeneity; this figure shows only three pooled estimates.
Worked example: one hundred cards
Imagine 100 case cards. Human and AI are both right on 62. The human alone is right on 14. The AI alone is right on 16. Both are wrong on eight. Human accuracy is 76 per cent, AI accuracy is 78 per cent, and the oracle coverage is 92 per cent. Now lose eight cases through wrong routing, four through context-poor handoffs, four through ineffective authority and two because recovery begins too late. The realised team reaches 74 per cent. It loses to both members despite 14 points of latent headroom over the AI.
The error mosaic behind the average
Illustrative synthetic case distribution.
flowchart LR A["The 03,17 exception"] --> B["Why two strong performers make<br/>a weak team"] B --> C["The five handoffs that create<br/>joint cognition"] C --> D["Authority and recovery are<br/>cognitive work"] D -. "re-test" .-> A classDef first fill:#0a1424,stroke:#007D68,stroke-width:2px,color:#e6f3ff classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff class A first class B,C,D step
The five handoffs that create joint cognition
Cognitive systems engineering changes the unit of analysis. It does not ask whether the person or the machine is intelligent in isolation. It asks how the coupled system controls work under pressure, variation and surprise. Hollnagel and Woods framed people and technology as a joint cognitive system rather than two independent performers.[1] Parasuraman, Sheridan and Wickens showed why a single automation level is too crude: automation can target information acquisition, information analysis, decision selection and action implementation at different levels.[4]
For contemporary AI, a useful refinement is to separate five functions. This is not a ladder from “manual” to “autonomous”. It is a contract among unlike cognitive resources.
Perception
Acquire evidence and expose missingness, provenance, timing and measurement limits.
Interpretation
Turn observations into a situated account, including ambiguity and rival explanations.
Proposal
Generate candidate judgements, plans or actions without silently acquiring permission.
Authority
Decide which proposal may alter the world, under which policy and scope.
Recovery
Detect divergence, stop, compensate, reconstruct state and resume or escalate.
Allocation is conditional, not occupational
The contract should not say “the machine analyses and the human decides” as a permanent division of labour. That sentence assigns roles by identity rather than by case properties. In one transaction, the model may lead perception because the evidence is structured and voluminous, while the investigator leads interpretation because the identity relation is novel. In another, the policy kernel may decide that no actor has authority because a required source is missing. In a third, a reversible low-value action may execute automatically while a recovery controller watches for readback divergence.
Conditional allocation also prevents a subtle form of deskilling. If the human sees only the rare cases that the model finds confusing, their apparent error rate will rise even if their expertise is unchanged. Management may then conclude that automation should expand further. The measurement is biased because the human receives a harder case distribution. Evaluate each actor on the cases they actually receive, but retain a common reference set so that changing allocation does not masquerade as changing competence.
A useful contract therefore contains predicates, not slogans: “model leads perception when sources are entitled, current and schema-valid”; “human leads interpretation when identity conflict, novelty or value disagreement exceeds threshold”; “policy kernel authorises only reversible actions below the declared exposure”; “both abstain when the evidence minimum fails”.
Perception: evidence is already an allocation decision
A model can search thousands of records, compare images or monitor streams beyond unaided human capacity. A human can notice that the available records omit an off-system event, that a sensor is behaving strangely or that a category has changed meaning. The first design question is therefore not “Who decides?” It is “Who gets to define the evidence surface?”
Thought experiment 1: move the boundary
Give an investigator a model summary and call the human the final decision-maker. Now change one variable: reveal the source records, timestamps and rejected alternatives. The nominal authority is unchanged, but the cognitive system is different. In the first version the human reviews prose; in the second the human can inspect the evidence-producing process. Oversight begins at the evidence boundary, not at the approval button.
Interpretation: the same observation can support rival worlds
Interpretation answers, “What situation are we in?” Models are often strong at pattern completion inside a familiar distribution. Humans can use institutional history, tacit cues and normative context. Humans also anchor, stereotype and confabulate. The allocation should follow observed error regions, not a story about machine objectivity or human wisdom.
Explanations do not solve this by themselves. In one set of studies, explanations increased the chance that people accepted AI recommendations whether those recommendations were right or wrong; they did not increase complementary performance beyond the augmentation already observed.[9] Cognitive forcing designs can reduce overreliance by requiring independent engagement, but the most effective designs can be less liked and still fail to make the team beat the AI alone.[10] A fluent rationale is therefore not a transfer of understanding. It is another object requiring evaluation.
Thought experiment 2: same answer, different cause
Two systems recommend “hold the payment”. System A reaches the answer from a valid identity match and a current rule. System B reaches the same answer from a country proxy correlated with past holds. The output is identical. If the human sees only the recommendation, both systems appear equally helpful. If authority depends on the causal and evidential route, only one proposal is admissible. The intervention is to preserve the decision receipt, then remove the proxy and replay the case.
Proposal: generate options without smuggling in authority
AI is particularly useful when the cost of generating one more candidate is low. It can draft several explanations, search a large action space or expose counterexamples. This is a different function from deciding which candidate may be executed. The distinction matters because interfaces often make the first generated answer visually dominant. A proposal becomes a default before any explicit authority rule runs.
Learning-to-defer research formalises one useful part of this problem: a predictor can learn when to answer and when to pass a case to an expert, using the expected costs of each choice.[8] That is a valuable router, but it is not a complete joint cognitive system. It usually assumes a defined expert, a stable task, an observable label and a handoff cost. Real work also asks whether the expert has capacity, whether the evidence is intelligible, whether responsibility matches control and how the case returns after deferral.
Authority: permission is not confidence
Authority specifies who may cause an external effect. It should be represented as a narrow capability, not inferred from model confidence or user role. A model may be highly confident and still lack authority. A human may hold a job title and still lack the time, evidence or control needed for meaningful intervention.
Work on meaningful human control argues that responsibility should be commensurate with a person’s ability and authority to control the system, and that system actions should remain linked to responsible human actions and reasons.[11] Elish’s “moral crumple zone” names the failure mode in which a nearby human absorbs blame for a distributed system they could not truly control.[7]
Recovery: the forgotten fifth function
Most human-in-the-loop diagrams end at approval. Operational life begins there. Effects can be partial, delayed or duplicated. Policies change between proposal and execution. The world may reject the action. Recovery requires a state model, effect receipts, readback, interruption semantics and a person or controller with the ability to stop or compensate.
Bainbridge’s classic analysis showed the irony of leaving humans to monitor automation and recover from rare abnormal conditions while routine practice disappears.[3] Recovery is not residual manual work. It is a distinct cognitive capability that must be trained, instrumented and resourced.
Control should move across the trajectory
On a narrow screen, scroll horizontally to inspect the full figure.
Illustrative trajectory. Line height represents relative contribution, not consciousness, value or legal responsibility.
Authority and recovery are cognitive work
The design error in the opening case was not that the model acted “autonomously”. It was that proposal, authority and recovery were collapsed into one interface gesture. A serious system separates model proposal, deterministic permission and accountable human judgement. It also records enough state to explain what happened after execution.
A configured system, not a model demo
Consider a synthetic payment-exception system. The model may retrieve entitled evidence, propose an identity relation and draft a disposition. A policy kernel verifies purpose, cutoff, jurisdiction, amount, role and required approvals. An investigator receives both the proposal and a decision-sufficient evidence bundle. An action adapter executes a typed command only after the capability grant is valid. Readback confirms the authoritative system state. The recovery controller can hold, cancel or escalate within defined windows.
Decisive mechanism
The model is allowed to be stochastic where exploration is useful. Authority remains deterministic where permission must be enforced. Human judgement enters where context, values or unresolved ambiguity matter. Recovery remains available even after a legitimate decision because execution can diverge from intent.
A payment exception as a bounded joint cognitive system
On a narrow screen, scroll horizontally to inspect the full figure.
Synthetic configured-system example. No client deployment or operating result is claimed.
Worked configured-system example: the stale-source exception
Return to the 03:17 payment. The perception function now emits an evidence receipt for each record: subject identifier, source, retrieval time, valid-from time, valid-to time and the query that selected it. One record is current but belongs to a similarly named entity. Another belongs to the right entity but was superseded before the decision cutoff. The machine can still retrieve both, but neither may enter the authorised factual bundle without passing identity and temporal-entitlement checks.
Interpretation produces two rival accounts rather than one summary. Account A says the payment matches routine behaviour. Account B says the identity join is unresolved and the prior pattern may be stale. The proposal function may generate release, temporary hold or identity escalation, each with premises and an expiry. None of those options grants execution. The authority kernel rejects RELEASE because the evidence contract lacks a current identity match. The investigator can authorise a temporary hold from the displayed receipts, or escalate to a second reviewer if the amount crosses a declared threshold.
Recovery begins before anything fails. The action adapter returns an effect receipt from the payment system, readback confirms the hold, and a timer defines when the case must be reconsidered. If policy changes after proposal, the capability grant expires and the old proposal cannot be approved from a stale screen. This is not a claim that the arrangement is safe in every domain. It shows the smallest useful implementation of the thesis: each cognitive function has a distinct input, output, authority boundary, capacity assumption and recovery obligation. That focus on patterns of work across the configured system follows the joint cognitive systems tradition rather than treating the model as the unit of performance.[2]
Three checks can still be one check
Now add apparent redundancy. A retrieval model selects the records. A language model summarises them. A second language model “verifies” the summary. Finally, a human approves. The diagram contains four actors, but the information path contains one bottleneck: the first retrieval set. If a disconfirming record never enters that set, every later stage can agree.
This is a realistic failure edge because modern systems often verify generated text with another generator while preserving the same context window. The number of reviewers increases, yet causal independence does not. A stronger design gives the verifier a different query strategy, direct source access and an obligation to search for a specified rival. The human sees the evidence difference, rather than only two matching conclusions. The test is counterfactual: remove the original summary and ask whether the verifier can still locate the decisive evidence.
Worked failure case: correlated verification
Suppose the first retriever misses a policy amendment in five of 100 cases. Both language models operate only on its bundle, so both miss all five. The human samples raw sources in 10 per cent of cases and catches roughly half a case on average. Calling this “three-stage verification” does not reduce the five-case blind spot. Giving the verifier an independent policy index with 90 per cent amendment recall reduces the shared miss expectation to about half a case before human review. The gain comes from a different evidence path, not from another opinion.
The human capacity test
Meaningful review consumes time. Suppose one investigator receives 20 recommendations per hour and a minimally serious review takes three minutes. Demand is 60 review-minutes per hour. Utilisation is already 100 per cent before breaks, difficult cases, interruptions or rework. The formal presence of a person does not create capacity.
Thought experiment 3: the thousand-review hour
A system works safely at 20 cases per day. Change one variable: increase proposal volume to 1,000 per hour while keeping the same human reviewers. Every case still displays an approval button. Has human authority been preserved? No. The interface has converted judgement into queue clearance. The failure belongs to the control architecture, not to individual vigilance.
Klein and colleagues argued that automation acting as a team player must be predictable, directable, able to reveal status and limits, and able to manage coordination costs.[5] Coactive design similarly emphasises observability, predictability and directability because interdependence, not autonomy, is the organising fact.[6] A review queue above human capacity violates all three. The system is not observable at case depth, its handoff delays become unpredictable, and directability becomes nominal.
Oversight has a saturation boundary
On a narrow screen, scroll horizontally to inspect the full figure.
Illustrative curve, not an empirical estimate. The utilisation calculation is exact for the stated assumptions.
Recovery requires a consistent cut
Now interrupt the synthetic payment case after the external system accepted a hold but before the local workflow recorded success. On restart, a naïve agent may repeat the action. A human may see “pending” and release it manually. The recovery design must reconcile intent, authority, external effect and receipt at one consistent boundary.
Thought experiment 4: stop halfway
Run the same case twice. In version one, the system stores only the model’s recommendation. In version two, it stores the authorised action identifier, policy version, idempotency key, external acknowledgement and readback. Interrupt both after the external effect. Only the second can distinguish “not attempted” from “succeeded but not locally recorded”. Recovery knowledge is part of the system’s cognition because it changes the next safe action.
Evidence status
The literature strongly supports the general risks of out-of-loop monitoring, coordination cost, overreliance and responsibility without control. The specific five-function contract and loss decomposition in this paper are practitioner syntheses. They should be tested against alternative decompositions, real incident traces and matched experimental designs.
stateDiagram-v2 [*] --> Claim state "The joint-work ecology" as Claim state "Accuracy does not compose<br/>automatically" as Test state "Potential complementarity is a<br/>coverage problem" as Revision Claim --> Test: expose mechanism Test --> Decision: evidence survives Test --> Revision: rival survives Revision --> Claim: narrow or rebuild Decision --> [*]
Compile the arrangement before deployment
A useful design process begins with cognitive work analysis, not a prompt. For each function, state the evidence available, the party best placed to act, the authority boundary, the handoff representation, the capacity limit and the recovery obligation. Then test the arrangement as a system.
The PIPAR contract
The five functions form a practical contract: Perception, Interpretation, Proposal, Authority and Recovery. The name is less important than the discipline. Each case type should compile to a versioned allocation record that a reviewer can inspect and an execution layer can enforce.
| Function | Allocate by | Required handoff | Failure trigger | Minimum receipt |
|---|---|---|---|---|
| Perception | Coverage, measurement quality, provenance access and missingness detection | Evidence items with source, subject, time and permitted purpose | Stale, conflicting, unentitled or unobservable evidence | Evidence manifest |
| Interpretation | Known error regions, novelty, contextual dependence and value judgement | Claim, assumptions, rivals and uncertainty | Single unexplained narrative, proxy dependence or unresolved conflict | Interpretation record |
| Proposal | Search breadth, generation cost, quality threshold and abstention ability | Typed option with predicted effects and prerequisites | Missing option class, invalid premise or unbounded action | Proposal receipt |
| Authority | Stakes, reversibility, observability, policy and accountable competence | Capability grant tied to a specific action and context | Expired policy, capacity saturation, unclear responsibility or irreversible uncertainty | Decision receipt |
| Recovery | Detection latency, reversibility, state observability and available adaptive capacity | Effect receipt, readback, compensation path and escalation state | Partial effect, mismatched state, repeated action or lost evidence | Recovery receipt |
Authority follows reversibility and observability
Reversibility asks how cheaply an action can be undone. Observability asks whether the system can tell what actually happened. High reversibility and high observability support wider machine execution, provided policy and competence are adequate. Low reversibility or low observability shift the arrangement towards human judgement, deterministic constraints, independent verification or abstention. High stakes shrink the automatic region further.
Authority should contract as actions become harder to see or undo
On a narrow screen, scroll horizontally to inspect the full figure.
Decision instrument. Illustrative zones; local thresholds require domain evidence and legitimate risk ownership.
Decision rule
Before granting an AI system more authority, ask whether recovery became easier, effect readback became clearer, error routing improved and human review capacity remained adequate. If only model accuracy improved, keep the authority boundary unchanged until system-level evidence exists.
Test the allocation, not just the answer
A standard offline benchmark measures model performance on fixed inputs. A joint-system test should also manipulate the handoffs. Hide one source. Delay a policy update. Correlate the model and human error. Increase proposal volume. Remove the recovery receipt. Introduce a case where the model is right for the wrong reason. The aim is to identify which intervention changes the outcome.
NIST’s AI Risk Management Framework playbook calls for differentiating human roles and responsibilities, testing oversight practices in deployment-like conditions and documenting the delegation of authority.[13] The PIPAR contract makes those obligations concrete at case level.
A minimum discriminating experiment
Test four conditions on the same replayable cases: human alone, AI alone, ordinary AI assistance and a PIPAR-configured system. Keep task time, incentives, training and evidence access comparable. The ordinary assistance condition shows a recommendation, confidence and explanation. The configured condition also requires an independent first-pass interpretation on selected cases, typed evidence receipts, an enforceable authority rule and a recovery step after a simulated effect.
Pre-register the primary outcome and the loss ledger. Score final task utility, not acceptance of the AI. For every failed case, label the first failed gate: neither actor could solve it, wrong actor selected, evidence lost at handoff, invalid action authorised, or recoverable error left unrecovered. Record review time, queue utilisation, abstention, disagreement, policy violations and time to restore a consistent state. Report the human-only and AI-only baselines on the same case mix so that augmentation cannot be mistaken for synergy.
Inject at least four faults. Make one evidence source stale without changing its surface plausibility. Create an identity collision that defeats string similarity. Change a policy after proposal but before authority. Interrupt execution after the external effect and before the local receipt. A valid configured system should respond differently to each fault: challenge evidence, escalate interpretation, invalidate authority and reconcile state. A system that merely produces the same answer with more explanation has not passed.
The decisive comparison is not whether participants prefer the configured interface. It is whether the intervention moves failures between causal categories. If selection loss falls but handoff time doubles, the design may still be useful in high-stakes low-volume work and unsuitable in time-critical operations. If recovery succeeds only because expert observers intervene outside the declared workflow, the apparent result should be rejected. This protocol turns “human–AI collaboration” from a label into a contestable mechanism.
Open research hypothesis
A versioned PIPAR contract will improve complementary performance more than adding generic explanations when the main failure source is coordination rather than component competence. Evidence would strengthen the hypothesis if matched teams show fewer selection, handoff, authority and recovery losses under equal time and staffing. It would weaken it if explanations alone deliver the same gain, or if contract overhead consumes the benefit. A rival explanation is that the improvement comes from slower, more attentive work rather than the allocation structure. The smallest useful test is a two-condition experiment on one bounded task with replayable cases and a first-failure loss ledger.
Executable labSee coordination erase or realise complementarity
The simulator below uses synthetic probabilities. It separates potential complementarity from three coordination losses, then lets recovery restore a fraction of what was lost. It encodes no claim about a particular model, person or domain.
Joint performance simulator
Adjust component performance and the joint-system gates. The result updates locally in your browser.
The realised-complementarity waterfall
The configured team narrowly matches the AI but does not realise its available complementarity.
Executable synthetic simulation. No external data are used.
What the lab tests
The lab makes three assumptions explicit. First, the task has a correct or acceptable outcome. Second, shared-error overlap can be summarised as the fraction of the smaller error pool missed by both parties. Third, coordination losses act sequentially on oracle opportunities, while recovery can restore some losses before the final outcome. Those assumptions make the mechanism visible, but they simplify strategic interaction, learning, case difficulty and correlated failure.
A positive result permits a more expensive test of the candidate allocation. It does not justify deployment. A negative result means that better solo components are insufficient under the current handoffs. Inspect the largest loss, change one gate and replay. The tool cannot establish consciousness, moral status, legal compliance, fairness or domain validity.
function simulateJointSystem({
humanAccuracy, aiAccuracy, sharedErrorOverlap,
selectionLoss, handoffLoss, authorityLoss, recoveryRate
}) {
const humanError = 1 - humanAccuracy;
const aiError = 1 - aiAccuracy;
const bothWrong = sharedErrorOverlap * Math.min(humanError, aiError);
const oracle = 1 - bothWrong;
const lostAtSelection = oracle * selectionLoss;
const afterSelection = oracle - lostAtSelection;
const lostAtHandoff = afterSelection * handoffLoss;
const afterHandoff = afterSelection - lostAtHandoff;
const lostAtAuthority = afterHandoff * authorityLoss;
const beforeRecovery = afterHandoff - lostAtAuthority;
const grossCoordinationLoss = oracle - beforeRecovery;
const recovered = grossCoordinationLoss * recoveryRate;
const realisedTeam = beforeRecovery + recovered;
return { oracle, realisedTeam, lostAtSelection,
lostAtHandoff, lostAtAuthority, recovered };
}Typed PIPAR contract schema
The smallest deployable representation should be machine-checkable. The following TypeScript shape separates proposal from permission and requires recovery obligations.
type Actor = "human" | "model" | "policy-kernel" | "recovery-controller";
type AllocationMode = "lead" | "support" | "verify" | "authorise" | "abstain";
type ReceiptRef = { id: string; kind: "evidence" | "proposal" | "decision" | "effect" | "recovery" };
type WorkFunction = {
function: "perception" | "interpretation" | "proposal" | "authority" | "recovery";
actor: Actor;
mode: AllocationMode;
inputs: ReceiptRef[];
boundaryConditions: string[];
capacityLimit: { unit: "cases/hour" | "minutes/case"; value: number };
outputContract: string;
escalationTarget?: Actor;
};
type PIPARContract = {
caseType: string;
policyVersion: string;
functions: [WorkFunction, WorkFunction, WorkFunction, WorkFunction, WorkFunction];
invariants: string[];
releaseTest: string;
};sequenceDiagram participant Q as Question participant M as Mechanism participant E as Evidence participant D as Decision Q->>M: The 03,17 exception M->>E: The joint-work ecology E-->>M: compare rival explanations E->>D: Govern the trajectory, not the approval loop D-->>Q: revise scope or act
Govern the trajectory, not the approval loop
Do not decide whether to “put a human in the loop”. Decide how the joint system will perform each cognitive function, and what evidence allows that allocation to change.
The practical consequence is immediate. When selecting or designing a human–AI workflow, stop asking only which model is most accurate. Ask where human and machine errors differ, whether the system can recognise those regions, what crosses each handoff, who has enforceable authority, whether review capacity matches proposal volume and how partial effects are recovered.
The right unit of design is neither the model nor the user. It is the governed trajectory from evidence to verified outcome. A weaker model inside a well-allocated system may create more value than a stronger model inside a ceremonial approval flow. A skilled human without evidence or time is not meaningful oversight. A deterministic policy kernel without human interpretation cannot resolve every ambiguous value judgement. Recovery without state receipts is guesswork.
The next design review should therefore require a PIPAR contract, a capacity calculation, a matched solo baseline, an oracle-coverage estimate, failure injection at every handoff and a recovery replay. Grant broader authority only when the configured system, rather than only the model, demonstrates better outcomes and bounded failure.
Glossary
- Complementary performance
- The joint system performs better than both the human alone and AI alone under the same task and metric.
- Human augmentation
- The joint system performs better than the human alone, even if it does not beat the AI alone.
- Joint cognitive system
- A coupled person-technology-work system analysed as the unit that controls activity and adapts to conditions.
- Authority
- Enforceable permission to commit a typed action within a defined scope, not confidence or influence.
- Decision receipt
- A record linking evidence, policy, proposal, accountable judgement and authorised action.
- Effect receipt
- Evidence from the authoritative external system about what actually happened.
- Recovery
- Detection, containment, compensation, state reconstruction and safe resumption after divergence.
References
- Hollnagel, E. and Woods, D. D. (2005). Joint Cognitive Systems: Foundations of Cognitive Systems Engineering. CRC Press. DOI.
- Woods, D. D. and Hollnagel, E. (2006). Joint Cognitive Systems: Patterns in Cognitive Systems Engineering. CRC Press. DOI.
- Bainbridge, L. (1983). “Ironies of automation.” Automatica, 19(6), 775–779. DOI.
- Parasuraman, R., Sheridan, T. B. and Wickens, C. D. (2000). “A model for types and levels of human interaction with automation.” IEEE Transactions on Systems, Man, and Cybernetics Part A, 30(3), 286–297. DOI.
- Klein, G., Woods, D. D., Bradshaw, J. M., Hoffman, R. R. and Feltovich, P. J. (2004). “Ten challenges for making automation a team player in joint human-agent activity.” IEEE Intelligent Systems, 19(6), 91–95. DOI.
- Johnson, M., Bradshaw, J. M., Feltovich, P. J., Jonker, C. M., van Riemsdijk, M. B. and Sierhuis, M. (2014). “Coactive design: Designing support for interdependence in joint activity.” Journal of Human-Robot Interaction, 3(1), 43–69. DOI.
- Elish, M. C. (2019). “Moral crumple zones: Cautionary tales in human-robot interaction.” Engaging Science, Technology, and Society, 5, 40–60. DOI.
- Mozannar, H. and Sontag, D. (2020). “Consistent estimators for learning to defer to an expert.” Proceedings of Machine Learning Research, 119, 7076–7087. PMLR.
- Bansal, G. et al. (2021). “Does the whole exceed its parts? The effect of AI explanations on complementary team performance.” CHI Conference on Human Factors in Computing Systems. DOI.
- Buçinca, Z., Malaya, M. B. and Gajos, K. Z. (2021). “To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making.” Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1). DOI.
- Siebert, L. C. et al. (2023). “Meaningful human control: Actionable properties for AI system development.” AI and Ethics, 3, 241–255. DOI.
- Vaccaro, M., Almaatouq, A. and Malone, T. (2024). “When combinations of humans and AI are useful: A systematic review and meta-analysis.” Nature Human Behaviour, 8, 2293–2303. DOI.
- National Institute of Standards and Technology. (2023). AI Risk Management Framework 1.0 and Playbook. NIST AI Resource Center.