03 · Three-View Process Evidence MapReconcile what is said, stored and done

A procedure can be internally consistent and still be false as a description of work. Logs can be precise and still omit phone calls, judgement and workaround. Observation can be vivid and still over-weight memorable episodes.

Use three views together. The declared view contains policy, standard operating procedures, training and managers’ accounts. It tells us what should happen and provides normative constraints. The recorded view contains timestamps, state changes, queue histories and system entries. It describes what selected systems can see. The enacted view is the work people actually perform, including side channels, copied notes, tacit sequencing and recovery. It reveals why the process survives despite gaps in the other views.

Disagreement is useful evidence. If policy requires same-day review, logs show three days and observation reveals a nightly export, the task is not to vote for one truth. The discrepancy identifies an access, capacity or system-design problem. Model selection should wait until the team knows which requirement is binding and which behaviour is adaptation.

The Process Understanding Sufficiency Test

Before selecting a model, answer twelve questions in plain language: What outcome matters? What starts the case? What ends it? Which case types and exceptions exist? What work and handoffs occur? Where are decisions made? Who has authority? Which information is authoritative? Which controls and recovery routes apply? What are the current distributions of time, quality and rework? What consequence follows a wrong action? Which technology-neutral capability would change the result? “Unknown” is acceptable only when paired with a bounded investigation and an owner.

Sufficiency is proportional to exposure. A reversible marketing draft may need a handful of representative episodes, a quality rubric and a named approver. A fraud intervention that can freeze access needs population analysis, appeal routes, subgroup evidence, authority limits and credible monitoring. The test prevents two symmetrical errors: analysing indefinitely and choosing prematurely.

04 · Work primitive decompositionBreak the verb before buying the engine

Broad labels such as “handle the claim” or “write the memo” hide parts that require different evidence, controls and technology.

Decompose the work into twelve primitives: sense a trigger; acquire evidence; authenticate identity or source; retrieve relevant material; transform format; calculate; classify; compare against a rule; predict an uncertain state; generate an artefact; decide or recommend; and act in a system. A real event can contain several primitives, but each has a different success criterion. Calculation might require exact repeatability. Retrieval requires coverage and citation. Generation may require factual grounding and tone. Action requires permissions, transaction integrity and recovery.

The bank memo that was not one task

A bank asks vendors to demonstrate “write the credit memo”. Observation reveals at least seven primitives: authenticate financial statements, retrieve facility history, calculate ratios, classify covenant status, identify missing evidence, generate a concise rationale and recommend a route. Arithmetic should not be probabilistic. Policy comparison should cite the governing version. The recommendation belongs within delegated authority. Once decomposed, the likely design is a stack of reliable records, deterministic calculations, controlled retrieval and a bounded drafting component. Comparing large models on prose preference before resolving those parts tests the least consequential layer first.

Decomposition also reveals where no automated component should act. A case may require a relationship manager to interpret a contradictory account from a customer whose circumstances fall outside precedent. A model might organise evidence and surface analogous cases, but the institutional judgement remains human. “Human in the loop” is too vague. The contract should state which primitive the person performs, what evidence they receive and whether they approve, correct, investigate or bear accountability.

05 · Model Necessity GateChoose the smallest justified capability class

The gate asks not “Which model wins?” but “What is the least complex intervention that satisfies the work-event contract?”

Start at level zero and move only when evidence eliminates the simpler level. Removal or redesign can make the activity unnecessary. Ownership or policy clarification may make the process operable. Deterministic rules can handle explicit stable conditions. Conventional search and software can organise known records. A narrow statistical model can rank or estimate a defined target. Generative capability can transform or compose under grounding controls. An agentic loop is justified only when multi-step adaptation and tool use are necessary and governable. Some work should remain unautomated.

Allocate discretion by consequence and uncertainty

Uncertainty alone does not determine automation. Low-uncertainty, low-consequence work can often run automatically under monitoring. High-uncertainty, low-consequence work may allow generation with sampled review. Low-uncertainty, high-consequence work favours deterministic controls, dual checks and clear appeal. High-uncertainty, high-consequence work calls for assistance rather than autonomous disposition: the system assembles and challenges evidence while an authorised person decides.

Consider suspected fraud. The model can retrieve linked events, cluster anomalies, draft competing hypotheses and identify missing evidence. It should not silently freeze a customer because a composite score crosses an opaque line. The work-event contract assigns the consequential action to a trained analyst, defines urgent escalation and records the grounds for intervention. Capability is still valuable, but discretion is bounded at the correct primitive.

06 · Process-to-Capability ContractSpecify the job without naming the product

A good contract describes observable performance in the operating context. It allows several architectures to compete and makes “no qualifying option” a legitimate result.

Write one contract for each bounded work event. Name the eligible case population, input evidence, required output, forbidden behaviour, operating threshold, latency, volume, integration, authority, review and recovery. State which party experiences success or failure. Keep vendor names and model families out. “Use a leading language model to summarise calls” is a procurement preference. “Within thirty seconds of call completion, produce a draft record that captures every mandatory field, distinguishes quotations from inference, cites the source interval and routes consent-sensitive content for review” is a capability requirement.

The contract should include distributions, not only averages. A mean time saving can coexist with severe delays for complex cases. Overall accuracy can hide systematic failures in rare but consequential categories. Minimum slice performance, abstention behaviour and time to recover can be more decision-relevant than a single composite score. Where the current process is weak, the baseline must be retained rather than silently replaced by an aspirational description.

Do not confuse stack choice with model choice

Many process requirements are satisfied by combinations. Stable document extraction may need a schema validator, optical character recognition and a specialist parser, with a small model only for ambiguous fields. Evaluation should compare at least one deterministic baseline, one specialist component and one modest general model on the same representative documents. A large model deserves selection only if its incremental performance clears the operational and economic threshold after review cost, latency, security and failure recovery are included.

Likewise, multilingual customer contact should not be collapsed into a single “language quality” score. The process needs language identification, retrieval from authorised material, accurate interpretation, culturally appropriate explanation and escalation when meaning is uncertain. Each supported language needs representative cases and affected-user review. A system that excels in English and deteriorates on the languages used by the most vulnerable customers fails the capability contract even if its average score is high.

Evidence proportionality

Contract depth should match consequence, not organisational enthusiasm. A small business automating invoice triage may use a one-page version: eligible invoices, required fields, tolerance rules, manual exceptions, weekly error sample, owner and disable switch. This can be rigorous without becoming bureaucratic. The same template expands for regulated or irreversible decisions. Proportionality preserves the discipline while keeping it usable.

07 · Benchmark Translation ProtocolMake every score answer an operating question

A benchmark is useful when its cases, measures and thresholds trace to a real process claim and an owner’s decision.

Translate in eight linked steps: process claim, work event, capability requirement, test case, system measure, operating threshold, downstream outcome and owner decision. The chain prevents a common substitution. The team wants fewer unresolved claims, but evaluates polished summaries; it wants correct benefit decisions, but measures similarity to a reference answer; it wants faster service, but ignores review time and rework. Each link asks whether the next remains a credible proxy for the previous one.

A claims average that concealed the decision

A claims team reports 93 per cent classification accuracy. The figure combines routine document types with rare cases involving disputed liability, possible fraud and urgent vulnerability. Most errors occur in the rare group, and each misroute delays a specialist intervention. The benchmark translation starts from the process claim: route cases early enough for the correct specialist action. It samples by consequence, measures critical-case recall and time to authorised review, and sets a zero-tolerance condition for prohibited automatic closure. The same model now has a different, decision-relevant evaluation.

A second thought experiment: swap the label

Suppose the vendor identity and architecture are hidden. Evaluators receive only process-level results, review burden, worst-slice performance, failure traces, cost and recovery evidence. Would they make the same choice? If preference changes when the model name disappears, part of the decision was status or familiarity rather than capability. Blind comparison cannot remove every bias, but it exposes whether the docket can stand on evidence.

Benchmarks also need expiry. Case mix, policy, user behaviour and upstream data can change. Record the period represented, known exclusions and the condition that triggers retesting. A threshold is not a permanent certification. It is a dated claim that a particular component, configuration and control system met a particular process need under observed conditions.

08 · Ten named casesThe gate produces different answers for different work

These cases are not miniature success stories. Each illustrates the diagnostic move that changes the technology decision.

The strongest rival: learning by trying models

The strongest objection is that process analysis can delay learning, formalise an obsolete workflow and prevent teams from discovering capabilities they could not specify in advance. Emergent work may leave little reliable event data. A short model experiment can reveal new ways to organise a service, not merely automate today’s steps. This rival is right when “understanding” becomes exhaustive documentation or when the current process is mistaken for a design constraint.

The answer is minimum sufficient understanding plus disciplined probes. Treat a reversible experiment as a process inquiry. Give it a clear case boundary, forbidden actions, observation plan and expiry. Ask what new work appeared, which decisions moved, which controls became necessary and whose outcome changed. The probe can challenge the current design without turning vendor behaviour into an unexamined operating model. Evidence informs imagination; it does not require preserving every legacy handoff.

Where this method ends

This architecture does not perform process mining, design a zero-based future process or manage a later cutover. Process mining can deepen the recorded view once event semantics and boundaries are understood. Zero-based redesign can begin after the current purpose, constraints and outcomes are sufficiently evidenced. Migration and retirement require separate operational decisions. Here the narrower task is upstream: prevent a technology choice from solidifying before the work and capability claim are coherent.

09 · Interactive decision designerTest the intervention before naming the model

Use the designer to express the process condition. Its five outcomes are operating decisions, not maturity levels.

10 · Model Choice DocketBind late, record why, reopen on evidence

The docket is the compact decision record that connects process evidence to selection, conditions and expiry.

Record the work-event contract, open evidence conflicts, rejected simpler interventions, representative evaluation set, thresholds, affected-party results, control design, total operating cost and named owner. State why the selected component is necessary relative to the best non-model baseline. List dependencies and the route if the component becomes unavailable. Give the decision a review date and explicit reopening triggers, such as policy change, population shift, elevated overrides or unacceptable recovery time.

The Late-Binding Rule is simple: do not commit to a model class, vendor or autonomous scope until the capability contract and necessity gate are stable enough to make comparison decision-relevant. Binding can occur separately for different primitives. A team may confidently choose deterministic calculation while continuing to investigate a drafting component. It may procure retrieval infrastructure while withholding action permissions. This modular timing reduces lock-in and makes disagreement visible.

Late binding is also economic discipline. The relevant cost includes integration, review, monitoring, security, appeals, incident recovery, vendor change and the opportunity cost of process redesign deferred. Apparent model savings often assume that human checking is free or that exceptions disappear. The docket forces those claims into the same denominator as the baseline.

What a selection meeting should sound like

The discussion should begin with the outcome and case distribution, not a product demonstration. Participants should be able to point from a failed case to the work primitive, source evidence, governing threshold and accountable decision. They should compare the proposed architecture with process repair, a rule, conventional software and a narrower model. They should know which result would cause them to stop. When this language is present, technical expertise becomes more useful because it is answering a real operating question.

Understanding the process before choosing the model is not a plea for slow analysis. It is a demand that experimentation has a unit, evidence has a consequence and scores have an owner. The smallest justified capability can then enter a process designed to use it, challenge it and survive without it.