The JSON was valid and the payment was wrong

A model reads a supplier invoice and returns the exact object its schema demands: supplier identifier, currency, amount, tax and purchase-order reference. Every field has the correct type. The workflow accepts the object, calculates a due date with ordinary code and sends a payment request. Only later does an analyst discover that the model combined the subtotal from page one with the tax total from a superseded page.

The failure is easy to misname. The model did not break JSON. The workflow engine did not miss a state transition. The payment interface accepted a valid request. Each component performed its local contract, yet the business outcome was wrong because no component owned the assertion that the extracted values belonged to the same invoice version. A structural guarantee was allowed to impersonate an evidentiary guarantee.

Structured output is valuable precisely because it narrows one class of failure. OpenAI’s own technical account distinguishes conformance to a JSON Schema from mistakes inside otherwise valid values.[14] That distinction should alter architecture. Syntax lets software parse a result. It does not establish provenance, freshness, policy compliance, semantic consistency or authority to commit.

The unit of reliable mixed-logic orchestration is an evidence-bearing state transition. The orchestrator must preserve authoritative state, type every step, test admissibility, assign branch and commitment rights, and leave receipts for effects and recovery. A model may interpret or propose. It must not silently convert uncertainty into durable business truth.

Thought experiment: the coin that becomes a command

Suppose a classifier is right nine times out of ten for a low-stakes internal label. Its uncertainty is tolerable because a person can notice an odd category and nothing outside the system changes. Now connect the same label directly to an access-revocation API. Model quality has not moved, but the consequence of one bit has. The workflow needs identity binding, a policy rule, an authorised actor and a recoverable write before that bit can become a command. Confidence cannot supply any of them.

The insight is broader than human approval. A person is not automatically the missing control. For repetitive low-consequence cases, deterministic assertions and sampled outcome review may outperform case-by-case sign-off. For discretionary or high-consequence cases, the person must receive evidence, see uncertainty, possess genuine authority and have enough time to decide. The orchestration problem is to put the right form of resolution at the transition where uncertainty meets consequence.

Part I · Control planeThe orchestrator owns durable state, not intelligence

Calling a model from a workflow engine does not make the model deterministic. It makes nondeterministic work visible inside a deterministic control system, provided the boundary is real.

This article’s Mixed-Logic Orchestration Plane has eleven services: case admission, authoritative state, a Step Type Registry, Typed Step Contracts, a deterministic control kernel, probabilistic execution cells, admissibility gates, a commitment gateway, recovery control, an execution ledger and lifecycle assurance. The design is vendor-neutral. A durable engine can implement parts of it, but product features do not decide business truth or authority.

Durable-execution systems illustrate a useful separation. Temporal’s architecture, for example, records an append-only history, requires workflow logic to be deterministic and places side effects in activities that are idempotent or non-retryable.[4] That is vendor engineering evidence, not proof that any particular model workflow is safe. It clarifies why replayable control logic and uncertain external effects need different contracts.

The state store contains current facts, permissions, reservations, unresolved conditions and prior effects. A model receives a bounded snapshot and a run identity, not ownership of that state. Its response becomes a proposed transition. The kernel checks that the starting version is still current, applies explicit routing conditions and records why the case moved. The execution ledger stores observable inputs, outputs, decisions and receipts, not hidden reasoning.

One credit case, eleven services

A commercial-credit case enters with authenticated identity, an eligible product and a starting record version. Code calculates policy ratios. A model extracts covenants from signed documents and cites exact pages. Reconciliation compares names, dates, amounts and facilities across authoritative records. A credit officer resolves material ambiguity and owns the decision. An atomic booking service writes the approved facility using an idempotency key. Every service contributes, but none can claim that another service’s condition has passed.

NIST’s AI RMF 1.0 treats task definition, intended use, human roles, testing, monitoring, safe failure and recovery as continuing system duties; the current AIRC page also states that the voluntary framework is being revised.[1] Its Generative AI Profile adds component interaction, pre-deployment testing and incident management.[2] These sources support lifecycle discipline, while the transition mechanics here are this article’s synthesis.

Part II · Transition grammarType the step before choosing the tool

A workflow is not deterministic or probabilistic as a whole. Each transition has a logic type, authority limit and failure grammar.

The Step Type Registry names pure transforms, authoritative reads, deterministic writes, probabilistic interpretations, probabilistic proposals, human judgements, external effects, waits, compensation and terminal states. The label changes design. A read may be retried under freshness rules. A non-idempotent write may not. An interpretation can cite evidence but cannot commit. A human judgement needs a named role, service time and expiry. Compensation needs its own authority and receipt.

BPMN 2.0.2 formalises events, gateways, messages, errors, termination and compensation, which makes it a useful notation vocabulary.[3] A diagram still cannot prove that an implementation preserves versions, runs the declared checks or reconciles an external effect. The registry joins the visual process to runtime contracts.

The Typed Step Contract goes beyond a schema

Every transition receives an identifier, type, version, owner, purpose and permitted population. It binds the input schema to an authoritative state version and evidence manifest; names the authenticated principal, operation, object and environment; defines either a deterministic invariant or bounded probabilistic task; and declares outputs, claims, assertions, budgets, side effects, failure states, retry class, receiver, receipt and reauthorisation trigger. A schema sits inside that contract. It is not the contract.

Negative control: the no-model invoice

An invoice arrives through a trusted structured channel. Supplier, order, receipt, tax treatment and tolerance rules are complete. The same inputs and policy version always produce the same posting decision. Inserting a model would add variability, evaluation work and another failure surface without resolving ambiguity. The registry selects deterministic reads, transforms and an idempotent write. This negative control matters because an architecture that always routes through AI is a product preference, not orchestration discipline.

Part III · AdmissibilityPut uncertainty inside a probabilistic sandwich

Deterministic preparation establishes what may be considered. A bounded model step interprets or proposes. Independent gates decide whether the result can influence state.

The Probabilistic Sandwich Pattern begins with authenticated admission, eligibility, consent, duplicate detection and a starting version. Approved sources become an evidence manifest. The model performs one named task under a run identity, fixed tools and budgets. Structural, evidentiary and semantic gates test the result. An authority gate chooses among declared routes. Only then can an authorised action create an effect receipt, followed by reconciliation against business state.

This pattern does not require a person after every model call. A service agent may classify intent and draft a remedy, while a deterministic entitlement service checks account, product, amount ceiling and duplicate status. A small reversible credit can proceed through a narrowly authorised write. The model’s role is still probabilistic, but the consequential claims are independently established.

The assertion lattice preserves hard stops

The Admissibility Assertion Lattice returns pass, fail, unresolved or not applicable for syntax, case binding, evidence, freshness, permission, arithmetic, semantic consistency, uncertainty, policy protection, authority, duplicate safety and downstream preconditions. Each answer carries evidence and an owner. The lattice is deliberately not a score. Perfect formatting cannot compensate for a permission failure; strong citations cannot make a stale state current; model agreement cannot grant decision rights.

Consider a procurement proposal that recommends a supplier using a claim for which it provides no source. The evidentiary gate does not ask the model to regenerate until a citation appears. It identifies the missing evidence, requests it from an approved source or routes to a person who can decide without that claim. Repetition may create consistency, but consistency without an external truth signal is not verification.

Thought experiment: three agreeing models and one missing fact

Three independently prompted models recommend the same supplier. Their agreement appears stronger than one answer, but all three used the same incomplete tender pack, which omitted a sanctions update. Add more model votes and consensus rises while truth does not. Now add one authoritative sanctions query and the recommendation changes. Parallel proposals can expose alternatives or reduce latency; they do not become independent verification when their evidence and blind spots are shared.

The Branch Authority Protocol confines models to declared routes. Deterministic conditions select deterministic branches. A model may rank allowed candidates, while no-match, multi-match, ambiguity and stale state remain explicit outcomes. Consequential branches require an independent rule or authorised person. Rejected alternatives and their evidence are retained only to the level needed for audit, privacy and challenge.

Part IV · Failure semanticsRetry the failure class, not the step name

A retry is another state transition with cost and consequence. It is safe only when the failure class, side-effect certainty and remaining budget permit it.

A transient read failure may justify bounded retry with exponential backoff and jitter. Throttling may need delay under a shared budget. Malformed output may permit one constrained repair. Unsupported content should request specific evidence or change route. Policy denial stays terminal until the case or authority changes. Stale state requires re-admission. An unknown write requires reconciliation before any new attempt. Human non-response becomes expiry, not silent consent.

AWS Step Functions documents named errors, retries, catches, timeouts, backoff and jitter as explicit state-machine behaviours.[5] AWS Well-Architected guidance explains request tokens and idempotent responses.[6] Google’s SRE account shows why retries can amplify overload and why backoff, jitter and load shedding matter.[7] These mechanisms become business-safe only when paired with effect certainty and authority.

The payment timeout with an unknown world state

A payment service times out after receiving a request. The workflow cannot infer that the payment failed, and it cannot infer that it succeeded. The kernel records an unknown-effect state tied to the original idempotency key. A reconciliation query checks the service or ledger. If a durable receipt exists, the workflow advances without repeating the payment. If confirmed absent, a bounded retry may be safe. If the service cannot answer, the case remains quarantined for repair.

The same discipline prevents nested retry storms. If a model client retries twice, an activity layer retries three times and a workflow retries the activity four times, one outage can create a multiplication of calls. The Probabilistic Execution Budget covers attempts, candidates, tokens, tools, time, spend, evidence requests, downstream calls and cumulative consequence across layers. A single kernel spends the budget and opens the circuit before the system turns one failure into a capacity event.

Part V · Commitment and repairCompensation moves forward; it does not erase history

A consequential workflow needs a map of compensable actions, pivot points, retryable forward work and effects that cannot be undone.

The Commitment and Compensation Map records the object, operation, recipient, environment, actor, preconditions, reversibility, atomicity, idempotency, receipt, reconciliation query, retry prohibition, repair route and point of no return for each effect. Long-lived transactions often cannot use one atomic database commit. The Sagas paper introduced decomposing such work into local transactions with compensating actions after partial execution.[18]

Cloud architecture guidance makes the practical caveat explicit: compensation is application-specific, must account for concurrent work and may require manual intervention.[8] The saga pattern distinguishes compensable, pivot and retryable transactions and exposes the trade-off between orchestration and choreography.[9] A refund does not unsend a disclosure. Revoking access does not erase what was seen. Cancelling a booking may charge a fee and disappoint a traveller. Recovery establishes a valid forward state.

Applied example: onboarding wakes into a new policy

An onboarding flow pauses while a candidate supplies evidence. During the wait, policy changes. The returning signal carries the old correlation identifier, but the kernel compares the stored policy and case versions with the current ones. The stale branch expires. Admission rebuilds the evidence set and re-evaluates eligibility. The model’s earlier summary remains historical evidence, not a current decision input. The person is told what changed rather than asked to approve an invisible version conflict.

Human decisions are states, not inboxes

A human gate specifies the decision object, evidence pack, unresolved question, permitted options, challenge route, service target, expiry and alternative path. Two parallel model branches that propose incompatible fraud actions do not race to a write. Their evidence packs enter a declared adjudication state. The investigator may choose, seek more evidence or abstain. If time expires, the workflow applies an interim protection or closes safely; non-response never becomes approval.

The EU AI Act creates jurisdiction-specific logging, information, deployer and human-oversight obligations for covered systems on a phased timetable.[11] ISO/IEC 42001:2023 provides an organisational management-system frame for objectives, operation, performance evaluation and improvement.[12] Neither source dictates this orchestration plane globally. They reinforce the need to connect technical events to named organisational responsibilities where they apply.

Proportionality and limitsThe strongest rival is unnecessary machinery

A typed control plane can become an expensive distributed system wrapped around a useful model call.

That objection is serious. Durable engines, validators, human queues, traces and compensation add latency, cost and change burden. In a low-consequence advisory task, a user may inspect the output and try again. Some model work cannot be independently checked without redoing it, and rigid gates can reject novel but valuable answers. Choreography can also avoid a central bottleneck when services are simple, loosely coupled and own their local outcomes.

The answer is proportionality, not universal orchestration. Keep known logic in ordinary code. Use one bounded model call when one interpretation is sufficient. Add durable coordination for long-running, multi-system, failure-prone or consequential work. Prefer sampled or outcome-based review when case-by-case approval adds little. Anthropic’s practitioner guidance similarly recommends beginning with the simplest pattern and distinguishes predefined workflows from model-directed agents.[13] It is vendor guidance, not comparative proof.

OpenTelemetry’s GenAI semantic conventions offer useful vocabulary for model operations, tools and tokens, but the relevant attributes are currently marked Development.[10] A trace should record what happened without dumping protected prompts by default. It is evidence for diagnosis, not an explanation of correctness. ReAct demonstrates interleaved reasoning, action and observation in studied tasks,[15] while AgentBench documents long-horizon, decision and instruction-following failure modes across its benchmark environments.[17] Neither authorises a production side effect.

Explicit failure boundary

Stop unsafe orchestration when no authoritative state can be identified; when evidence cannot be bound to the case; when a semantic claim cannot be checked or assigned to a qualified decision-maker; when branch or commitment authority is absent; when an external effect cannot be distinguished from an unknown timeout; when compensation has no workable owner; or when the ledger cannot reconcile the business state. The correct terminal state may be abstain, reject, partial completion, quarantine or manual service. Completion is not compulsory.

LLMCompiler reports benefits from separating planning, task fetching and parallel execution in its evaluated function-calling tasks.[16] Parallelism can reduce latency, but it also creates branch races, shared-evidence blind spots and cumulative budgets. Parallel proposals should synchronise at a deterministic adjudication point. A majority of models is not a substitute for an external fact, and the first branch to finish does not gain authority by speed.

Twelve worked routesMake ordinary and broken states equally designable

The following compact cases type the decisive transition and show where the workflow must route. They are worked design examples, not client deployments.

Executable decision instrumentMixed-Logic Orchestration Designer

Load a named scenario or change the control conditions. The instrument returns one of five routes and exposes the state, checks, authority, failure semantics and receipts behind it.

How to use the result

The route is an architecture hypothesis, not approval to deploy. Write contracts for every probabilistic, human and consequential node; exercise ordinary, stale, conflicting, timeout and partial-effect states; reconcile receipts to authoritative records; compare the non-AI route; and confirm that operations can staff dead-letter, adjudication and repair queues. The instrument cannot establish local model performance, legal compliance, fairness, human effectiveness or economic value. Those require representative tests and accountable decisions.

ConclusionLet uncertainty participate without inheriting the workflow

A reliable mixed-logic workflow does not demand certainty from a model. It demands honesty from the architecture. Rules remain rules. Interpretations remain claims. Proposals remain proposals. Human discretion is a declared state. Writes expose their atomicity and duplicate behaviour. Timeouts preserve uncertainty. Compensation acknowledges that the world moved. Every transition leaves enough evidence to decide what can happen next.

The Evidence-State Transition Ledger is the practical memory of that honesty: case and correlation identifiers, prior and proposed versions, step type and contract, principal and permission, evidence snapshot, configuration versions, assertions, selected route, attempts, budget balance, effects, idempotency, acknowledgements, recovery and final state. It records externally observable control facts, not private model reasoning. Monitoring should keep those views separate rather than compressing them into a cheerful health score.

The design decision is therefore precise: keep the state machine deterministic, place probabilistic work inside typed cells, make admissibility independent, give commitment its own authority, and engineer failure as carefully as success. When that control plane would cost more than the consequence warrants, simplify it. When no safe state, truth signal, authority or recovery path exists, stop. Mixed logic becomes dependable only when none of its parts is allowed to pretend it is another.