The moment a suggestion becomes a power

Consider a calendar assistant. It notices eleven meetings whose titles look duplicated and says, with 93 per cent confidence, that they should be removed. The number may be well calibrated. It still says nothing about whether the assistant may delete events, whether the duplicates were intentional, whether guests have accepted them or whether the user delegated cleanup rather than irreversible cancellation. Epistemic confidence and operational authority answer different questions.

A language model estimates a useful continuation under a context. An authority decision states that a particular principal may perform a particular action on a particular resource under current conditions. The first is naturally probabilistic. The second is normative and state-dependent. Mixing them creates a quiet category error: a plausible plan begins to look like a permitted plan merely because both are represented in the same text or tool-call object.

Model alignment does not remove this distinction. A well-trained model may usually refuse prohibited actions, cite policy and ask for clarification. Those are desirable behaviours, but they remain behaviours of a component whose outputs can change with prompt phrasing, retrieved content, sampling, model update or attack. A prohibition enforced only by the model is therefore a prediction about what it will choose. A prohibition enforced at the tool boundary is a constraint on what the system can do. Safety-critical authority should survive a model swap, a decoding change and a hostile context.

This also explains why a model’s rationale cannot be the sole authorisation evidence. The rationale is generated by the same process as the proposal and may be post-hoc, incomplete or strategically shaped. It can help a reviewer understand the intended plan, but it cannot attest that an operator is still entitled, that a beneficiary is unchanged or that a required approval exists. Those facts need independent issuers and stable identifiers.

A minimal causal model

Let the model produce a proposal p from instructions, context and its learned parameters. Let a separate kernel evaluate that proposal against identity i, authoritative world state s_t, policy version π and evidence e. If permitted, the kernel emits a capability c. A tool gateway executes only when the capability validates against the requested action and a fresh state check.

Proposal p

A typed recommendation, not a credential and not an effect.

Kernel K

A versioned decision function whose inputs and result can be replayed.

World state s_t

The authoritative facts at the decision cutoff, not the model’s recollection.

Capability c

A narrow, expiring grant that names the exact action and constraints.

Validator V

The enforcement check at the tool boundary, including freshness and consumption.

Take the supplier payment again. The model proposes “transfer £4,200 to Vendor-17” with confidence 0.91. The kernel sees a treasury manager authenticated with a hardware key, an existing beneficiary, two matching evidence items, an amount below the automatic threshold and state read eight seconds ago. It issues a capability for that exact vendor, amount and invoice, valid for sixty seconds and usable once. Change only the beneficiary from existing to new. The model output remains identical, but the kernel denies execution. The intervention isolates authority from model belief.

What “deterministic” means here

The kernel need not be static, simplistic or ignorant of uncertainty. It may consume a fraud score, anomaly alert or identity confidence. The requirement is narrower: for a fixed set of signed inputs, a fixed policy version and a fixed state snapshot, it should return the same decision and obligations. Any probabilistic score becomes an explicit input with provenance, not an invisible mood inside the authoriser.

This definition makes a decision replayable. An investigator can ask which state, policy and evidence produced the grant. A tester can vary one input and observe the boundary. A policy analyst can check whether a refactor widens authority. Cedar, for example, separates authorisation policy from application logic, uses default deny and supports formal analysis of policy behaviour.[4] The older security principles are even more direct: fail-safe defaults, complete mediation, least privilege, separation of privilege and economy of mechanism.[1]

The word kernel is deliberate but limited. It means the small trusted decision and enforcement base on which authority depends, not an operating-system kernel and not a claim of formal verification. The seL4 work demonstrates why small trusted mechanisms matter: sufficiently constrained code can be specified and checked far more deeply than a sprawling application.[7] Smallness is valuable because it makes the authority logic inspectable, testable and harder to bypass.

flowchart TD
  A["The invoice that passed every<br/>language check"] --> B["The moment a suggestion<br/>becomes a power"]
  B --> C["Authority should be issued,<br/>narrowed and consumed"]
  C --> D["Build the effect path so the<br/>model cannot bypass it"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#D95545,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Authority should be issued, narrowed and consumed

This experiment explains why prompt guardrails, model self-critique and policy reminders are not authority controls. They may improve the proposal distribution. They do not make a forbidden state transition impossible. Current agent guidance similarly recommends structured extraction, tool approvals and isolation, while warning that these techniques reduce rather than eliminate prompt-injection risk.[9] The architectural question is not whether the model has heard the rule. It is whether every effect encounters an enforcement point that cannot be skipped.

The kernel’s minimum evidence

An authorisation check becomes useful when it binds the proposed effect to facts that are more than asserted by the proposal. In a consequential system, the kernel should receive at least the following kinds of input. They can be implemented with existing identity, policy and system-of-record services rather than invented as model “memory”.

NIST’s zero-trust architecture separates a policy decision point from a policy enforcement point and insists on granular, per-request authorisation close to the resource.[2] Zanzibar shows another relevant property: authorisation decisions can respect causal ordering as permissions and object contents change.[5] An agent architecture should inherit both lessons. The grant must be based on the right state, then enforced where the effect occurs.

One asymmetry is especially useful: new information may improve a proposal, but it must not create new authority. Retrieval can reveal a changed address, an urgent message or a previously unseen instruction. It can support a recommendation. It cannot enlarge the user’s delegation, manufacture a role or satisfy a dual-control obligation merely by saying that approval occurred. Authority-bearing facts should come from named issuers such as identity services, policy stores, approval records and systems of record.

Consider a customer-service agent asked to send a case summary to an external address. A retrieved email claims that the customer has consented. The model may use that email as evidence to propose an export. The kernel should still check the consent registry, the permitted fields, the recipient domain, the purpose and the case identifier. If the consent registry has no matching record, the correct outcome may be “require verification”, even when the email looks authentic. This is not a judgement that the email is false. It is a judgement that the email is not an authorised source for that permission.

Four decisions are better than one bit

Binary allow or deny is often too crude for agent work. A useful kernel should return one of four outcomes: deny, narrow, require approval or permit. Narrowing is especially important. It lets the system preserve useful work without accepting the model’s requested authority. A clerk who proposes “submit payment” may receive a capability only to “save payment draft”. An assistant that asks to export a complete case file may be limited to three authorised fields for one named case.

This is not the model voluntarily restraining itself. The returned capability contains less power than the proposal requested. Capability systems have long used attenuation to delegate a subset of authority. Macaroons, for example, support contextual caveats that confine when, where, by whom and for what purpose a credential may be exercised.[6] The same pattern fits agentic execution: exact resource, exact action, exact upper bound, short expiry, one use and a purpose binding.

A typed action is more than valid JSON

A schema can prove that a field named amount is numeric. It cannot prove that a generic call such as run_sql(query) means “update one invoice status” rather than “drop the payments table”. The action contract must expose the semantics that policy needs to constrain. Prefer mark_invoice_reviewed(invoice_id, expected_version) to execute_database_command. Prefer submit_payment(payee_id, amount, invoice_id, idempotency_key) to call_api(url, body).

Tool design therefore sets the resolution of authorisation. If a tool bundles several effects, the kernel can only allow or deny the bundle. If a tool accepts unrestricted code, the kernel would need to understand arbitrary program behaviour, which defeats the premise. Narrow tools are not cosmetic API hygiene. They are the vocabulary in which enforceable authority can be stated.

Human approval is an obligation, not a substitute

A person may be required to approve a high-impact action. That does not remove the need for the kernel. The system must still decide which actions require approval, present decision-sufficient evidence, bind the approval to the exact proposal and prevent later parameter changes. Otherwise “approved” becomes a reusable feeling rather than a cryptographic or transactional fact.

Human review also degrades under volume. Anthropic reports that users approved roughly 93 per cent of repeated permission prompts in one product context and argues for containment that limits what an agent can do rather than relying only on turn-by-turn supervision.[10] That telemetry is not universal evidence, but it exposes a general mechanism: frequent low-information approvals train people to click. The kernel should spend human attention only where policy cannot safely resolve the case.

Build the effect path so the model cannot bypass it

The architecture is easiest to understand as two planes. The proposal plane contains the model, retrieval, planning and critique. It may be complex and probabilistic. The authority plane contains identity resolution, policy evaluation, capability issue, enforcement, receipts and recovery. It should be comparatively small. The model receives tool descriptions, not ambient credentials. The enforcement gateway, not the model runtime, holds the ability to invoke consequential operations.

A complete effect path

A production implementation can follow seven steps. The user’s objective first becomes an intent record with principal, purpose, constraints and expiry. The model receives only the context permitted for that intent. It produces a typed proposal without credentials. A schema validator rejects malformed fields but does not authorise semantics. The kernel resolves identity, current entitlements, world state, evidence and policy. It records a decision, and when permitted issues a capability. The policy enforcement gateway validates and consumes that capability before calling the tool. Finally, the system reads back the effect and stores an outcome receipt.

This sequence separates three things often collapsed into “tool use”: a model proposal, an authority decision and an executed effect. It also separates decision receipts from effect receipts. A tool may acknowledge a request yet fail to change state. A network retry may execute twice unless an idempotency key is bound to the capability. A downstream system may accept a value but normalise it unexpectedly. Authorised execution is not verified outcome.

A useful decision receipt is closer to a transaction manifest than to a chat transcript. It should identify the principal and delegator, the proposal hash, action type, resource, policy version, state versions, evidence references, decision, obligations and capability identifier. It should also record which inputs were authoritative and which were model-supplied. That distinction lets an auditor reconstruct the decision without treating every token in the prompt as equally trusted.

Replay then becomes an engineering instrument. Hold the proposal constant and replace state v41 with v42 after a role revocation. The kernel should change from permit to deny. Hold state constant and replace policy v17 with v18 after a threshold reduction. The expected decision should change in a documented way. Hold both constant and swap the underlying model. The authority result should not change at all. These counterfactual replays test the claimed separation rather than merely checking that a happy-path run completed.

Before live execution, the same path can operate against an operational twin or dry-run adapter. The kernel issues a capability whose audience is the simulator, not the production gateway. The model receives the predicted effect and can revise its proposal, but no production credential is exposed. When the action is later promoted, a fresh decision must be made against live state. A successful simulation is evidence about the action contract, not a transferable permission to execute.

The arithmetic is simple. If N is the number of proposals, q is the unsafe-proposal rate and e is the probability that an unsafe proposal escapes the kernel, expected harmful executions are:

This is an accounting identity when e is defined conditionally on an unsafe proposal. It is not a safety guarantee. The critical question is whether the kernel’s failure modes are genuinely different from the model’s. If both rely on the same untrusted text, the same classifier or the same stale database, their errors may share one cause and e will remain high.

The adversary is often a confused deputy

An agent commonly sees text from users, files, websites, messages and other agents. Some of that content is evidence. Some is instruction-shaped data. If the model can exercise the user’s broad credential, malicious content can recruit it as a deputy. OWASP’s agentic risk framework names goal hijacking, tool misuse, and identity and privilege abuse among the central threats.[8] The kernel responds structurally: the document may influence a proposal, but it cannot grant identity, change delegation or widen a capability.

function evaluateProposal(input) {
  const now = new Date().toISOString();
  const deny = (reason) => ({ decision: "DENY", reason, at: now });

  if (input.stateAgeSeconds > 30) {
    return deny("Authoritative state is stale; refresh and re-evaluate.");
  }

  if (input.action === "read_invoice") {
    return issueCapability(input, "read_invoice", { ttlSeconds: 300 });
  }

  if (input.action === "submit_payment" && input.principal === "ap_clerk") {
    return {
      decision: "NARROW",
      reason: "Clerks may prepare, not submit, payments.",
      allowedAction: "save_payment_draft",
      at: now
    };
  }

  if (input.action === "submit_payment") {
    if (input.principal !== "treasury_manager") return deny("Role lacks payment authority.");
    if (input.identity !== "hardware") return deny("Hardware-backed identity required.");
    if (input.payee !== "existing") return deny("New or changed beneficiary requires separate verification.");
    if (input.evidenceCount < 2) return deny("Two independent evidence items required.");
    if (input.amount > 10000 && !input.approval) {
      return { decision: "REQUIRE_APPROVAL", reason: "Dual approval required above £10,000.", at: now };
    }
    return issueCapability(input, "submit_payment", {
      ttlSeconds: 60,
      maxAmount: input.amount,
      uses: 1
    });
  }

  return deny("No policy permits this action in the demonstration.");
}

stateDiagram-v2
  [*] --> Claim
  state "A minimal causal model" as Claim
  state "What “deterministic” means here" as Test
  state "The kernel’s minimum evidence" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

The kernel’s strongest boundary is semantic

The architecture fails if the policy cannot describe what matters. Suppose each transfer below £1,000 is allowed without approval. An agent submits one hundred transfers of £999 to the same beneficiary. Every individual action satisfies the single-call rule. The trajectory violates the purpose. The kernel did exactly what it was asked to do and still enabled harm.

This does not overturn the thesis. It specifies the state the kernel must see. A per-action kernel can enforce per-action constraints. Sequence-level policy needs a trajectory ledger and atomic counters. Cross-system effects need a shared or federated state. Irreversible actions need stricter obligations and recovery semantics. The kernel remains the place where those constraints become enforceable, but its assurance is no stronger than the action vocabulary and world state supplied to it.

Five ways the boundary can fail

First, policy may be wrong or incomplete. Second, the system of record may be stale, forged or semantically ambiguous. Third, a broad tool may hide the real effect. Fourth, a bypass path may let the model or another service reach the tool directly. Fifth, individually permitted actions may compose into a forbidden trajectory. Each failure has a different remedy: policy analysis, state provenance, narrower tools, network and credential containment, or sequence-aware constraints.

Policy evolution creates another boundary. A field called beneficiary_verified may once have meant that bank details were checked by a person. A later service may set it after an automated database match. The name is unchanged, but the assurance has weakened. Versioning only the rule is insufficient; the semantics and issuer of every authority-bearing attribute must also be versioned. Otherwise replay produces the same Boolean decision while relying on a different real-world claim.

There is also a legitimate alternative. For low-impact, reversible work with no external effect, a separate kernel may add more complexity than value. A writing assistant can generate ten versions of a paragraph without an authorisation service. A sandboxed coding agent may edit an isolated branch under filesystem and network containment. The boundary becomes necessary when a proposal can cross into another person’s data, money, rights, commitments or shared operational state.

The strongest practical objection is cost. A separate decision service adds latency, policy maintenance, state reads and failure modes. That objection is real. The answer is not to mediate every token or harmless draft. It is to place the boundary at effects and to make permits short but cacheable only within their exact scope. Read-only access can use longer, resource-bounded capabilities. Irreversible writes can require fresh state. Batch operations can be authorised as a typed batch with a total limit rather than thousands of unrelated prompts. The design should spend control effort in proportion to consequence.

A second objection is that business policy is rarely complete enough for deterministic rules. True. The kernel does not have to settle every case. Its decision language includes obligations and abstention. Where facts are ambiguous, it can narrow the action to a draft, request a named reviewer or deny until a missing record is supplied. Determinism applies to the routing decision given the inputs, not to the claim that all organisational judgement can be reduced to code.

Another alternative is to use a probabilistic model as the authoriser because natural-language policy is difficult to formalise. That model can be useful as a sensor or triage layer. It should not silently create permission. When policy meaning is too ambiguous for deterministic resolution, the deterministic result can be “require qualified review”. This preserves a clear fact: uncertainty routed the case to judgement; it did not itself become authority.

sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: The invoice that passed every language check
  M->>E: A minimal causal model
  E-->>M: compare rival explanations
  E->>D: Keep proposals stochastic and grants explicit
  D-->>Q: revise scope or act