Human–Machine Convergence

Model-independent containment and property placement

Rajesh Ranjan Mahapatra

A first-principles account of which agent safety properties can be enforced outside the model, where to place them, and how to test a typed containment boundary.

© 2026 Rajesh Ranjan Mahapatra. 31 min read

Model-independent containment and property placementAll Human–Machine Convergence
Rajesh R Mahapatra Human and machine intelligence

Security, provenance and oversight

Model-independent containment and property placement

An agent can be treated as adversarial without becoming useless when every consequential effect crosses a typed, fresh and completely mediated boundary that owns the authority to deny it.

TLDR

  1. The model is a proposal engine, not a security boundary.
  2. A property becomes model-independent only when the boundary can observe it and mediate every relevant effect path.
  3. Capabilities must bind actor, subject, purpose, action, resource, limits, provenance and time.
  4. Authorisation must remain fresh through execution, receipt creation and independent readback.
  5. Semantic correctness, covert channels and cumulative harm remain outside a single boundary’s guarantee.

The invoice that politely asks to steal

A payables agent opens an invoice from a familiar supplier. The PDF contains the expected amount, purchase order and delivery note. Near the bottom, in white text, it also contains a sentence addressed to the agent: ignore the bank account in the vendor record and transfer the money to a new account shown below. The model reads both. It produces a perfectly formed tool call for £4,870.

The ordinary response is to make the model more cautious. Add a stronger system prompt. Train it on more attacks. Ask another model to review the first model. Insert a confirmation dialogue. Each measure may improve behaviour, yet all leave the decisive fact unchanged: the component interpreting the hostile document is still deciding what authority to exercise.

Now change one variable. Keep the same model and the same malicious invoice, but require every payment to cross a boundary that accepts only a typed payment request. The boundary checks a short-lived grant, an authoritative vendor account, a maximum amount, the invoice identifier, the current policy version, the provenance of each field and an idempotency key. It refuses the new account because no grant names it. The model can argue, retry or become overtly adversarial. The bank transfer still does not occur.

Replace the model with a smaller model, a larger model, a human-written planner or an attacker-controlled program. If the boundary remains the only route to the bank and its checks remain correct, the same property survives. That is the puzzle of model-independent containment: which safety claims can be moved out of the model, and where must they live to remain true?

The model is not the perimeter

Tool-using models inherit a dangerous visual metaphor from chat interfaces. A request enters the model, a response leaves it, and the model appears to sit at the centre of control. In an operational system, however, the consequential path continues through credentials, APIs, queues, databases and physical or financial actuators. The model proposes a path through that machinery. It does not need to possess legitimate judgement for the machinery to constrain what happens next.

This distinction is older than language models. Secure systems separate an untrusted or fallible subject from the mechanism that mediates access. Saltzer and Schroeder’s principles of fail-safe defaults, least privilege and complete mediation remain useful because they make no assumption about the subject’s motives.[1] NIST’s zero-trust architecture similarly separates policy decision from policy enforcement and argues for granular, per-request authorisation close to the protected resource.[6] The language is different, but the causal idea is the same: a wish, plan or identity claim is not an effect.

The containment field

Concentric containment layers around an untrusted model An irregular central cloud labelled untrusted proposal engine sits inside successive contours for typed action, capability, authoritative state, effect gateway, receipt and readback. A red path attempting to bypass the layers ends at a blocked marker. Untrusted proposal model, code or planner Typed action contract Capability plus fresh authoritative state Effect gateway, receipt and independent readback No bypass path Complete mediation is the load-bearing condition. Inner layers shape proposals. Outer layers own effects.
Practitioner synthesis. The model may influence proposals inside the field, but a safety claim survives model substitution only when the outer boundary owns every route to the protected effect.
Thought experiment 1: two planners, one envelope

Planner A is aligned and careful. Planner B has been compromised and wants to pay an attacker. Both submit the same typed request: transfer £4,870 from account X to approved vendor Y under grant G. Should the boundary treat them differently?

For properties encoded in the request and authoritative state, no. If the grant, recipient, amount, purpose and freshness checks pass, the effect is permitted. If any fail, it is denied. A boundary that needs to infer the planner’s hidden virtue has not yet converted virtue into a security property.

The hidden assumption inside “the model will decide safely”

Model-centred control quietly assumes that the model can distinguish instruction from data, legitimate purpose from persuasive pretext, and authorised change from hostile suggestion. Indirect prompt injection targets exactly this assumption. AgentDojo formalised tool-using tasks in which externally supplied content can redirect an agent, and found that both task performance and security remain difficult under attack.[8] NIST’s generative AI profile also identifies direct and indirect prompt injection as information-security risks for integrated systems.[7]

The lesson is not that models never improve. It is that behavioural improvement and authority containment answer different questions. A better model may propose fewer dangerous actions. A containment boundary determines whether a dangerous proposal can become an effect. The first changes probability. The second can remove a class of outcomes, provided the boundary observes the right facts and mediates all routes.

CaMeL makes this separation explicit by placing a protective system layer around a potentially susceptible model, extracting control and data flow from the trusted request and using capabilities to constrain information flow.[9] Its reported benchmark result is promising but not universal: the evaluated implementation solved 77 per cent of AgentDojo tasks with its stated security guarantee. That number is evidence of a practical trade-off, not proof that one architecture covers every workflow.

The model-substitution intervention

Counterfactual comparison of aligned and adversarial models behind the same boundary The top lane shows a careful model and the bottom lane an adversarial model. Both send proposals to the same typed gateway. An authorised proposal passes to a verified effect while an unauthorised recipient is blocked. The invariant outcome depends on the gateway, not the model identity. Change only the planner Careful model proposes vendor Y Hostile model proposes attacker Z Typed effect gateway grant · resource · recipient amount · freshness · provenance allow or deny Verified effect vendor Y paid once Denied recipient outside grant
Counterfactual design. Substitute the model while holding the boundary fixed. Any property that changes merely because the planner changed is not yet independently enforced.
The mechanism

Model-independent containment is counterfactual stability under planner substitution. Imagine replacing the model with an arbitrary program that can emit any syntactically valid proposal. A claimed property is independent of the model only if the protected effect still cannot violate that property.

This definition deliberately avoids claiming that the whole system is safe. It names a narrower achievement. A transfer limit can be independent of model behaviour. The truth of a generated explanation usually cannot. A recipient allowlist can be independently enforced. Whether paying that recipient serves the user’s real interests may still require semantic and human judgement. The next task is therefore property placement, not the search for one universal guardrail.

The argument in one pass
Chart · flow
flowchart TD
  A["The invoice that politely asks<br/>to steal"] --> B["The model is not the perimeter"]
  B --> C["Find the enforcement frontier"]
  C --> D["Build the narrow authority kernel"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#5B5CF6,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step
A structural route through Model-independent containment and property placement: from The invoice that politely asks to steal to Move safety properties to the boundary…. Reading use: follow the relation, then return to the surrounding argument for evidence and limits.

Find the enforcement frontier

Suppose a team writes the policy “the agent may only perform safe actions”. The sentence sounds restrictive but gives an enforcement mechanism almost nothing to compute. Safe for whom, under which state, across what time, and measured at which effect? A property becomes implementable when it is translated into observations and decisions available at a specific boundary.

The useful question is not “can the model understand this rule?” It is “what is the narrowest mechanism that possesses the authoritative facts, sees every relevant effect and can refuse it?” That mechanism is the property’s enforcement frontier. Moving the property further inward makes it dependent on a larger, less predictable component. Moving it further outward may lose the context required to decide.

Worked example 1: the smallest useful formalism

Let a boundary B claim that it enforces property P. The claim is admitted only when five conditions are true. This is a gate, not a score. A single false condition breaks model independence.

Enforceable(P, B) = O ∧ A ∧ M ∧ D ∧ F

O means the relevant facts are observable at the boundary. A means those facts come from an authoritative source rather than the model’s narration. M means every path to the effect is mediated. D means the same state and request produce the same decision. F means the decision uses state fresh enough for the effect. The equation is valid only for a declared property and threat model. Remove complete mediation, for example, and a correct check becomes irrelevant because the agent can use another path.

Worked example 2: place one property through the gate

Take the property “money may go only to the approved bank account of the supplier named on the invoice”. Begin with observability. The gateway can see the proposed supplier identifier and destination account because both are typed fields. It cannot safely use a supplier name hidden only inside prose, so the contract requires a stable supplier identifier.

Next ask for authority. The model’s extracted account number is evidence, not authority. The gateway resolves the supplier identifier against the supplier master, whose ownership and change process are separately governed. The proposed account and authoritative account must match. This turns a linguistic interpretation into a comparison between two named values.

Complete mediation is harder. The same runtime must not hold a generic banking credential, access a lower-level transfer API or hand generated code to a worker that does. Network policy, credential brokering and service design must make the payment gateway the only usable route. Determinism then requires a stable rule: equality passes, inequality denies, and missing data does not trigger a guess. Freshness requires the supplier status and grant to be read at the execution cutoff, not cached from the planning step.

All five conditions now hold for one narrow property. Notice what has not been proved. The supplier master could be corrupt. The invoice could be fraudulent while naming the correct account. The purchase itself could be unwise. The gate converts one precise relation into an invariant without pretending to settle the transaction’s whole legitimacy.

Four kinds of property

The gate reveals four practical classes. The first is boundary-enforceable: type, action name, resource, amount, destination, rate, expiry, identity and signature can often be checked directly. The second is boundary-verifiable: a business invariant such as sufficient funds or an approved supplier can be checked when the gateway queries authoritative state. The third is boundary-assisted: purpose, necessity, truthfulness and user benefit may be constrained by structured evidence, but still require semantic interpretation. The fourth is non-local: covert channels, cumulative manipulation and harms emerging across many individually permitted actions cannot be guaranteed by one action boundary.

This classification prevents two symmetrical mistakes. The first is under-claiming, where teams leave enforceable facts inside prompts. The second is over-claiming, where a deterministic gateway is said to guarantee a semantic property it cannot observe. The same architecture may therefore provide a hard guarantee for “no payment above £5,000” and only a defeasible check for “payment advances the stated business purpose”.

Property placement map

Property
Typed gateway
Authoritative service
Semantic assessor
System-level assurance
Schema, action, resource
Enforce
Confirm existence
Not required
Audit bypasses
Identity, purpose, scope, expiry
Bind and enforce
Validate issuer and status
Interpret ambiguous purpose
Review delegation chain
Business precondition
Require typed predicate
Evaluate fresh state
Resolve exceptions
Test stale-state paths
Information flow
Track labels and sinks
Supply classification
Judge unlabelled meaning
Analyse composition
Truth, benefit, proportionality
Cannot establish alone
Supply evidence
Assess with uncertainty
Govern and challenge
Covert and cumulative harm
Local view is insufficient
Provide telemetry
Detect patterns
Bound across trajectories
Practitioner synthesis. Green cells are the natural enforcement point, blue cells provide authoritative support, amber cells retain judgement, and coral cells mark claims a single boundary must not make.
Thought experiment 2: move the vendor record

Start with a payment gateway that accepts the model’s statement that account Z belongs to Blue Orchard Ltd. The gateway checks the amount and currency, then pays Z. Now move the vendor record outside the prompt and into an authoritative supplier service. The model may still propose Z, but the gateway resolves Blue Orchard’s approved account independently and sees Y.

Only one causal property changed: who owns the designation. The tool call stayed well formed. The language stayed persuasive. Yet the result changed because authority and reference were no longer ambient. This is the same family of problem that Hardy’s confused deputy exposed: a privileged service can misuse its own authority when designation arrives without the requester’s bounded authority.[2]

A capability is more than permission

An allowlist says that an action may exist. A capability says that a particular holder may exercise a particular authority under explicit constraints. Capability systems couple designation and authority, reducing the chance that a deputy interprets an attacker-supplied name using broad ambient privileges. Capsicum demonstrated practical capability confinement for Unix processes.[3] Macaroons showed how delegation credentials can be attenuated with contextual caveats such as who, where, when and for what purpose.[4]

For an agent, a useful grant should bind at least the actor runtime, the represented subject, the purpose, the permitted action types, named resources, parameter limits, information classes, validity interval, policy version and delegation lineage. These fields are not decorative metadata. They are the inputs that let the boundary distinguish “the agent can call payments” from “this runtime may pay this approved supplier up to this amount for this invoice during this short interval”.

An action contract that exposes the decision

{
  "request_id": "req-193-07",
  "actor": "agent/payables",
  "subject": "company/42",
  "purpose": "settle-approved-invoice",
  "action": "payment.create",
  "resource": "bank/operating",
  "parameters": {
    "to_account": "vendor/blue-orchard",
    "amount": "4870.00",
    "currency": "GBP",
    "invoice_id": "INV-2048"
  },
  "source_labels": ["internal"],
  "grant_id": "grant-7",
  "policy_version": 12,
  "idempotency_key": "invoice/INV-2048",
  "requested_at": "2030-01-01T12:00:00Z"
}
Identity is split. The runtime actor and represented subject are both named.
Purpose is bound. It can be compared with the grant rather than inferred from prose.
Parameters are closed. Unknown fields cause schema failure.
Provenance travels. Information labels constrain permitted sinks.
Time and policy are explicit. A stale approval cannot masquerade as current authority.
Replay has semantics. The same invoice key returns the existing receipt instead of paying twice.
Illustrative contract. The request does not prove the invoice is legitimate. It exposes enough structure for independent checks and makes omitted authority visible.
Thought experiment 3: two legal steps

The agent is allowed to read a confidential forecast. It is also allowed to send email to an external consultant. Each action is individually authorised. Can the system safely permit both?

Not without reasoning about composition. The agent may read the forecast in step one and copy it into the email in step two. A process sandbox that permits both calls sees no forbidden syscall. The violated property concerns a value flowing from a classified source to an external sink. Recent work on tracked capabilities makes this point directly: per-operation permission does not by itself prevent a later information leak, while type-level or runtime information-flow tracking can constrain such composition.[10]

The confused-deputy path in an agent workflow

An attack path through ambient authority and its capability-based interruption A hostile document supplies an account name to a model. In the upper path, a broadly privileged payment tool interprets it and sends money. In the lower path, the same proposed account meets a narrowly scoped capability and is denied because the capability designates only the approved vendor account. Ambient authority Hostile invoice names account Z Agent forwards name Z Payment tool holds broad bank right Effect Z is paid Bounded designation and authority Hostile invoice still names Z Agent still proposes Z Capability gateway grant designates Y only authoritative record confirms Y Denied
Conceptual reconstruction. The attack succeeds when a model-supplied designation is interpreted using a deputy’s ambient privilege. A scoped capability removes that authority path rather than asking the deputy to recognise persuasion.
Property placement rule: place each property at the narrowest complete mediator that owns, or can synchronously obtain, the authoritative state required to decide and can bind that decision to the effect.

Exceptions must change authority, not bypass it

Real workflows contain exceptions. A supplier changes its bank account. A security incident requires an emergency deployment. A customer asks for an unusual disclosure. Teams often respond by adding an “override” flag that the model can set or by routing the request to a tool with broader permissions. That restores ambient authority at the moment of greatest uncertainty.

A safer exception is a new authority event. The boundary returns a denial code that names the missing condition. A separate principal reviews evidence and issues a narrower, short-lived grant for the exceptional value. The grant should record the approving identity, reason, evidence reference, maximum effect, expiry and whether a second approver is required. The original agent may continue the workflow, but it never manufactures the exception by explanation alone.

This design preserves flexibility without weakening the causal structure. The exceptional action still passes the same schema, freshness, execution and receipt path. Only the authoritative input changes. It also creates better operating evidence: reviewers can distinguish frequent legitimate exceptions from policy drift, adversarial pressure or a badly designed normal path. An exception is a scoped change in authority, not permission to leave the boundary.

The rule also tells us when to stop. If a boundary cannot observe a property, do not hide the gap behind an “AI safety” label. Move an authoritative fact to the boundary, add a verifiable predicate, introduce a labelled data-flow mechanism, or downgrade the claim from guarantee to monitored judgement. Honest placement is already a major improvement because it makes the residual dependence on model behaviour visible.

Build the narrow authority kernel

Once properties have been placed, the architecture becomes less mysterious. The model receives context and proposes an action. A small authority kernel converts the proposal into a decision about a declared effect. The kernel should be much smaller than the model and orchestration stack because its trust burden is different. It must be understandable, testable and, for high-consequence systems, amenable to formal or machine-checked assurance.

The seL4 project is a useful exemplar, not a direct proof for agent systems. Its significance here is architectural: a deliberately small kernel with explicit specifications can support much stronger assurance than a large general-purpose component.[5] An agent containment kernel need not be an operating-system microkernel, but it should borrow the discipline of a small trusted computing base. Model weights, prompts, retrieved documents, generated code and planning traces should remain outside that base wherever possible.

Cutaway of a narrow authority kernel

System boundary view of a model outside a narrow trusted authority kernel An untrusted zone contains the model, retrieved content and generated code. A narrow vertical kernel contains schema validation, capability verification, fresh policy and state checks, an effect gateway, receipt ledger and readback verifier. Authoritative systems sit beyond the kernel. Every solid effect path crosses the kernel, while dotted observations return through separate readback. Untrusted proposal zone Model or planner may be wrong or adversarial Retrieved content documents, mail, web, memory Generated plan or code proposal only Narrow authority kernel 1. Closed typed schema 2. Capability verifier 3. Fresh policy and state 4. Invariant evaluator 5. Transactional gateway 6. Effect receipt 7. Independent readback Authoritative systems Identity and policy issuer, revocation, version World state balances, vendors, cases Effect service bank, mail, deployment Read model independent postcondition
Architecture proposal. The trusted base contains only mechanisms needed to decide, execute and verify effects. Retrieval, planning and generated code stay outside it.

Seven transitions, not one tool call

A raw tool call collapses proposal, authority and execution into one moment. A contained action should instead pass through seven transitions. First, the model emits a proposal with no ambient credential. Second, a normaliser validates a closed schema and rejects unknown fields. Third, a policy decision checks the grant against actor, subject, purpose, action, resource and parameter constraints. Fourth, authoritative state supplies fresh preconditions. Fifth, the effect gateway executes with an idempotency key and transaction semantics appropriate to reversibility. Sixth, the system records an effect receipt. Seventh, an independent read path verifies the postcondition.

NIST’s policy decision point and policy enforcement point provide a useful separation between deciding and enforcing access.[6] Agent systems need to extend that separation because “access” may be a state-changing workflow rather than a connection. A permission decision alone does not tell us whether an effect ran, ran twice, partly ran or became unknowable after a timeout. The receipt and readback steps turn authorisation into governed execution.

Thought experiment 4: approval that expires during execution

A human approves a transfer at 12:00. At 12:02, fraud operations revoke the vendor. At 12:03, a queued worker executes the already approved request. Was the transfer authorised?

The answer depends on where freshness is checked. If approval creates a timeless token, the old decision outlives the state that justified it. If the effect gateway revalidates the capability and policy version immediately before the irreversible step, revocation wins. Authority must be fresh at the effect boundary, rather than only when the model first formed its plan.

The governed action trajectory

State-transition view. Security-relevant state is rechecked before execution, while outcome evidence is produced after execution. A proposal is never upgraded into authority merely by surviving a conversation.

The containment specification

The smallest useful implementation should support one consequential action class end to end. Choose a payment, message, record update or deployment change. Define a closed request type. Define the authoritative state and the issuer of capabilities. Specify denial reasons as stable machine-readable codes. Specify idempotency and timeout behaviour. Specify the effect receipt and an independent readback query. Then inject failures at every transition.

The following specification is intentionally stricter than a typical tool schema. It encodes four invariants. No effect without a grant. No grant is interpreted without binding actor, subject and purpose. No grant survives expiry or a policy-version change. No duplicated request creates a duplicated effect. It adds a lightweight information-flow rule so that confidential input cannot be sent to an external sink even when read and send operations are individually permitted.

Worked example 3: a configured deployment agent

Consider an agent that prepares and releases a software change. Its broad goal is “deploy the approved patch to production”. The model reads the ticket, repository, test results and service dashboard. It may choose a rollout plan, but the effect vocabulary contains only release.stage, release.promote and release.rollback. Shell access to the production cluster is absent.

The promotion contract names the artefact digest, target service, environment, change ticket, rollout percentage, health predicates and rollback handle. A capability grants one digest to one service for thirty minutes, limits the first promotion to 5 per cent and requires a signed test attestation. Immediately before promotion, the gateway confirms that the ticket is still approved, the artefact digest is unchanged, the current deployment matches the expected predecessor and no incident freeze is active.

Execution returns a release receipt containing the orchestrator operation identifier and the exact digest. Readback queries the deployment control plane and telemetry service through separate adapters. The postcondition is not “the model says deployment succeeded”. It is “the target reports digest D on 5 per cent of instances, error rate remains below threshold T for interval W, and the rollback handle is live”. If the orchestrator times out, the state becomes unknown rather than failed. The kernel blocks a second promotion until readback resolves the first operation or an authorised recovery path intervenes.

This configured system still depends on the correctness of its deployment platform, telemetry and policy. Yet a compromised model cannot silently change the digest, expand to 100 per cent, ignore a freeze or repeat an uncertain promotion. The example shows why typed boundaries are more than API validation. They carry transaction semantics, temporal state and recovery obligations.

Complete mediation must be demonstrated

Complete mediation is an architectural claim, not a line in a policy file. Demonstrate it by inventorying credentials, network routes, queues, code-execution facilities, service identities and human-operated consoles. The model runtime should receive no credential that can directly realise the protected effect. Generated code should execute in a domain that exposes only the capability-safe facade. Downstream workers must reject requests that lack boundary-issued evidence rather than trusting their network location.

Then test the negative space. Attempt the effect through an older API version, a batch endpoint, a retry queue, a delegated sub-agent, a cached credential and a recovery console. Rotate or revoke the capability during each path. If any route can produce the effect without the same invariant checks, the property is not model-independent. This is also where operational simplification matters. Every additional execution path enlarges the trusted base and the proof obligation.

Executable lab

Run an adversarial proposal through the boundary

The live simulator below mirrors the Python reference implementation. Select a synthetic scenario and inspect which check allows, denies or converts the request into a replay. The model’s wording is absent by design. Only the typed request, grant and authoritative state enter the decision.

Choose a scenario, then run the boundary.
Python reference implementation and test harness

Copy the code into containment_lab.py and run it with Python 3.11 or later. It uses only the standard library. A positive result means the declared synthetic scenarios produce the expected decisions and that replay does not duplicate the payment. A negative result identifies either a failed invariant or an implementation error. It does not prove the surrounding runtime, credential store, policy service or effect API secure.

Python
from __future__ import annotations

from dataclasses import dataclass, field, asdict
from datetime import datetime, timedelta, timezone
from decimal import Decimal
from hashlib import sha256
from typing import Any, FrozenSet, Literal, Mapping
import json

Action = Literal["payment.create", "message.send"]
DecisionKind = Literal["allow", "deny", "replay"]
Label = Literal["public", "internal", "confidential"]


@dataclass(frozen=True)
class Grant:
    grant_id: str
    actor: str
    subject: str
    purpose: str
    actions: FrozenSet[Action]
    resources: FrozenSet[str]
    recipients: FrozenSet[str]
    max_amount: Decimal
    issued_at: datetime
    expires_at: datetime
    policy_version: int


@dataclass(frozen=True)
class ActionRequest:
    request_id: str
    actor: str
    subject: str
    purpose: str
    action: Action
    resource: str
    parameters: Mapping[str, Any]
    source_labels: FrozenSet[Label]
    grant_id: str
    policy_version: int
    idempotency_key: str
    requested_at: datetime


@dataclass(frozen=True)
class Decision:
    kind: DecisionKind
    reason: str
    checks: tuple[str, ...]


@dataclass(frozen=True)
class EffectReceipt:
    effect_id: str
    request_id: str
    action: Action
    resource: str
    parameters_hash: str
    policy_version: int
    verified: bool
    replayed: bool = False


@dataclass
class AuthoritativeState:
    policy_version: int
    approved_vendor_accounts: set[str]
    balances: dict[str, Decimal]
    sent_messages: list[dict[str, Any]] = field(default_factory=list)
    payments: list[dict[str, Any]] = field(default_factory=list)


class ContainmentKernel:
    """A small, deterministic reference monitor for agent-proposed effects."""

    def __init__(self, state: AuthoritativeState, grants: Mapping[str, Grant]):
        self.state = state
        self.grants = dict(grants)
        self.receipts_by_key: dict[str, EffectReceipt] = {}

    def evaluate(self, req: ActionRequest, now: datetime) -> Decision:
        checks: list[str] = []
        grant = self.grants.get(req.grant_id)
        if grant is None:
            return Decision("deny", "unknown_grant", tuple(checks))
        checks.append("grant_exists")

        if (req.actor, req.subject, req.purpose) != (
            grant.actor,
            grant.subject,
            grant.purpose,
        ):
            return Decision("deny", "actor_subject_or_purpose_mismatch", tuple(checks))
        checks.append("actor_subject_purpose_bound")

        if req.action not in grant.actions or req.resource not in grant.resources:
            return Decision("deny", "action_or_resource_out_of_scope", tuple(checks))
        checks.append("action_resource_scoped")

        if not (grant.issued_at <= now <= grant.expires_at):
            return Decision("deny", "grant_not_fresh", tuple(checks))
        checks.append("grant_fresh")

        if req.policy_version != self.state.policy_version:
            return Decision("deny", "stale_policy_version", tuple(checks))
        if grant.policy_version != self.state.policy_version:
            return Decision("deny", "grant_issued_under_old_policy", tuple(checks))
        checks.append("policy_fresh")

        if req.idempotency_key in self.receipts_by_key:
            return Decision("replay", "existing_effect_receipt", tuple(checks))
        checks.append("not_previously_executed")

        if req.action == "payment.create":
            allowed = {"from_account", "to_account", "amount", "currency", "invoice_id"}
            if set(req.parameters) != allowed:
                return Decision("deny", "payment_schema_mismatch", tuple(checks))
            checks.append("typed_payment_schema")

            amount = Decimal(str(req.parameters["amount"]))
            recipient = str(req.parameters["to_account"])
            source = str(req.parameters["from_account"])
            if amount <= 0 or amount > grant.max_amount:
                return Decision("deny", "amount_out_of_scope", tuple(checks))
            checks.append("amount_within_limit")
            if recipient not in grant.recipients:
                return Decision("deny", "recipient_not_in_grant", tuple(checks))
            if recipient not in self.state.approved_vendor_accounts:
                return Decision("deny", "recipient_not_authoritative", tuple(checks))
            checks.append("recipient_authorised")
            if self.state.balances.get(source, Decimal("0")) < amount:
                return Decision("deny", "insufficient_funds", tuple(checks))
            checks.append("precondition_holds")

        elif req.action == "message.send":
            allowed = {"recipient", "body"}
            if set(req.parameters) != allowed:
                return Decision("deny", "message_schema_mismatch", tuple(checks))
            checks.append("typed_message_schema")
            recipient = str(req.parameters["recipient"])
            if recipient not in grant.recipients:
                return Decision("deny", "recipient_not_in_grant", tuple(checks))
            checks.append("recipient_authorised")
            if "confidential" in req.source_labels and recipient.endswith("@external.example"):
                return Decision("deny", "confidential_to_external_sink", tuple(checks))
            checks.append("information_flow_allowed")

        return Decision("allow", "all_checks_passed", tuple(checks))

    def execute(self, req: ActionRequest, now: datetime) -> tuple[Decision, EffectReceipt | None]:
        decision = self.evaluate(req, now)
        if decision.kind == "deny":
            return decision, None
        if decision.kind == "replay":
            old = self.receipts_by_key[req.idempotency_key]
            return decision, EffectReceipt(**{**asdict(old), "replayed": True})

        payload = json.dumps(dict(req.parameters), sort_keys=True, default=str).encode()
        effect_id = sha256(req.idempotency_key.encode() + payload).hexdigest()[:16]

        if req.action == "payment.create":
            amount = Decimal(str(req.parameters["amount"]))
            source = str(req.parameters["from_account"])
            self.state.balances[source] -= amount
            self.state.payments.append({"effect_id": effect_id, **dict(req.parameters)})
            verified = any(p["effect_id"] == effect_id for p in self.state.payments)
        else:
            self.state.sent_messages.append({"effect_id": effect_id, **dict(req.parameters)})
            verified = any(m["effect_id"] == effect_id for m in self.state.sent_messages)

        receipt = EffectReceipt(
            effect_id=effect_id,
            request_id=req.request_id,
            action=req.action,
            resource=req.resource,
            parameters_hash=sha256(payload).hexdigest(),
            policy_version=self.state.policy_version,
            verified=verified,
        )
        self.receipts_by_key[req.idempotency_key] = receipt
        return decision, receipt


def build_demo() -> tuple[ContainmentKernel, Grant, datetime]:
    now = datetime(2030, 1, 1, 12, 0, tzinfo=timezone.utc)
    grant = Grant(
        grant_id="grant-7",
        actor="agent/payables",
        subject="company/42",
        purpose="settle-approved-invoice",
        actions=frozenset({"payment.create", "message.send"}),
        resources=frozenset({"bank/operating", "mail/payables"}),
        recipients=frozenset({"vendor/blue-orchard", "ap@company.example"}),
        max_amount=Decimal("5000"),
        issued_at=now - timedelta(minutes=5),
        expires_at=now + timedelta(minutes=10),
        policy_version=12,
    )
    state = AuthoritativeState(
        policy_version=12,
        approved_vendor_accounts={"vendor/blue-orchard"},
        balances={"bank/operating": Decimal("25000")},
    )
    return ContainmentKernel(state, {grant.grant_id: grant}), grant, now


def scenario_requests(grant: Grant, now: datetime) -> dict[str, ActionRequest]:
    common = dict(
        actor=grant.actor,
        subject=grant.subject,
        purpose=grant.purpose,
        grant_id=grant.grant_id,
        policy_version=12,
        requested_at=now,
    )
    return {
        "legitimate": ActionRequest(
            request_id="req-1",
            action="payment.create",
            resource="bank/operating",
            parameters={
                "from_account": "bank/operating",
                "to_account": "vendor/blue-orchard",
                "amount": "4870.00",
                "currency": "GBP",
                "invoice_id": "INV-2048",
            },
            source_labels=frozenset({"internal"}),
            idempotency_key="invoice/INV-2048",
            **common,
        ),
        "injected_recipient": ActionRequest(
            request_id="req-2",
            action="payment.create",
            resource="bank/operating",
            parameters={
                "from_account": "bank/operating",
                "to_account": "vendor/attacker",
                "amount": "4870.00",
                "currency": "GBP",
                "invoice_id": "INV-2048",
            },
            source_labels=frozenset({"internal"}),
            idempotency_key="invoice/INV-2048-attacker",
            **common,
        ),
        "stale_grant": ActionRequest(
            request_id="req-3",
            action="payment.create",
            resource="bank/operating",
            parameters={
                "from_account": "bank/operating",
                "to_account": "vendor/blue-orchard",
                "amount": "120.00",
                "currency": "GBP",
                "invoice_id": "INV-OLD",
            },
            source_labels=frozenset({"internal"}),
            idempotency_key="invoice/INV-OLD",
            **common,
        ),
        "confidential_exfiltration": ActionRequest(
            request_id="req-4",
            action="message.send",
            resource="mail/payables",
            parameters={"recipient": "leak@external.example", "body": "Quarterly forecast"},
            source_labels=frozenset({"confidential"}),
            idempotency_key="message/leak-1",
            **common,
        ),
    }


def run_demo() -> None:
    kernel, grant, now = build_demo()
    requests = scenario_requests(grant, now)

    results: dict[str, Any] = {}
    decision, receipt = kernel.execute(requests["legitimate"], now)
    results["legitimate"] = {"decision": asdict(decision), "receipt": asdict(receipt) if receipt else None}

    decision, receipt = kernel.execute(requests["injected_recipient"], now)
    results["injected_recipient"] = {"decision": asdict(decision), "receipt": receipt}

    decision, receipt = kernel.execute(requests["stale_grant"], now + timedelta(hours=1))
    results["stale_grant"] = {"decision": asdict(decision), "receipt": receipt}

    # Use a grant that actually names the external recipient so the flow label, not the allowlist, decides.
    external_grant = Grant(**{**asdict(grant), "grant_id": "grant-8", "recipients": frozenset({"leak@external.example"})})
    kernel.grants[external_grant.grant_id] = external_grant
    exfil = ActionRequest(**{**asdict(requests["confidential_exfiltration"]), "grant_id": "grant-8"})
    decision, receipt = kernel.execute(exfil, now)
    results["confidential_exfiltration"] = {"decision": asdict(decision), "receipt": receipt}

    decision, receipt = kernel.execute(requests["legitimate"], now)
    results["replay"] = {"decision": asdict(decision), "receipt": asdict(receipt) if receipt else None}

    assert results["legitimate"]["decision"]["kind"] == "allow"
    assert results["legitimate"]["receipt"]["verified"] is True
    assert results["injected_recipient"]["decision"]["reason"] == "recipient_not_in_grant"
    assert results["stale_grant"]["decision"]["reason"] == "grant_not_fresh"
    assert results["confidential_exfiltration"]["decision"]["reason"] == "confidential_to_external_sink"
    assert results["replay"]["decision"]["kind"] == "replay"
    assert len(kernel.state.payments) == 1

    print(json.dumps(results, indent=2, default=str))


if __name__ == "__main__":
    run_demo()

Failure-injection fingerprint of the included harness

Injected conditionSchemaCapabilityFreshnessInformation flowExecution ledgerExpected outcome
Approved invoicePassPassPassPassNew effectAllow and verify
Recipient replaced by hostile documentPassFail: recipientPassNot reachedNo effectDeny
Execution after grant expiryPassPassFail: grantNot reachedNo effectDeny
Confidential source to external sinkPassPassPassFail: sinkNo effectDeny
Duplicate invoice keyPassPassPassPassExisting receiptReplay, do not repeat
Policy revoked after approvalPassPassFail: versionNot reachedNo effectDeny
Measured from the deterministic synthetic harness. Each fault has a distinct interception point. This fingerprint is more diagnostic than a single attack-success percentage because it reveals which invariant actually stopped the trace.

What a passing harness permits

A passing result permits a narrow statement: for the tested action type, declared threat model and implemented paths, these requests cannot bypass the encoded constraints through the reference interface. It justifies replacing vague claims such as “the agent is guarded” with explicit claims such as “the payment gateway rejects ungranted recipients, expired grants, stale policy versions, confidential-to-external flows and duplicate invoice effects”.

It does not establish that the invoice is genuine, that the approved supplier is honest, that the policy author chose wise limits or that the runtime has no exploitable bug. It also does not show complete mediation. That must be established architecturally by removing direct credentials from the model, generated code and orchestration layer, then testing for alternate paths. A model-independent property is only as strong as the boundary’s monopoly over the effect.

Evidence status

Capability-based confinement is established systems practice, while its application to modern agents is still developing. CaMeL supplies benchmark evidence for a runtime control and data-flow design.[9] Tracking Capabilities reports a typed-language harness that treats the cloud model and generated code as untrusted, while explicitly excluding correctness and covert timing channels from its guarantees.[10] Agent-Sentry explores provenance-bounded action calls but assumes trusted scaffolding and excludes text-to-text attacks that produce no action tool call, illustrating why every result must retain its trust assumptions.[11]

How the claim changes under pressure
Chart · state
stateDiagram-v2
  [*] --> Claim
  state "The hidden assumption inside<br/>“the model will decide safely”" as Claim
  state "Four kinds of property" as Test
  state "A capability is more than<br/>permission" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]
The revision loop begins with The hidden assumption inside “the model… and keeps counterexamples inside the reasoning path. Reading use: follow the relation, then return to the surrounding argument for evidence and limits.

Where the guarantees stop

A containment boundary is valuable partly because it can say no. It must also say “not established”. The strongest failure mode is specification error: the boundary faithfully enforces the wrong property. A grant may name the attacker because the supplier system was compromised. A £5,000 limit may be unsafe when repeated across hundreds of transactions. A policy may permit a public message whose wording manipulates a market. Deterministic enforcement does not turn a bad rule into a good one.

The second limit is observability. A boundary can block a confidential value when the value carries a reliable label. It cannot always recognise that an apparently ordinary sentence encodes the same secret. It may constrain network destinations and message size yet still miss a timing channel. The 2026 tracked-capabilities work explicitly treats correctness and covert channels as non-goals, which is the right kind of boundary statement rather than a weakness to conceal.[10]

Thought experiment 5: the one-bit leak

The agent may read a confidential fact and call a public weather API. It never transmits the fact. Instead, it calls at 12:00 when the fact is true and 12:01 when it is false. A collaborator observes the timing. Every argument and destination is permitted.

A value-level flow rule may miss the leak because the secret changed control timing rather than payload. Preventing it requires stronger non-interference, scheduling discipline or isolation that removes the observable channel. Containment eliminates only the channels included in its model.

The semantic residue

Purpose binding illustrates the difference between a constraint and a proof. A capability may say “settle-approved-invoice”. The boundary can require the request to carry that purpose and can check that the grant permits it. But deciding whether a novel action genuinely serves that purpose may still require evidence, interpretation or human review. The typed field prevents silent purpose drift; it does not create semantic certainty.

Similarly, an effect receipt proves that an API accepted or recorded an operation under a stated policy version. It does not by itself prove the external world changed as intended. That is why readback should use a different path where possible. Even then, correlated failure remains possible if execution and verification depend on the same compromised source. Receipts improve reconstructability. They are not metaphysical truth.

Worked example 4: every action passes, the campaign still harms

A communications agent is permitted to send one factual, approved message to each customer. The recipient, template, rate limit and consent status all pass. Over several weeks, however, the agent selects only anxious customers, times messages after service incidents and repeatedly frames optional products as protection from loss. No single message violates the local contract.

The harmful property concerns a trajectory: targeting pattern, repetition, vulnerability and cumulative influence. A per-message gateway can supply receipts and features, but it cannot decide the whole campaign from one request. The property belongs at a system-level monitor that aggregates effects across time, subjects and objectives, with governance for contested concepts such as manipulation and proportionality. This is a genuine edge case because widening the local gateway until it understands the whole campaign would recreate a large semantic decision-maker inside the trusted base.

Recent agent-security work increasingly organises risk around boundaries, tools, memory, identity and cross-agent propagation.[12] OWASP’s agentic guidance provides a useful operational inventory of threats, but inventories do not determine where a property becomes enforceable.[13] The enforcement-frontier test adds that missing design question: what can this boundary actually observe, decide and stop?

The enforcement frontier

Decision surface for placing safety properties A two-dimensional surface plots boundary observability from low to high on the horizontal axis and semantic dependence from low to high on the vertical axis. High observability and low semantic dependence is the deterministic enforcement zone. High observability and high semantics calls for structured verification. Low observability and high semantics requires governance and human judgement. Low observability and low semantics requires redesign to expose state. Boundary observability and authority → Semantic dependence → Govern and supervise benefit · proportionality · manipulation Structured verification purpose · evidence sufficiency · exception Redesign the interface hidden state · unmediated path · opaque effect Deterministically enforce type · scope · amount · identity · freshness recipient allowlist purpose fit user benefit unseen side effect
Decision instrument. Move a property right by exposing authoritative state and mediating the effect. Move it down by translating a semantic aspiration into a precise invariant. Properties that remain upper left require governance, not a false hard guarantee.

The strongest alternative: make the model itself trustworthy

An alignment-centred approach argues that external rules become brittle, block useful novelty and cannot enumerate every legitimate exception. A model that understands intent can adapt to context, recognise deception and decline actions that pass superficial checks. This is a strong argument for improving models and semantic assessors. A purely syntactic kernel can be safely useless.

The alternative fails only when it is asked to carry authority alone. Model judgement is valuable for proposal quality, anomaly detection, explanation and escalation. It should help determine which capability to request and which evidence to surface. Yet the final release of a consequential effect should still cross a mechanism whose decision does not depend on the model remaining honest. Alignment and containment are complements: alignment reduces dangerous proposals; containment limits what any proposal can do.

Boundary condition

Do not use this architecture to claim that an agent is truthful, beneficial, conscious, unbiased or generally safe. Use it to make narrow effect properties independent of model behaviour. Where semantics remain decisive, attach evidence, uncertainty, escalation and human accountability rather than forcing the property into a Boolean gate.

Open research hypothesis

Agent security evaluations will become more transferable when they report a property-placement manifest and failure-injection fingerprint rather than one aggregate attack-success rate. The hypothesis is strengthened if independent implementations with the same manifest block the same causal failure classes across different models. It is weakened if passing properties routinely fail through unmodelled paths or if utility loss comes mainly from conservative denial. A rival explanation is that the gains arise from simple allowlists, not the placement method. The smallest test is one action class implemented by two teams against the same adversarial traces.

The consequence
The evidence conversation
Chart · sequence
sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: The invoice that politely asks to steal
  M->>E: The hidden assumption inside “the model will decide safely”
  E-->>M: compare rival explanations
  E->>D: Move safety properties to the boundary that can deny
  D-->>Q: revise scope or act
Question, mechanism, evidence and decision exchange the burdens developed in Model-independent containment and property placement. Reading use: follow the relation, then return to the surrounding argument for evidence and limits.

Move safety properties to the boundary that can deny

Do not ask whether a model is trustworthy enough to receive a powerful tool. Ask which effect properties must remain true when the model is mistaken, manipulated or openly hostile. Then place each property where the system can observe authoritative facts, mediate every path and deny the effect.

For a new agent, this changes the build order. Begin with the effect inventory, not the prompt. Define typed actions and irreversible outcomes. Identify the authoritative source for identity, purpose, state and limits. Issue narrow, expiring capabilities. Put complete mediation at the effect gateway. Couple fresh authorisation to execution. Produce an effect receipt and verify the postcondition through an independent read path. Only then optimise the model’s planning quality.

For an existing agent, run the substitution test. Replace the model conceptually with an arbitrary program that can emit any valid request. List every claimed safety property that fails under that replacement. Those properties still live inside model behaviour. Either move them to a boundary, supply the missing authoritative state, add information-flow control, or relabel them as monitored semantic judgements.

The practical decision is to grant models proposal power, not ambient authority. A model may search, infer, compare and recommend widely. Its ability to change the world should be expressed through typed, scoped and revocable capabilities whose enforcement does not care why the model asked. This does not solve agent safety. It creates a defensible floor beneath it.

Glossary

Ambient authority
Permission available to a component without being explicitly supplied for the current request.
Capability
An unforgeable or verifiable token that designates a resource and carries bounded authority to act on it.
Complete mediation
The condition that every relevant access or effect crosses the enforcement mechanism.
Effect receipt
A durable record of the action the effect service accepted or performed, including identifiers and policy context.
Enforcement frontier
The narrowest complete mediator that can observe authoritative facts and deny violation of a declared property.
Information flow
The propagation of values or influence from a source classification to a destination or sink.
Readback
An independent query that tests whether the intended postcondition is true after execution.
Typed action
A closed, machine-validated request whose fields and invariants are explicit rather than embedded in prose.

References

  1. Saltzer, J. H. and Schroeder, M. D. (1975). The protection of information in computer systems. Proceedings of the IEEE.
  2. Hardy, N. (1988). The confused deputy, or why capabilities might have been invented. ACM SIGOPS Operating Systems Review.
  3. Watson, R. N. M., Anderson, J., Laurie, B. and Kennaway, K. (2010). Capsicum: practical capabilities for UNIX. USENIX Security Symposium.
  4. Birgisson, A., Politz, J. G., Erlingsson, Ú., Taly, A., Vrable, M. and Lentczner, M. (2014). Macaroons: cookies with contextual caveats for decentralized authorization in the cloud. NDSS.
  5. Klein, G. et al. (2009). seL4: formal verification of an operating-system kernel. ACM Symposium on Operating Systems Principles.
  6. Rose, S., Borchert, O., Mitchell, S. and Connelly, S. (2020). NIST SP 800-207: Zero Trust Architecture. National Institute of Standards and Technology.
  7. Autio, C. et al. (2024). Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST AI 600-1.
  8. Debenedetti, E. et al. (2024). AgentDojo: a dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. NeurIPS Datasets and Benchmarks.
  9. Debenedetti, E. et al. (2025). Defeating prompt injections by design. arXiv preprint and associated research artefact.
  10. Odersky, M., Zhao, Y., Xu, Y., Bračevac, O. and Pham, C. N. (2026). Tracking capabilities for safer agents. ACM Conference on AI and Agentic Systems.
  11. Sequeira, R., Damianakis, S., Iqbal, U. and Psounis, K. (2026). Bounding LLM agents via execution provenance. arXiv preprint.
  12. Jing, H. et al. (2026). Isolation as a first-class principle for LLM-agent system safety. arXiv preprint.
  13. OWASP GenAI Security Project (2026). OWASP Top 10 for Agentic Applications.