Executive briefThe provider met its terms; the enterprise still failed

A customer challenges a rejected application. The AI provider stayed within its availability target, processed the agreed fields and responded inside its latency commitment. Its support desk correctly says the service performed to contract. The enterprise cannot explain which model version produced the recommendation, retrieve the relevant evidence, correct the decision before its customer deadline or show that evaluation covered this population. The supplier met its terms. The enterprise failed its obligation.

That is the central third-party AI problem. Procurement can transfer development, hosting, evaluation, data processing or human-review activities. It also creates dependencies. It does not transfer the enterprise's accountability for choosing the capability, placing it in a decision, authorising action, protecting people, monitoring outcomes, responding to incidents or providing correction and redress.

The dependency is rarely one company. A branded application may call a model endpoint hosted on a cloud region, use proprietary embeddings and vector storage, licence source data, employ an evaluation vendor and route difficult cases to an outsourced human operation. Contracting with the application provider does not make that chain observable, controllable, recoverable or answerable.

This paper proposes a capability-level Third-Party AI Accountability Ledger. Its twelve records connect the business principal and exact decision to service identity, provider chain, data, responsibility, controls, evidence, change, incidents, continuity and exit. An Accountability Gap Graph exposes five breaks. An Evidence Rights Ladder sets proportional proof and cooperation rights. None of these mechanisms is a vendor score or a claim that a certification proves fitness.

The strongest rival: proportional reliance on capable providers

Hyperscale providers may have stronger security, resilience, testing and operational expertise than a customer could build. Demanding source access, bespoke audit and individual approval over every sub-component can delay useful deployment, exclude smaller providers and produce contract rights nobody can exercise. Low-consequence services with strong isolation may need only standard evidence.

The answer is not maximal inspection. It is decision-relevant assurance scaled to consequence, asymmetry and dependence. The enterprise needs enough evidence to make its own decision, enough intervention to contain its own exposure and enough cooperation to meet its own deadlines. Full source code is rarely the decisive missing right. Service identity, relevant test coverage, change notice, logs, preserved incident evidence, response cooperation and exit support often matter more.

Unit of governanceStart with the business capability, not the supplier record

A supplier record asks who was contracted. A capability record asks what enterprise outcome now depends on whom. Define the exact model contribution, business decision, external action, delegated authority and completion boundary. Name the affected population and credible consequence. The same model service can be a bounded writing assistant in one capability and a consequential decision input in another.

Record the enterprise business principal who can accept residual exposure and fund controls. Procurement owns sourcing mechanics; security may challenge technical controls; legal interprets duties; model risk may validate a use; operations manages continuity. None replaces the principal accountable for the customer or business outcome. A committee can advise but cannot be the unnamed owner.

Resolve the service object below the marketing name: provider, service, endpoint, model or family, region, configuration, version and effective date. A stable endpoint alias can conceal a model replacement. Open weights can still depend on a model originator, licence, tokenizer, cloud, serving stack, data, hardware and scarce expertise. Deployment form changes the chain; it does not erase it.

Third-Party AI Accountability LedgerTwelve records make responsibility operational

Every record is versioned against the capability and its current provider chain. The ledger connects rather than merely lists. An evidence right points to a duty it supports; a change right points to its revalidation trigger; an incident clock points to enterprise reporting and customer-remedy deadlines; exit points to the live state and commitments that must move.

Accountability Gap GraphFive breaks turn contractual comfort into operational failure

An orphan duty exists when no accountable actor owns an obligation. It commonly appears between provider incident detection and enterprise customer response. A paper duty names an actor that lacks authority, access, time or capacity. An audit clause that requires months of notice while a regulator or customer deadline runs in days is a paper duty.

An evidence break prevents the enterprise from obtaining proof needed for its own decision. A provider model card can describe intended use yet omit the enterprise population, configuration or current endpoint. A change break lets a provider alter model, configuration, data, location, terms or sub-provider before impact assessment and revalidation. A remedy break leaves the affected person without correction or redress because each party points to another.

Graph edges carry duty, authority, evidence, notification, intervention, remedy and exit obligations. Test both directions. Can the enterprise receive notice and proof? Can it issue a stop, preservation, correction or deletion instruction? Can the provider comply before the enterprise deadline? A contract clause is an edge candidate. Exercise, architecture and operating evidence determine whether it works.

Evidence Rights LadderDemand the rung that makes the enterprise decision supportable

Rung 1 is public documentation: policies, model cards, datasheets, security statements and standard certifications. It can support screening for a bounded service. Rung 2 adds customer-specific service identity, configuration, region, usage and change records. Rung 3 provides use-relevant testing and operational evidence, including populations, failure classes, logs and corrective actions.

Rung 4 provides independent assurance or practical audit access where the enterprise needs challenge beyond provider assertion. Rung 5 secures regulatory access, incident cooperation and preserved evidence so the enterprise can investigate and meet binding duties. A right can be delivered through shared assessments, pooled audit, independent reports, supervised access or direct inspection. The mechanism matters less than whether it answers the decision in time.

The required rung rises with consequence, information asymmetry and dependence. A standard writing assistant with no external action, bounded data and easy substitution may stop at Rungs 1 and 2. Regulated decision support with weak population evidence may need Rung 3 or 4. A critical action capability whose incident evidence is solely held by a provider may require Rung 5.

Certifications, model cards, datasheets and SLSA provenance are valuable inputs with boundaries. A certification covers stated systems and periods. A model card reports intended use and evaluation. A datasheet improves data provenance. SLSA describes artifact source or build properties, not model fitness, data rights, service behaviour or customer remedy. Map each artefact to the claim it can support.

Rights must meet architectureA contract cannot perform an intervention

Contract terms and technical design must form one control. A notice clause is useful only if the enterprise receives a machine-identifiable service change, routes it to an accountable owner, assesses the affected capability and can hold, pin, isolate, roll back or revalidate before exposure. An audit right is useful only if its scope, lead time, evidence format, confidentiality route, cost and remediation mechanism allow it to answer the relevant question.

Responsibility allocation must follow actual control. A provider can secure its infrastructure while the enterprise remains responsible for data selection, configuration, entitlements, decision design, monitoring and customer response. The current PRA SS2/21 states that regulated firms remain accountable for regulatory obligations when they outsource. The November 2024 version is current on 2 September 2026; a March 2026 future version becomes effective on 18 March 2027. The time state belongs in the ledger.

US interagency third-party guidance describes a lifecycle of planning, due diligence and selection, contract negotiation, ongoing monitoring and termination, scaled to risk and criticality. It also recognises that requested information may be unavailable. The response is an accountable assessment of whether the bank can manage the relationship, not a presumption that missing evidence is harmless. Similar logic applies outside banking without importing supervisory guidance as universal law.

DORA, GDPR and the EU AI Act allocate duties to defined actors in their applicable scopes. Contractual responsibility can support execution, cooperation and evidence. It does not rewrite a statutory role. The AI Act's value-chain provisions make provider and third-party relationships operationally relevant; GDPR controller and processor arrangements preserve defined responsibilities and assistance; DORA addresses ICT third-party risk for covered financial entities. The ledger records applicability instead of merging these regimes.

Architecture closes rights that paper cannot. Deterministic validation can constrain provider output. Human commitment can separate a provider recommendation from an external decision. Token, data and tool isolation can reduce blast radius. Enterprise telemetry can preserve minimal decision receipts when provider logs are inaccessible. A manual fallback can maintain an essential service. These controls do not excuse a missing duty; they make residual dependence explicit and bounded.

Service identity and revalidationA stable endpoint can conceal a different capability

Third-party AI changes at several layers: base model, fine-tune, safety policy, system instruction, tool access, retrieval source, embedding model, routing, sampling configuration, region, sub-provider, data practice, terms or human-review process. A provider may regard the change as routine service improvement while the enterprise experiences a different decision input.

Record change identity at the lowest observable level and preserve the configuration the enterprise controls. Required notice depends on materiality to the capability, not the provider's product label. Define events that trigger impact assessment, regression testing, approval, customer communication, evidence refresh, rollout restriction or exit. Emergency security changes need a separate path that permits rapid provider action while preserving notice, scope, evidence and retrospective validation.

Version pinning is useful only if the pinned object includes material configuration and if the provider maintains it safely. Rollback is useful only while previous state, data and interfaces remain available and lawful. Where neither is possible, isolate the provider behind a stable enterprise contract and reduce exposure during change. A model-family name and an unchanged URL are not evidence of behavioural continuity.

Continuity and concentrationTwo suppliers can be one failure domain

Vendor count is not diversification. Nominally separate applications may share a foundation model, cloud, region, identity provider, source dataset, vector service, safety API or human-review firm. Map those common nodes and contractual substitution limits. Concentration can also be internal: one skill set, integration pattern or evaluation vendor may make alternatives unusable during an outage.

Continuity testing starts with a capability outcome and its state. Simulate provider unavailability, degraded quality, capacity denial, region loss, compromised evidence and sudden termination. Decide whether to queue, degrade, switch, use a deterministic or human path, restrict scope or stop. A fallback must preserve authority, data constraints, evidence and customer communication; availability alone is not continuity.

Exit includes the service afterlife. Return data, prompts, retrieved context, configurations, fine-tuning artefacts, evaluation results, logs, decision receipts, open incidents, customer cases and in-flight commitments as applicable. Retain the evidence needed for legal, regulatory, audit and customer duties. Verify deletion and revoke entitlements. Reconcile actions that were prepared but not committed and commitments that were made but not completed.

Portability is one control, not the whole exit. A replacement provider may accept the same API while producing different behaviour and evidence. Transition assistance, state reconstruction, parallel validation, customer obligations and decommissioning all need owners and tested clocks. Where a capability cannot be supported through failure or exit, the principal must accept restricted exposure or decline the dependency.

Incident and customer challengeThe shortest external clock governs the cooperation design

Provider notice is not the start of enterprise accountability. Define detection sources on both sides, reportable thresholds, initial notice, update cadence, preservation instruction, investigation roles, containment authority and evidence format. Map every provider clock against enterprise regulatory, contractual, operational and customer-remedy deadlines. A notice that complies with the service agreement can still arrive too late.

Preservation must begin before logs rotate, model identity changes or human notes disappear. The enterprise should maintain its own decision receipt with the minimum necessary data: request and response identifiers, service and configuration identity, policy and approval state, material context references, action or outcome, timestamps and responsible principal. Privacy, security and retention constraints still apply.

Customer challenge crosses the chain in the opposite direction. The enterprise receives the complaint, identifies the affected decision, obtains provider evidence, applies human authority, corrects data or outcome, communicates reasons where required, provides redress and feeds systemic learning into monitoring. Outsourcing the help desk does not outsource ownership of the decision. The provider can assist without becoming the enterprise principal.

Test a provider meeting its own terms while the enterprise misses a tighter obligation. This negative case reveals the real gap. If notice, evidence or remedy cannot cross the chain in time, strengthen rights, isolate the dependency, reduce exposure or reject the capability.

Ten worked casesComplete the ledger, then decide what remains with the enterprise

Each case below compresses all twelve ledger records. The first column links principal, capability, service and provider chain. The second covers data, responsibility, control dependency, evidence and change. The third completes incident, continuity and exit, then names gaps, Evidence Rights Ladder rung and retained enterprise action. These are governance patterns, not universal legal conclusions.

The seventh case is the critical negative control: contractual compliance by the provider can coexist with enterprise failure. The tenth marks a real boundary. A compensating control must actually supply the missing authority, evidence or recovery; a residual-risk memo cannot manufacture an executable edge.

Interactive instrumentThird-Party AI Accountability Board

Load a named scenario or select the closest facts. The board exposes a routing decision and the unresolved chain, not a vendor score, certification count or legal conclusion.

Executive cadenceGovern providers at the speed of capability change

At selection, the business principal defines the capability, authority, population, consequence and completion boundary. Procurement and architecture map the actual service object and provider chain. Legal, risk, security, data and assurance set responsibility and evidence rights in proportion to the use. A certification is screened for scope and version, not counted as approval.

Before release, owners test the duty edges: evidence delivery, service identity, material-change notice, intervention, incident preservation, customer correction, fallback and exit. The enterprise validates its population and configuration. Open orphan, paper, evidence, change or remedy gaps are closed, isolated, explicitly bounded or used to decline onboarding.

During operation, the ledger follows provider, model, configuration, region, terms, data and sub-provider changes. Enterprise telemetry and provider records reconcile. Incidents and customer challenges traverse the whole chain. The capability owner retains the decision to reduce exposure, stop, correct and remedy.

At least on every material change or renewal, leadership reviews concentration, evidence freshness, exercised rights, open findings, recovery and exit readiness. Termination is complete only when state and commitments reconcile, required evidence remains accessible, deletion is verified and customers retain a working remedy route.