Context Caches under Banking Privacy Constraints

A cache hit can be a privacy incident

A relationship manager asks a banking assistant to summarise the open conditions for a corporate borrower. The system retrieves policy, facility state and customer documents, builds a context packet and drafts an answer. Ten minutes later, another employee asks a similar question. Reusing the first packet would reduce latency and cost. It could also disclose a customer name, use a document the second employee cannot see, carry an expired exception or reveal that the first user asked the question.

The risk is not confined to application caches. Search engines cache query results. policy engines cache decisions. connectors cache group membership. databases cache rows. model providers may cache prompt prefixes. browsers and content-delivery networks implement HTTP caching. observability systems retain payloads. A deletion can remove the source while a derived prompt, vector result or response survives elsewhere.

A context cache is a governed derived-data store. Its key must express the security and evidence equivalence required for reuse. Its value needs classification, lineage, encryption, retention and an invalidation contract. A fast lookup cannot bypass a fresh authorisation decision or silently convert a historical packet into current evidence.

This paper uses a fictional “Seabrook Bank” and a composite borrower called “Lattice Marine Services.” Customer names, records, timings, cache ratios, thresholds and incidents are invented design fixtures. Public standards and vendor documentation support the technical discussion. No production measurement or legal conclusion is claimed.

“Cache” names several different controls

Architecture conversations often use one word for layers with different values and threats. A processor cache can reuse a public policy embedding. A retrieval cache can reuse ranked source IDs. A context-packet cache can retain customer text. A provider prompt cache can reuse a prefix inside a managed service. A response cache can preserve model prose containing decisions and personal information.

The first design task is to enumerate layers. Each layer has its own security principal, data form, reuse scope and deletion path.

Some layers improve performance without retaining sensitive content. Caching a tokenizer result for a public policy is different from caching a customer packet. Some layers can retain a one-way identifier yet still leak through access patterns. The inventory should include managed services and client devices, not only code owned by the agent team.

The HTTP caching specification, RFC 9111, defines cache behaviour and directives such as private and no-store in the HTTP context. RFC 5861 defines extensions including stale-while-revalidate and stale-if-error. These standards are useful for web delivery. They do not establish that stale customer evidence is safe for a banking decision. Application-level context needs a stricter, proposition-aware contract.

Caching semantics must be defined at the layer where evidence meaning changes. An HTTP directive cannot decide whether a superseded covenant remains applicable.

The lattice marine case exposes five boundaries

Lattice Marine has a revolving facility, two guarantors and an annual review in progress. Priya is the assigned relationship manager. She asks which conditions remain open. The packet contains current workflow actions, the executed facility amendment, a temporary credit exception and a policy procedure. The answer is useful for Priya.

Five minutes later, an operations analyst asks about another Lattice Marine task. The analyst can see workflow status but not confidential credit papers. Later that day, Priya moves to another portfolio and loses access. Overnight, the exception expires. The next morning, a source document is corrected. A week later, retention policy removes a submitted supporting file.

A simple key such as customer_id + question_hash fails every boundary. It ignores user and service identity, delegation, purpose, case, decision type, source versions, policy release, effective time and processing version. Adding user_id prevents some cross-user reuse but does not handle a revoked user, the same user acting for another purpose, or stale evidence.

The safe question is not “are the prompts similar?” It is “would the current request be authorised to build exactly this value from exactly these dependencies now?” If the answer is expensive to prove, rebuilding may be safer and cheaper than cache assurance.

Build keys from security and evidence equivalence

A cache key is not merely a hash. It is a canonical representation of the dimensions that must be equal for reuse. Seabrook Bank distinguishes key material from validation dependencies. Stable request features can enter the key. Fast-changing facts such as role membership or source status are revalidated through version tokens or event receipts.

The principal chain includes human, agent service and downstream service identities. Delegation matters because the agent may have narrower rights than the user. Purpose names the approved workflow, not a free-text label supplied by the model. Resource scope uses canonical customer, case and document IDs. Effective time distinguishes current queries from historical reconstruction. Processing release covers parser, retriever, prompt and model versions where their change affects the value.

Hashing a customer ID or prompt does not anonymise it. Low-entropy inputs can be guessed, and the entry value remains sensitive. Seabrook uses a keyed digest with controlled key rotation to prevent simple dictionary probing. Cryptographic key management follows institution policy. FIPS 197 specifies AES, while NIST SP 800-57 Part 1 provides general key-management guidance. These documents inform algorithms and lifecycle; they do not design the application’s privacy model.

Encryption protects content from some readers. It does not make an over-broad reuse authorised.

Use a shareability lattice, not one cache tier

The bank classifies candidate values by the broadest safe reuse scope. A value can move only downward to a narrower scope without fresh review. The hierarchy begins with public immutable content, then institution-wide approved content, role or purpose-bound content, case-bound content, principal-bound content and request-bound content.

This is not a sensitivity ranking alone. A public policy paragraph can be shared broadly, but a user-specific annotation on that paragraph cannot. A customer document may be visible to several case members yet still require principal-level reauthorisation. A model response can be less reusable than its evidence because wording may include assumptions and a time-sensitive conclusion.

The lattice makes exceptions explicit. If a team wants to reuse a customer-independent prompt prefix across users, the prefix must contain only common instructions and approved static content. Dynamic customer evidence begins after the cache boundary. If a provider’s cache works on the longest common prefix, ordering becomes a privacy design decision.

Authorise before the lookup and again before return

Checking the cache before identity or policy saves milliseconds by moving the trust boundary to an untrusted key. Seabrook verifies the request envelope first. The key builder receives a canonical principal, purpose and resource scope. On a hit, the service checks current entitlement and every material dependency before decrypting or returning content.

The bank never treats possession of a cache key as capability. Keys are derived inside the service, not supplied by clients. Entry metadata does not expose customer names, denied resources or query text to general operators. Administrative scans use opaque identifiers and tightly controlled break-glass paths.

On return, each source reference is rechecked where risk requires it. A user’s access to the case may remain while one document becomes restricted. The context packet cannot rely only on the case-level permit used when it was built. The validator can remove an ineligible non-decisive item and rebuild coverage, but it must rebuild or fail when the resource set or evidence contract materially changes.

Denied decisions can sometimes be cached more broadly than permits, but even a denial may reveal resource existence or a user’s status. Seabrook uses non-revealing denial classes and short bounds. It does not let a stale denial block newly granted access indefinitely.

Invalidation is a business event graph

Time-to-live is only one invalidation input. The Lattice packet depends on Priya’s portfolio assignment, the agent’s delegated capability, the customer and case, facility state, exception validity, document versions, policy release, index build and evidence contract. Any material dependency can invalidate it.

Entries record dependency IDs rather than copying all content into the invalidation index. A material revocation writes a tombstone through a high-priority path. Reads check tombstones before returning values. Asynchronous workers then remove encrypted values, provider-side cache references where controllable, search artefacts and derived entries.

Events are idempotent and ordered by dependency. Reconciliation catches missed or out-of-order messages. It compares active entries with source and policy versions and samples the encrypted store for orphan values. A cache service that cannot enumerate or target entries by dependency is unsuitable for sensitive context unless entries are strictly request-bound and expire within an approved short interval.

The failure mode is explicit. If the policy service or dependency validator is unavailable, a consequential permit entry is not returned under stale-if-error. The user receives an incomplete or unavailable state. RFC 5861 may be appropriate for public content delivery, but stale customer authority is not a performance fallback.

Availability does not grant permission to use yesterday’s access decision.

Provider prompt caching needs product-specific analysis

Managed model services now document prompt or context caching. Amazon Bedrock prompt caching, Google Gemini context caching, Azure OpenAI prompt caching, OpenAI prompt caching and Anthropic prompt caching expose different controls, durations, eligibility rules and observability. These pages can change. Architecture review should verify the service version, region, model and contract at use time.

The bank needs answers to five questions. What representation is cached? What exact prefix or context determines reuse? What tenant and account boundary applies? How long can the value persist, and can the customer delete it? Does the provider use cached content for any other purpose? Billing discounts and latency do not answer these questions.

Prefix design matters. Put stable public instructions and approved general schemas first. Put user, customer, case and evidence content after the boundary. Do not rearrange sensitive text merely to increase cache hits. If a provider hashes a prefix, identical text across requests may still create correlation or reuse questions that require documented tenant isolation.

The application should not infer that a provider cache hit means the evidence is current. Provider caching reuses computation for input tokens; it does not revalidate a document’s authority or a user’s access. The bank still builds a fresh authorised evidence packet or validates its own packet cache before forming the model input.

Provider metrics can also leak. Cache-hit telemetry, prefix hashes or token counts may correlate with customer activity. Access to these logs is restricted, and dashboards aggregate where possible. Diagnostic payload capture remains off unless a governed incident process approves it.

Retrieval caches must preserve admission stages

A retrieval system can cache query embeddings, candidate lists, fused rankings, reranker scores or final evidence sets. Each stage has a different safe reuse boundary. Query embeddings can encode sensitive terms. Candidate lists can reveal document existence. Final evidence sets depend on current rights, applicability and source status.

Seabrook permits reuse of embeddings for approved public policy queries. Customer-query embeddings are request-bound or encrypted in a purpose-bound namespace. Candidate IDs can be cached only inside the resource and tenant scope, and every hit passes fresh permission and source-status checks. A cached score never causes an item to bypass admission.

The system records whether filters applied before candidate generation or after it. Post-filtering a broad cached candidate set may expose identifiers, scores or timing to a service that should not see them. Logs and caches created before filtering carry the sensitivity of the broad set.

Metrics distinguish raw hits from safe hits. A high candidate-cache hit rate with a low post-validation reuse rate may add privacy surface without meaningful latency benefit. The bank can remove that layer rather than weaken validation.

Response caching is usually the wrong optimisation

A generated response combines evidence, interpretation, model behaviour and speech-act constraints. It may contain a date-relative statement, a missing-evidence conclusion or a recommendation reserved for one role. Reusing it for another request requires proving equivalence across more dimensions than rebuilding from a compact evidence packet.

Seabrook does not cache consequential customer responses as reusable answers. It stores the original output as part of the case record where policy requires it. That record is immutable evidence of what was shown, not a cache entry authorised to answer a new question.

Public educational answers and static help content can be cached under ordinary content rules. Even then, source and release versions matter. A product FAQ should not be served after policy changes merely because the question text matches.

The safer optimisation is to cache lower-level immutable artefacts: public policy embeddings, document extraction tied to an immutable object, generic schemas and model-independent calculations. The final packet still performs fresh authorisation, source status and claim validation.

Cache computation that is stable. Recompute judgement that is contextual.

Encryption and network isolation are necessary, not sufficient

Sensitive entries are encrypted in transit and at rest. Service identities receive least-privilege access to namespaces. Network paths stay within approved boundaries. Keys are separated by environment and, where justified, tenant or sensitivity domain. Backups and replicas receive the same classification and deletion policy.

AWS documents ElastiCache encryption at rest. Redis documents Cloud database security, including roles, network security, TLS and disk encryption. Google documents Memorystore in-transit encryption. These are product capabilities, not complete evidence that a particular cache configuration meets a bank’s privacy or resilience requirements.

Encryption does not prevent an authorised but over-broad application service from returning another user’s entry. It does not remove sensitive data from logs. It does not invalidate a key when source rights change. It does not hide access timing or object size. Application controls and operational testing remain necessary.

The NIST zero-trust architecture publication, SP 800-207, emphasises resource-focused access decisions rather than implicit trust in network location. Seabrook applies that principle to cache entries: being inside a private subnet does not make every context value readable to every workload.

Cache timing and metadata can reveal activity

Even when content is encrypted, a cache can leak through hit timing, key enumeration, size, eviction and telemetry. An attacker who can submit chosen queries may infer that a phrase or customer-specific prefix was processed. A shared cache under load can reveal another tenant’s access pattern through latency.

The bank reduces exposure by using keyed, non-enumerable identifiers; separating sensitive namespaces; restricting metrics; padding or bucketing sizes where justified, and avoiding shared cross-tenant caches for customer context. Rate controls and anomaly detection watch probing patterns. Error messages do not distinguish “entry exists but you lack access” from “no entry.”

Constant-time behaviour across a distributed cache may be impractical. The threat assessment focuses on which actors can measure timing, what they can choose, and what facts a hit would reveal. Highly restricted content may be non-cacheable because the side-channel assurance cost exceeds the latency benefit.

Privacy requirements become technical invariants

The EU General Data Protection Regulation is available in the official EUR-Lex text, and the European Data Protection Board publishes Guidelines 4/2019 on data protection by design and by default. Applicability and interpretation require legal and privacy owners. The architecture does not claim that a cache design is legally compliant.

It translates approved privacy requirements into testable invariants: do not cache more content than the purpose needs; do not share beyond the authorised audience; retain for a bounded period; maintain source and derivation lineage; support correction and deletion; expose use to accountable operators, and fail closed when equivalence cannot be proven.

The NIST Privacy Framework provides an organisational privacy-risk resource, while NIST SP 800-122 addresses confidentiality of personally identifiable information. Seabrook uses them as reference points for inventory, governance and protection, alongside local requirements.

Erasure is a dependency walk, not a key delete

When Lattice Marine’s supporting file reaches an approved deletion event, the source store removes or tombstones it according to policy. The event then traverses derived artefacts: rendered pages, OCR, embeddings, retrieval results, context packets, response records where eligible, logs, evaluation samples and backups. Cache entries are only one branch.

Provider-side caches create a special question: can the institution target a specific prefix or context for deletion, or only wait for expiry? If targeted removal is unavailable, sensitive content may be excluded or retention must fit approved constraints. The decision record states the limitation rather than assuming application deletion reaches the provider.

The erasure service uses lineage IDs, not plaintext searches, to find dependants. It produces counts and receipts without exposing contents to general operators. Holds and exceptions are explicit, time-bound and owned. Restoration tests ensure that deleted entries do not become active when a backup is recovered.

Corrections need supersession rather than simple deletion. Historical packets retain what was used at the time under approved retention, while current cache lookups cannot return the old value as evidence. The lineage service marks dependent summaries stale and identifies prior cases for materiality review.

The composite walkthrough: a safe hit and a forced miss

At 09:00, Priya asks for open credit-review conditions. The context service verifies her identity, assignment, purpose and case. It builds a packet from current workflow state, the executed amendment, the temporary exception and applicable policy. The packet is encrypted in a principal-and-case namespace with dependency receipts and a short clock bound.

At 09:02, Priya asks for the sources behind condition three. The request has the same principal, purpose, case, evidence-contract release and dependency versions. The cache service reauthorises her and validates the document and exception states. It returns the packet, and the interface opens the cited amendment page. This is a safe hit.

At 09:05, the operations analyst asks about workflow actions. The key differs by principal and purpose. The service does not reveal that Priya’s entry exists. It builds a narrower packet containing workflow status but not credit papers. Similar wording does not create reuse.

At 13:00, Priya’s assignment is revoked. An identity event tombstones entries dependent on the assignment. Even if invalidation lags, the fresh policy check denies her next request. At midnight, the exception expires, invalidating any current assessment packet. The public policy embedding remains reusable because its content and authority did not change.

On day two, the amendment metadata is corrected. Entries depending on the old document-status version become stale. The context packet is rebuilt, and the correction receipt links affected prior outputs. On day seven, the supporting file deletion triggers the lineage walk. The system verifies removal from its own caches and records provider-expiry limitations for any managed cache that cannot be targeted.

The walkthrough demonstrates why “TTL of fifteen minutes” is not a privacy design. A role can change in one minute; a stable public embedding can be valid for months. Validity follows dependency semantics, not one universal clock.

Failure modes and safe responses

Cache failures often look like performance defects until sensitive content appears. The threat model includes accidental collision, stale authorisation, cross-purpose reuse, undeleted derivations, fail-open behaviour and operator exposure.

Incidents preserve metadata needed to determine affected entries without copying values into tickets. Investigators receive elevated, case-bound access to decrypt where necessary. The bank rotates keys only when the threat requires it; indiscriminate rotation can cause availability issues without removing plaintext from logs or backups.

Governance follows the cache lifecycle

The context team cannot own source retention, identity revocation, provider contracts and cryptographic keys by itself. Seabrook assigns control ownership across the value lifecycle. Workflow owners approve purpose and speech acts. Data owners approve source reuse. Security owns identity, namespace and negative testing patterns. Privacy and legal owners determine applicable constraints. Platform teams operate storage and events. Independent validation challenges evidence.

The data contract states allowed values, maximum share scope, classification, regions, encryption, clock bound, event dependencies, deletion, backup, provider use and fallback. A code change cannot widen share scope without reapproval. Infrastructure templates create namespaces and keys from approved declarations rather than manual console settings.

Access review covers application roles and humans. A support role that can read all cache values is treated as privileged content access, not ordinary operations. Break-glass use links to an incident, expires automatically and triggers review. Provider support arrangements receive the same attention.

Change triggers include cache product upgrades, topology or region changes, key-schema changes, new prompt-cache capability, new model provider, broader document classes, longer retention, new metrics and any request to share across principals. The regression scope follows affected invariants. A storage patch may need resilience tests; a key change needs collision and isolation tests; a provider change needs contract, privacy and prompt-cache tests.

Incidents classify exposure by content and observation. Returning plaintext to the wrong user differs from a timing signal, but both require analysis. The dependency index identifies potentially affected entries and requests. Notifications and legal obligations are decided by authorised teams, not the agent.

A release gate that can say no

Before a cache layer is enabled, the team must demonstrate a material benefit under controlled semantics. The release gate asks whether the value is rebuildable, whether its share scope is explicit, whether current authorisation happens on every hit, whether dependencies are complete, whether deletion reaches every copy, whether failures close safely and whether provider assumptions are evidenced.

An entry class is rejected when any critical field is unknown. Unknown provider retention, inability to separate tenants, missing source versions, no targeted invalidation for volatile state, or no way to account for deletion are not documentation gaps to fill after launch. They change the architecture choice.

The gate also rejects misleading metrics. A raw 80 percent hit ratio means little if half the hits fail revalidation or apply only to public text. Report safe hits by data class, latency after controls, source load avoided, invalidation performance and residual retention. Do not combine public and customer caches in one success rate.

The strongest outcome can be a narrower cache. The bank may approve public policy prefixes and immutable extraction while rejecting customer context packets. That is not a failed optimisation programme. It is evidence that the decision process distinguished stable computation from volatile authority.

What must never be hidden in a cache

Some values belong in systems of record or not in storage at all. A model’s unapproved decision, a hidden risk label, a raw chain-of-thought transcript, a broad service token, a SAR-derived hint in a lending namespace and an unauthorised copy of a customer document should never acquire legitimacy through persistence.

Temporary storage does not change the rule. If processing needs a short-lived sensitive value, use request-local memory inside the approved boundary, minimise it, prevent swapping or diagnostics where required, and destroy it after use. A crash dump, tracing agent or debug snapshot can otherwise turn memory into durable storage.

The system also avoids negative knowledge caches that reveal protected facts. “Customer has no investigation” and “customer investigation denied” are different internal states but may require the same external response. Cache metadata uses non-revealing classes.

Observability should prove controls without copying values

Cache teams need enough evidence to diagnose misses, races and invalidation without turning telemetry into another plaintext cache. Seabrook’s default event records an opaque request ID, principal-class reference, purpose code, namespace, entry class, hit or miss state, dependency-validation result, source-build state, latency bucket, invalidation generation and control error. It does not record the prompt, customer name, document text or decrypted value.

Detailed content tracing is a separate governed mode. An authorised incident owner selects a bounded case, purpose and duration. The trace uses redaction, access expiry and a dedicated store. Enabling it for one case cannot switch on payload capture for the whole service. Every access and export is reviewed.

Cardinality deserves care. Hashing a customer ID into a metric label can still create one label per customer and allow correlation. Routine metrics aggregate by cache class, workflow, region and control outcome. Opaque request IDs live in access-controlled logs for a shorter operational period. Analysts join them to case information only through an approved investigation.

Canary values test the telemetry boundary. Synthetic customer documents contain unique markers that should appear only in the approved source and request. Automated scans search model requests, logs, metrics, crash reports, support exports and cache snapshots. A hit outside the allowed set blocks release or opens an incident.

Operational dashboards show safe-hit rate, revalidation rejection, event-to-tombstone latency, purge completion, orphan-entry reconciliation, policy-outage denials and break-glass access. A sudden fall in revalidation rejection can mean fewer changes, but it can also mean the check stopped running. Sentinels therefore pair rates with direct canary transactions.

Observability data follows its own retention and erasure rules. Removing the cache entry does not automatically remove an incident trace. The lineage record links them so the approved policy can decide each artefact explicitly. The control record should survive long enough to evidence behaviour, while the sensitive value should not survive merely to make monitoring convenient.

Service objectives combine performance and control. A latency objective applies only to requests that completed fresh authorisation and dependency checks. A cache service cannot meet its objective by returning content when a policy dependency was unavailable. Error budgets count privacy-boundary failures separately from ordinary misses or timeouts; one cross-tenant return is not averaged into millions of successful reads.

Capacity forecasts use entry classes rather than customer identifiers. Operators can see that principal-bound credit packets consume a certain tier without seeing who owns them. When an entry class grows unexpectedly, investigation begins with deployment, workflow and retention metadata. Content access occurs only if those signals cannot explain the change and an authorised case is opened.

Runbooks state which actions are safe without decryption: isolate a namespace, block writes, apply a global deny generation, disable provider caching, purge a release, rotate a namespace key or fall back to fresh source builds. This lets responders contain risk quickly while preserving evidence for a controlled investigation.

After containment, recovery starts from governed sources and current policy rather than trusting surviving entries. The team rebuilds only approved cache classes, verifies tombstones and key generations in every region, reruns isolation canaries, and samples receipts before reopening traffic. Customer-context caches remain empty until the evidence shows that current authorisation, invalidation and erasure controls are operating together. This cold-start discipline keeps a restored optimisation from becoming a restored exposure.

Source position: standards, products and boundaries

RFC 9111 and RFC 5861 support discussion of HTTP cache controls and stale extensions. They do not make stale application evidence safe. NIST publications support privacy, PII protection, zero trust, cryptography and key-management concepts within their stated scopes. The EU sources support legal and regulatory context that requires local interpretation; they are not used here to declare compliance.

Vendor prompt-cache and managed-cache pages describe current product capabilities. Their semantics differ and can change. They do not establish tenant isolation, deletion, residency or contractual fitness for Seabrook Bank. The institution must verify configuration, contract, region, model and date.

This synthesis is technical architecture guidance, not legal advice, a vendor endorsement or evidence of a deployed control environment. Product documentation and regulatory expectations should be checked at implementation and release time.

The decision rule

Cache public, immutable and broadly authorised computation where reuse is clear. Cache customer-derived values only when the bank can state and test the exact principal, purpose, resource, policy, time, version, retention and deletion equivalence. Reauthorise every hit. Validate material dependencies. Treat source and entitlement events as invalidation signals. Keep provider caches inside a product-specific risk decision.

Do not cache consequential model responses as reusable decisions. Prefer lower-level immutable artefacts and typed receipts. Fail closed when authorisation or dependency services are unavailable. Measure safe hits after controls, not raw hits before them. Remove a cache layer when its assurance surface exceeds its demonstrated benefit.

The fastest context is not the one found in memory. It is the smallest evidence packet that can still prove why reuse was permitted now.

That proof is the real unit of reuse.