BigQuery as the Memory Layer for Agent Platforms

Every agent architecture diagram has a box labelled "memory," and most teams fill it with whatever vector database was trending that quarter. In the enterprises I work with, that box increasingly says BigQuery, and for reasons that have less to do with vector search benchmarks and more to do with governance, cost mechanics. The fact that the data agents need to remember is already there.

Let me be precise about scope. Agents need several kinds of memory: short-term session state (conversation context (keep it in your runtime or a low-latency store), operational memory (recent facts needing millisecond lookups) a transactional or cache tier). Analytical memory: the accumulated record of what agents did, what users asked, what documents say, and what the business knows. That third category is where BigQuery is, in my experience, the right default on GCP.

Why the warehouse, not another database

Three arguments carry the decision in architecture reviews:

The data is already governed there. In a bank, the customer, product, and transaction data an agent needs context from already lives in BigQuery under existing access controls, lineage, and retention policy. Copying it into a standalone vector store creates a second, worse-governed copy of regulated data: a finding waiting to happen. Bringing embeddings to the warehouse keeps one governance regime.

Agent exhaust is analytical data. Every tool call, grounding decision, and response your agents produce is an event stream you will need for audit, evaluation, and model improvement. Land it in BigQuery from day one and your audit trail, your eval sets, and your "what did the agent do on March 3rd" answer all come from the same tables.

SQL is the interface your organisation already trusts. When memory is tables, your risk team can query it, your BI stack can dashboard it. Your data engineers can maintain it without learning a new system.

The embedding and vector search machinery

BigQuery's AI surface has matured into something genuinely usable:

• ML.GENERATE_EMBEDDING calls an Agent Platform embedding endpoint over a table column through a BigQuery remote model. This makes batch embedding generation a SQL operation rather than a separate pipeline.
• VECTOR_SEARCH performs similarity search over an embedding column, with vector indexes (IVF and TreeAH types) to accelerate approximate search over large tables. Without an index it brute-forces: exact, but you pay in scanned compute.
• AI.GENERATE is the current scalar SQL function for Gemini inference over structured and unstructured values; it returns a STRUCT containing the result, full response and per-row status, and can return fields defined by an output schema. AI.GENERATE_TEXT is the table-valued text-generation path through a BigQuery ML remote model. Both consume model quota and can return row-level errors even when the query job completes, so production batch jobs must inspect status and retry only failed rows. Google documents the choice and current limits in Choose a text generation function and the AI.GENERATE reference.

The honest performance caveat: BigQuery vector search is an analytical capability, not a universal serving tier. Warehouse latency suits batch retrieval, analytical lookups, evaluation and memory consolidation. A strict conversational latency target may require Agent Search or Vector Search as a governed projection. BigQuery remains the system of record that can rebuild those serving indexes.

Patterns that work

The reflection pipeline. Nightly scheduled queries embed the day's agent conversations, cluster them, and write summarised "lessons" to a memory table the agents query for context. Long-term memory as a batch job: boring, cheap, auditable.

Semantic joins for entity resolution. VECTOR_SEARCH can match unstructured mentions against embedded master-data descriptions. Treat it as a candidate-generation step: exact identifiers and deterministic rules should take precedence, and a labelled entity-resolution set must establish whether the semantic path improves recall without creating unacceptable false matches.

Governed text-to-SQL surfaces. Expose curated, documented views (not raw tables) to SQL-generating agents. The view layer is your contract: column descriptions become the schema context in the prompt, and IAM on the views bounds what the agent can ever see.

Eval sets as tables. Golden questions, expected groundings, and historical agent outputs live in BigQuery; evaluation runs are queries. When the model risk team asks how you validate agent behaviour, you show them tables and scheduled jobs, not a notebook.

Cost patterns: where teams get burned

BigQuery's economics reward deliberate design and punish naive RAG habits.

Compute model first. On-demand pricing bills per bytes scanned; capacity pricing (editions) bills for slot commitments with autoscaling. Agent workloads are spiky and repetitive: thousands of small, similar queries. Spiky exploration starts on on-demand; sustained agent traffic almost always crosses into slot-commitment territory. Model the crossover before launch, not after the first invoice.

Bytes scanned is the enemy. An agent that issues SELECT * against an unpartitioned event table can scan the full eligible table on every turn. Partition memory and event tables by the time key used in access paths, cluster on frequently filtered entity keys, and enforce partition filters where the query contract permits it. Measure bytes processed and latency before and after; do not import an “order of magnitude” saving from another workload.

Embedding generation carries model-endpoint cost as well as BigQuery compute. ML.GENERATE_EMBEDDING calls a paid embedding endpoint per row. Embed incrementally, using change timestamps to select new or altered records. A full-corpus re-embedding run should be a controlled model migration, not a routine schedule.

Cache and materialise. Agents ask the same analytical questions repeatedly. Materialised views for the common aggregates, and an application-level cache keyed on normalised question embeddings, remove a shocking fraction of query volume.

Set guardrails. Per-project and per-user custom quotas on bytes scanned, and budget alerts scoped to the agent platform's project. An agent in a retry loop is a cost incident; make it a bounded one.

Governance: the actual reason this wins

Everything above is optimisation. This section is why the architecture gets approved.

• IAM on datasets and views gives each agent service account exactly the surface it needs. Per-agent identities, no shared platform account.
• Column-level security via policy tags (Dataplex/Data Catalog taxonomy) keeps PII columns invisible to agents that do not need them: the query fails at parse time, which is exactly the failure mode you want. Row-level security scopes multi-tenant memory tables by business line.
• Dynamic data masking lets an agent compute over sensitive columns' shape without reading raw values where policy allows.
• Audit logs and lineage come free: every agent query is in Cloud Audit Logs, and lineage tracking answers "which source tables influenced this memory" without extra tooling.
• Retention and deletion are table-level policies. When legal says conversation memory expires at N days, that is a partition expiration setting, not an engineering project. Deletion requests against memory stores (a genuinely hard problem in opaque vector databases) become DML.

That last point deserves the emphasis: the right-to-erasure story for a vector store is usually hand-waving. In BigQuery it is a DELETE statement with an audit log entry. In a regulated environment, that argument alone has settled the memory-layer debate more than once.

Define the memory contract before selecting the store

“Memory” is too broad to be an architecture requirement. A useful contract names the subject, writer, reader, retention period, freshness, correction process and permitted purpose. It also states whether a remembered item is authoritative, inferred or merely observed. Without that distinction, a model-generated summary can quietly acquire the status of a client fact.

The safest design treats every memory write as a proposal to a governed data product. Runtime traces can land immediately because they record events. A durable client preference needs a declared purpose and provenance. A model-generated lesson should remain an inference with confidence, expiry and contributing evidence. It must never overwrite a system-of-record field.

This classification also determines correction. Events are appended with compensating events. Authoritative facts are corrected in their owning system and then propagated. Preferences are versioned and can be revoked. Inferences are recomputed or expired. One generic memory_upsert tool cannot preserve these different semantics. Separate interfaces make the model's authority visible.

Build the agent ledger as a reconstructable event model

An audit table should not be a dump of whatever the observability library emitted. Design an event model that remains stable while models and orchestration frameworks change. A practical grain is one row per component event: user input received, policy decision made, retrieval result selected, tool proposed, tool executed, approval supplied and answer delivered.

Each row needs an immutable event identifier, trace and parent identifiers, event time, principal, journey, component version, data classification and outcome. Larger payloads can live in a protected object store with a content hash and retention policy. The BigQuery row retains the evidence pointer. This avoids forcing every prompt and document into a broad analytical dataset.

The hierarchy enables several questions without parsing prose. Which policy version allowed a payment tool? Which retrieved passage supported a claim? Which model release changed refusal rates for vulnerable customers? Which sessions used a document later withdrawn? Those are relational questions. BigQuery is effective because joins, temporal filters and cohort analysis are ordinary operations.

Keep identifiers consistent across online traces and analytical rows. The runtime should return the trace identifier with an operational error so support staff can find the full chain. Evaluation jobs should retain the same turn identifier while adding their own run and rubric versions. Observability explains what happened now; the ledger preserves what can be proved later.

Google's BigQuery audit-log reference explains platform access records. Those logs show data-plane activity but not the full business decision. The application event model must add policy intent, human approval, tool semantics and outcome.

Promote governed projections instead of copying uncontrolled memory

Low-latency retrieval may require Agent Search, Vector Search or another serving index. Treat that index as a projection, not a second authority. A promotion job selects approved rows from curated BigQuery views, transforms them into chunks, attaches access metadata and writes a versioned index. The job also writes a manifest containing source partitions, transformation version, embedding model and item counts.

Never mutate the only live index during a model or chunking change. Build a new version, run a frozen evaluation set, send shadow queries and switch an alias only after acceptance. The manifest makes rollback deterministic. It also lets reviewers identify every answer produced from an affected corpus version.

Access metadata must enter the projection before retrieval. Post-filtering a shared result set can leak existence, ranking and snippets. The serving path should accept a signed entitlement scope and restrict candidate generation. The source view and index must use the same classification vocabulary. Otherwise an item can be approved in one layer and misread in another.

The Vector Search filtering documentation describes token and numeric restrictions available in that serving technology. The exact mechanism varies by product. The invariant is constant: unauthorised material must not become a candidate.

Deletion, correction and retention need end-to-end proofs

A deletion statement in BigQuery is only the first step when downstream projections, caches and evidence objects exist. Every derivative needs a documented deletion or rebuild path. The inventory should cover table partitions, materialised views, search indexes, caches, exports, backups and evaluation datasets. Legal holds need a separate path because ordinary expiration must not silently remove preserved records.

Use a deletion ledger rather than relying on a successful job status. Each request records the subject, legal basis, approved scope, affected assets, execution time and verification outcome. A verifier queries the source and every derivative after propagation. Failures become work items with owners. This produces evidence that a request was completed, not only attempted.

Retention should follow purpose. A short-lived conversational checkpoint, an operational decision record and a model-validation dataset do not need the same period. Longer retention can improve analysis while increasing privacy and discovery exposure. Make that trade-off visible to the accountable owner. Partition expiration is an implementation of the approved schedule, not the schedule itself.

Evaluate the memory layer as a business control

Warehouse health metrics do not show whether memory helps the agent. Measure four levels. Data tests cover completeness, schema and freshness. Retrieval tests cover recall, precision and entitlement. Decision tests cover whether the correct fact changed the proposed action. Outcome tests cover rework, client harm and human effort.

A useful ablation removes one memory source at a time. If deleting inferred long-term memories does not change task success, stop paying to create them. If structured account facts materially reduce wrong recommendations, invest in their freshness. This approach prevents memory volume from becoming a vanity metric.

Operating cost should be allocated by journey and stage. Record bytes scanned, slot time, model calls, index reads and storage against the trace identifier. Then calculate cost per completed business outcome, not cost per model turn. A lower per-turn design can be more expensive if it causes retries or human correction.

The BigQuery INFORMATION_SCHEMA job views provide query and resource-consumption details for this analysis. Join them to the application ledger through labels or mapped job identifiers. The resulting evidence supports capacity decisions with observed distributions rather than generic thresholds.

The decision is not BigQuery versus a vector database. The decision is which layer owns truth, which layer serves each latency class, and how every derivative remains governed. BigQuery earns the central role when it provides the durable, queryable and reconstructable record from which other stores are built.

Consolidate experience without manufacturing facts

Long-term agent memory is often described as “reflection”: summarise recent conversations, identify a lesson and retrieve it later. That pattern is useful for operating knowledge but dangerous for client facts. A generated summary can compress uncertainty, omit an exception or combine two people. Once stored beside authoritative records, its polished language can make it appear equally reliable.

Separate observation, interpretation and approved knowledge. The event ledger records what happened. An enrichment job proposes an interpretation with its source event identifiers. A deterministic gate checks schema, consent, sensitive-data policy and expiry. Material knowledge then needs an accountable reviewer or an approved rule before it enters a reusable view.

For example, repeated support calls may suggest that a product explanation is confusing. The stored insight should identify the query cluster, time window, sample size and method. It should not claim that a named client “does not understand the product.” The former supports service improvement. The latter creates an inferred personal attribute without a clear purpose.

Consolidation also needs decay. Procedures change, product terms expire and behavioural patterns drift. Give inferred memories a valid_until value and a source-version dependency. A policy update can invalidate conclusions drawn from the earlier edition. Retrieval should exclude expired inferences automatically rather than asking the model to notice a date in prose.

Use an evidence threshold based on consequence. A routing hint can be promoted from repeated low-risk observations. A recommendation affecting client treatment needs stronger evidence and review. A durable fact about identity, balance or eligibility should come only from the authoritative source. Reflection may improve the system's hypothesis; it must not create business truth.

Establish a practical migration path

Most institutions do not begin with a clean memory architecture. They have model traces in logging tools, conversation exports in object storage, evaluation labels in spreadsheets and one or more vector stores. A successful migration does not move everything into one giant table. It establishes a governed record and then reconnects each purpose to it.

Begin with the decision ledger. Define stable session, turn, event, evidence and configuration identifiers. Land new production events first while leaving historical systems in place. This creates a clean cutover date and avoids delaying controls for a difficult backfill.

Next, build curated views for investigations and evaluation. Reconcile a sample against the original telemetry. Where historical payloads lack version or evidence identifiers, label the limitation rather than inventing values. A partial but honest record is more useful than false completeness.

Then move retrieval projections under manifest-based promotion. Keep the current index serving while the BigQuery-backed pipeline builds a parallel version. Compare results on a labelled set and shadow traffic. Switch only when recall, entitlement and latency meet the journey's thresholds. Retain the previous index through an agreed rollback period.

Finally, retire duplicate stores by purpose. A low-latency session database may remain because BigQuery does not replace it. An unmanaged copy of conversation history may not. Document the surviving stores, their authority and their deletion path. This is a portfolio decision, not a technology-purity exercise.

The migration programme should measure control coverage as well as completion. Useful measures include the proportion of delivered answers with reconstructable evidence, indexed items with current entitlement metadata, deletion requests verified across derivatives, and production configurations represented in a release manifest. These measures show whether the estate is becoming governable.

Assign ownership at the seams

Memory systems fail at interfaces between teams. The data platform owns tables but not the meaning of a client journey. The agent team owns orchestration but not source-data quality. Privacy sets purpose and retention but does not operate index deletion. Make seam ownership explicit.

The data-product owner is accountable for conformance, freshness and lineage. The journey owner decides which facts are appropriate and how stale data affects the user. The agent-platform owner operates runtime event capture and projection tooling. Privacy approves purpose, retention and subject-rights handling. Model risk or independent validation tests the complete behaviour where the institution's framework requires it.

Incidents should follow the same split. A late source event belongs first to the data product. An authorised fact ignored by generation belongs to the agent. A restricted document entering a candidate set belongs to projection controls. A deletion that succeeded in the warehouse but not the index spans platform and privacy ownership.

The seams need service-level agreements and shared identifiers. Without them, each team can show its component was healthy while the client received a wrong answer. End-to-end evidence prevents local success from disguising system failure.

The shape of the recommendation

Use BigQuery as the analytical memory and system of record for your agent platform: all agent events, embeddings, eval sets, and long-term memory tables live there under warehouse governance. Generate embeddings incrementally with ML.GENERATE_EMBEDDING, serve batch and analytical retrieval with VECTOR_SEARCH plus vector indexes, and project hot-path retrieval into a purpose-built serving index downstream. Partition, cluster, quota, and commit to slots when traffic stabilises.

It is not the most fashionable memory architecture. It is the one that survives both the invoice and the audit.

The memory topology in one view

BigQuery should not be made to serve every latency class. System of record and serving index are different roles. The warehouse owns governed history. A lower-latency index serves a controlled projection and can be rebuilt.

The implementation surface is documented in BigQuery vector search, embedding generation, row-level security and column-level access control. Read the current limits before choosing a serving path. Product availability is a design input; governance responsibility remains with the operator.