The shape of the answer
The original retrieval-augmented generation formulation paired a generator with non-parametric memory, retrieving passages that could supply knowledge absent from model parameters.[1] Dense passage retrieval made semantic candidate selection practical for open-domain question answering.[2] Those ideas remain powerful. They also carry a quiet assumption: a useful passage can be ranked independently because the answer-bearing evidence is sufficiently local.
Locality is not the same as shortness. A twenty-page policy can still have a local answer if one section states the applicable notice period. Conversely, a two-sentence answer may be non-local when one sentence identifies a party, a second establishes a guarantee, and a later event cancels it. Chunking changes the unit the retriever can see. It does not change the dependency structure of the question.
Three burdens make the chunk assumption fail. Identity burden appears when names, aliases and mentions must resolve to the same real entity, or to deliberately different ones. Relational burden appears when the answer depends on a typed path whose intermediate nodes may never co-occur with the query. temporal burden appears when event order, effective intervals or the time at which the system learned a fact can reverse the answer.
Thought experiment one: the namesake switch
Keep every word in the corpus fixed except one identifier. Two companies now share the display name “North Quay Components”. Their documents remain semantically similar, and the vector scores barely move. Yet the correct facilities change completely. The causal variable is identity, not relevance. A stable party ID and an authorised resolution step fix the failure; a larger embedding does not.
This gives a practical test. Ask what must remain invariant while the wording changes. If the answer survives paraphrase but fails when an identifier changes, identity carries the task. If the answer changes when one edge is removed, relation carries it. If it changes when one event moves earlier or later, time carries it. The minimal sufficient representation is the smallest structure that preserves those answer-changing variables.
A counterfactual test for representation sufficiency
Let an answer be written as A = f(I, R, V, K, D). Here I is resolved identity, R is the eligible relationship path, V is valid time, K is knowledge or recorded time, and D is documentary language. The notation is not a claim that every answer is neatly calculable. It is a discipline for asking which inputs can change the output.
Vary one input while holding the others fixed. Replace the company ID but keep the name and documents. If the answer changes, I cannot be delegated to semantic similarity. Remove one intermediate guarantee edge while keeping all passages. If the answer changes, the retrieval plan must preserve R. Move the facility-closure event across the decision time. If the answer changes, V matters. Delay when the correction is recorded. If the reconstructed historical decision changes, K matters. Finally, paraphrase the clause while preserving its legal effect. If the retrieval result changes, the documentary mechanism must handle D robustly.
This test exposes a common category mistake. Teams often evaluate an embedding model against questions whose gold answer already assumes correct entity resolution and current state. The retriever appears responsible for the whole result, even though the benchmark silently supplied the hardest joins. In a live system, those joins are not free. They are separate decisions with owners, permissions, failure states and evidence.
A sufficient representation does not need to contain every fact. It needs to preserve every variable whose counterfactual change would alter the answer within the declared scope. That is why a compact state snapshot can be safer than a huge context window, and why a graph of approved edges can be more useful than a richer graph of every extracted association. More context is not more truth when the missing property is structure.
flowchart LR A["One question, three kinds of truth"] --> B["On this page"] B --> C["The shape of the answer"] C --> D["Three mechanisms, three<br/>legitimate jobs"] D -. "re-test" .-> A classDef first fill:#0a1424,stroke:#7B4CD8,stroke-width:2px,color:#e6f3ff classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff class A first class B,C,D step
Three mechanisms, three legitimate jobs
RAG, GraphRAG and relational state are often compared as retrieval products. That comparison hides their mechanisms. RAG ranks candidate text. GraphRAG organises documentary structure to support expansion or summarisation. Relational state maintains identified facts under controlled change. The same backend can implement more than one mechanism, but the semantics remain distinct.
RAG lens
What text may be relevant?
Ranks chunks by lexical, dense or hybrid similarity, often followed by reranking.
Strength: source language and local detail.
Blind spot: similarity does not establish identity, path validity or current truth.
GraphRAG lens
What connected evidence or global structure may matter?
Uses entities, relations, communities or graph-guided expansion to assemble context.
Strength: dispersed and corpus-wide dependencies.
Blind spot: extracted edges and summaries are derived artefacts.
State lens
What identified fact is valid and knowable at this time?
Uses keys, constraints, transactions, events and temporal queries.
Strength: invariants and controlled change.
Blind spot: rich contractual language does not fit fixed columns by magic.
RAG is a candidate generator
Dense retrieval can find paraphrases that lexical search misses, while hybrid retrieval can preserve exact identifiers and rare terms. The BEIR benchmark found substantial variation across datasets and showed that BM25 remained a robust zero-shot baseline, while more expensive reranking and late-interaction methods performed strongly on average.[3] The durable lesson is not that one retriever wins. It is that ranking quality depends on the corpus and the information need.
A retrieved passage carries at least four separate claims: that the source is the right document, that the version is applicable, that the passage supports the proposition, and that the requester may see it. A similarity score establishes none of these. Metadata filters help only when identifiers, status, time and entitlement were correctly produced before indexing. Vector proximity is evidence about relevance, not evidence about truth or authority.
GraphRAG reconstructs documentary structure
The Microsoft GraphRAG method extracts entities and relationships, clusters the resulting graph and produces community summaries for query-focused summarisation across large corpora.[4] Its official documentation distinguishes local, global and DRIFT search and describes the graph, text units and community reports used during querying.[5] The open repository also warns that indexing can be expensive and that the code is a methodology demonstration rather than an officially supported offering.[6]
That mechanism is valuable when the question asks for themes across a corpus, when evidence is distributed over several documents, or when a bridge entity guides retrieval. Multi-hop datasets such as HotpotQA and MuSiQue were designed around dependencies between supporting facts, including questions where one reasoning step depends on another.[7][8] Graph structure can make those dependencies available to the retriever instead of hoping that one embedding query brings every hop into the top-k set.
Yet a graph edge has no inherent evidential grade. RDF formalises graphs as sets of subject-predicate-object triples, and SPARQL formalises graph-pattern matching.[9][10] Neither standard says that an edge extracted from a meeting note is an approved legal relationship. GraphRAG must therefore retain edge class, source span, extraction release, confidence where meaningful, permitted use and effective interval. An extracted controls edge may generate a lead. It should not silently widen customer scope.
A documentary graph is a read model
A GraphRAG index is normally built by transformation. Documents are chunked, entities are resolved or proposed, relations are extracted, communities are calculated and summaries are generated. Every stage can lose, merge or reinterpret information. The resulting graph is therefore a read model optimised for particular questions. It is valuable precisely because it is not a copy of the source corpus. The same fact also limits what it can author.
Suppose a memo says, “The supplier appears dependent on East Port Motors.” An extractor creates supplier_of. A later community summary calls the companies “tightly connected”. Neither operation proves a contractual dependency. The source itself was tentative; extraction discretised the wording; summarisation strengthened it. Without provenance and evidence class, the graph has converted an observation into an assertion and an assertion into apparent state.
The safe pattern is to retain at least three layers. The source layer contains immutable document versions and spans. The derived graph layer contains extracted entities, relations, clusters and summaries with lineage. The adjudicated relationship layer contains the small set of edges approved for operational use. A derived edge may suggest which source to inspect. Promotion into the adjudicated layer requires a named process, evidence and effective time. This is slower than treating every edge equally, but it makes correction and challenge possible.
GraphRAG solves context selection, not institutional truth maintenance. When a source is corrected, the graph release must be rebuilt or patched and every dependent summary reconsidered. When an adjudicated relation changes, the authoritative service records the event immediately and retrieval projections catch up under a declared lag. Conflating the two creates a system that is expensive to update yet still unable to explain what was true when.
Relational and event state preserve invariants
Operational facts have lifecycle rules. A facility cannot be closed and awaiting first drawdown under the same definition. A case transition may require a prior state, an authorised principal and a reason. Transaction isolation exists because individually plausible reads can form an impossible combination when concurrent updates interleave. PostgreSQL documents multiversion concurrency control and serialisable isolation semantics; Spanner is a published example of externally consistent distributed transactions.[11][12]
Relational state here does not mean “put everything in SQL”. It means that exact identity, constraints, versioned updates and time semantics are part of the answer. A graph database can hold such state if its service contract enforces them. A relational database can execute bounded graph traversals. The relevant distinction is authoritative mutable state versus derived retrieval structure, not rows versus nodes.
Thought experiment two: the delayed correction
Keep the same entities, documents and graph topology. Move only one correction event. A guarantee ended on Monday, but the source system learned this on Wednesday. A Tuesday review asks two different questions: “What was legally true on Tuesday?” and “What could the reviewer have known on Tuesday?” Valid time answers the first; transaction or recorded time answers the second. A timeless graph makes the two histories indistinguishable.
Negative control: the static handbook question
Ask, “How many calendar days does the supplier handbook allow for an appeal?” The current handbook has one applicable version, one paragraph states the answer, no entity resolution is required and no relation must be traversed. A well-evaluated hybrid retriever is the serious baseline. Building a graph adds extraction cost and new error paths without changing the answer. This control prevents the argument from becoming “graphs everywhere”.
A matched retrieval experiment
Architecture arguments become vague when each method is tested on a different task. The small laboratory below keeps the corpus and entities fixed, then changes only the query burden. It uses five synthetic chunks, three graph edges, of which two are authoritative and one is extracted, plus an event ledger. The candidate retriever is a dependency-free BM25-like scorer, deliberately simple enough to inspect. The graph traversal admits only authoritative edges. The state function evaluates valid time and recorded time.
The first query asks what a named clause requires. The answer is local to one chunk. The second asks which facility is connected through a guarantor. The bridge fact sits in a different chunk, so top-two text retrieval misses it. The third asks which connected facility still requires action after a closure event. The graph still identifies the facility, while a state-only scan returns an unrelated active facility because it lacks party and clause scope. Only the compiled plan can apply the closure inside the correct relational and documentary boundary.
Worked example one: the smallest sufficient plan
The local question retrieves chunk D3 first and needs no graph. The relational question retrieves D1 and a namesake chunk, but misses D2, the bridge between guarantor and facility. A bounded traversal from canonical party P-10428 follows only guaranteed_by and guarantees, reaching F-19; the document lookup then finds its clause. For the current-action question, a state-only scan returns active facility F-88 because it has no party or clause scope. The graph-plus-document plan returns F-19 but cannot see that it closed. The compiled plan joins both boundaries, reads F-19 as closed on 1 August using what was recorded by that date, and therefore returns an empty actionable set.
Notice what the compiler does not do. It does not ask a model to infer that similarly named parties are identical. It does not let a model-extracted similar_name edge expand scope. It does not resolve the conflict by counting two old documents against one closure event. Evidence precedence follows the proposition: state owns status, approved paths own scope, and executed documents own clause wording.
Run the complete Python artefact
The script uses only the Python standard library. Save it as graphrag_relational_state_lab.py and run python graphrag_relational_state_lab.py. Expected output ends with All positive and negative checks passed.
from __future__ import annotations
from dataclasses import dataclass
from datetime import date
from math import log
from collections import Counter, defaultdict, deque
import re
TOKEN = re.compile(r"[a-z0-9-]+")
@dataclass(frozen=True)
class Chunk:
id: str
text: str
entities: tuple[str, ...]
@dataclass(frozen=True)
class Edge:
source: str
relation: str
target: str
authority: str # authoritative | asserted | extracted
valid_from: date
valid_to: date | None
source_chunk: str
@dataclass(frozen=True)
class StateEvent:
entity: str
field: str
value: str
valid_from: date
recorded_at: date
CHUNKS = [
Chunk(
"D1",
"North Quay Components, party P-10428, has Alder Parent plc as its approved guarantor.",
("P-10428", "P-700"),
),
Chunk(
"D2",
"Alder Parent plc guarantees facility F-19 for the borrower group.",
("P-700", "F-19"),
),
Chunk(
"D3",
"Facility F-19 contains a change-in-control clause requiring lender review before an ownership change closes.",
("F-19",),
),
Chunk(
"D4",
"A repayment notice dated 15 July states that facility F-19 was closed after settlement.",
("F-19",),
),
Chunk(
"D5",
"North Quay Components Services, party P-88201, has facility F-88 and is a different legal entity.",
("P-88201", "F-88"),
),
]
EDGES = [
Edge("P-10428", "guaranteed_by", "P-700", "authoritative", date(2025, 1, 1), None, "D1"),
Edge("P-700", "guarantees", "F-19", "authoritative", date(2025, 1, 1), None, "D2"),
# An extracted namesake edge is intentionally ineligible for authoritative scope.
Edge("P-10428", "similar_name", "P-88201", "extracted", date(2026, 1, 1), None, "D5"),
]
EVENTS = [
StateEvent("F-19", "status", "active", date(2025, 1, 1), date(2025, 1, 1)),
StateEvent("F-19", "status", "closed", date(2026, 7, 15), date(2026, 7, 16)),
StateEvent("F-88", "status", "active", date(2025, 1, 1), date(2025, 1, 1)),
]
STOP = {
"a", "an", "and", "as", "at", "before", "by", "does", "for", "has", "in", "is", "its",
"of", "on", "the", "through", "to", "what", "which", "with"
}
def tokens(text: str) -> list[str]:
return [t for t in TOKEN.findall(text.lower()) if t not in STOP]
def bm25_like(query: str, chunks: list[Chunk], top_k: int = 2) -> list[tuple[str, float]]:
"""Small dependency-free candidate retriever, not a production BM25 implementation."""
q = Counter(tokens(query))
docs = [Counter(tokens(c.text)) for c in chunks]
n = len(chunks)
df = Counter()
for d in docs:
for term in d:
df[term] += 1
ranked: list[tuple[str, float]] = []
for chunk, d in zip(chunks, docs):
score = 0.0
for term, qtf in q.items():
if term not in d:
continue
idf = log((n + 1) / (df[term] + 0.5)) + 1
score += qtf * idf * (1 + log(d[term]))
ranked.append((chunk.id, round(score, 4)))
ranked.sort(key=lambda x: (-x[1], x[0]))
return ranked[:top_k]
def bounded_paths(start: str, valid_at: date, max_depth: int = 3) -> list[tuple[str, ...]]:
"""Traverse only authoritative edges that are valid at the requested date."""
graph: dict[str, list[Edge]] = defaultdict(list)
for edge in EDGES:
is_valid = edge.valid_from <= valid_at and (edge.valid_to is None or valid_at < edge.valid_to)
if edge.authority == "authoritative" and is_valid:
graph[edge.source].append(edge)
queue = deque([(start, (start,), 0)])
paths: list[tuple[str, ...]] = []
while queue:
node, path, depth = queue.popleft()
if depth >= max_depth:
continue
for edge in graph[node]:
new_path = path + (edge.relation, edge.target)
paths.append(new_path)
queue.append((edge.target, new_path, depth + 1))
return paths
def state_as_of(entity: str, field: str, valid_at: date, known_at: date) -> str | None:
eligible = [
e for e in EVENTS
if e.entity == entity
and e.field == field
and e.valid_from <= valid_at
and e.recorded_at <= known_at
]
if not eligible:
return None
eligible.sort(key=lambda e: (e.valid_from, e.recorded_at))
return eligible[-1].value
def graph_facilities(party_id: str, valid_at: date) -> set[str]:
facilities: set[str] = set()
for path in bounded_paths(party_id, valid_at):
target = path[-1]
if target.startswith("F-"):
facilities.add(target)
return facilities
def documentary_clause_facilities(candidate_ids: set[str]) -> set[str]:
result = set()
for chunk in CHUNKS:
if "change-in-control clause" in chunk.text.lower():
for entity in chunk.entities:
if entity in candidate_ids:
result.add(entity)
return result
def state_only_active_facilities(as_of: date, known_at: date) -> set[str]:
"""A deliberately over-broad baseline with no party or document scope."""
facilities = {event.entity for event in EVENTS if event.entity.startswith("F-")}
return {
facility for facility in facilities
if state_as_of(facility, "status", as_of, known_at) == "active"
}
def compiled_current_actions(party_id: str, as_of: date, known_at: date) -> set[str]:
connected = graph_facilities(party_id, as_of)
clause_scope = documentary_clause_facilities(connected)
return {
facility for facility in clause_scope
if state_as_of(facility, "status", as_of, known_at) == "active"
}
def run() -> None:
q_local = "What does the change-in-control clause in facility F-19 require?"
q_relation = "Which facility connected to North Quay Components through its guarantor has a change-in-control clause?"
q_current = "Which connected facilities still require action on 1 August 2026?"
local_top = bm25_like(q_local, CHUNKS)
relation_top = bm25_like(q_relation, CHUNKS)
current_top = bm25_like(q_current, CHUNKS)
as_of = date(2026, 8, 1)
graph_scope = graph_facilities("P-10428", as_of)
graph_answer = graph_scope & documentary_clause_facilities(graph_scope)
state_only_answer = state_only_active_facilities(as_of, as_of)
compiled_answer = compiled_current_actions("P-10428", as_of, as_of)
print("Top-2 local chunks:", local_top)
print("Top-2 relational chunks:", relation_top)
print("Top-2 current-state chunks:", current_top)
print("Authoritative graph path facilities:", sorted(graph_scope))
print("Graph + document answer before state check:", sorted(graph_answer))
print("F-19 state on 2026-08-01, known then:", state_as_of("F-19", "status", date(2026, 8, 1), date(2026, 8, 1)))
print("State-only active facilities without party scope:", sorted(state_only_answer))
print("Compiled facilities requiring action:", sorted(compiled_answer))
# Positive cases
assert local_top[0][0] == "D3", "Local passage retrieval should find the clause directly."
assert graph_answer == {"F-19"}, "Typed traversal should connect the party to the relevant facility."
assert compiled_answer == set(), "The closed facility must not remain actionable."
assert state_only_answer == {"F-88"}, "State alone must expose its missing relationship scope."
# Negative cases
assert "D2" not in {doc_id for doc_id, _ in relation_top}, (
"The top-2 chunk baseline should miss the bridge document, exposing the multi-hop failure."
)
assert all("P-88201" not in path for path in bounded_paths("P-10428", as_of)), (
"An extracted namesake edge must not expand authoritative scope."
)
assert state_as_of("F-19", "status", date(2026, 7, 15), date(2026, 7, 15)) == "active", (
"A correction recorded tomorrow must not rewrite what the system knew today."
)
print("All positive and negative checks passed.")
if __name__ == "__main__":
run()
The laboratory is intentionally not a benchmark claim. Its job is causal isolation. By holding the corpus constant, it shows why method comparisons that average single-hop, multi-hop and time-sensitive questions can obscure the design boundary. Recent comparative work similarly reports that conventional RAG is strong on detailed single-hop questions, while local community GraphRAG performs better on several multi-hop datasets; global community summaries can lose detail, and knowledge-graph-only retrieval can suffer when graph construction omits answer entities.[13]
GraphRAG-Bench was proposed because final-answer datasets do not adequately expose graph construction, retrieval and reasoning as separate stages.[14] That decomposition is essential in enterprise evaluation. A wrong answer may originate in entity resolution, edge extraction, graph retrieval, document admission, state lag or generation. One aggregate “RAG accuracy” number cannot tell an owner what to repair.
The production-shaped join
The North Quay question becomes manageable when it is decomposed into proposition slots before any retrieval call. The system needs a canonical party, a current case, a set of approved relationship paths, current facility status, the governing document versions, relevant clauses and the applicable policy. “Needs action” is then derived under a named workflow rule. No individual store owns that final proposition.
The first operation resolves identity inside the authorised case scope. Human-readable names generate candidates but never become join keys. If two parties remain plausible, the system asks for confirmation or returns an ambiguity state. This is more than data hygiene. A cross-store join is an authorisation event because a false merge can move evidence across a customer boundary.
P-10428
Admitted case: OC-8841
Facility: F-19
Guarantor: P-700
P-88201
No admitted case
Facility: F-88
Separate legal entity
After identity, the compiler creates a dependency graph of reads. It fetches the case snapshot and scope first. It traverses only approved edge types, directions, depths and effective intervals. It then searches executed documents inside the resulting party and facility set. Reads can run in parallel only when they do not depend on a preceding scope decision. This order is not an optimisation detail; it prevents a broad document search from discovering resources the requester was never allowed to know existed.
Attribute-based access control gives a useful vocabulary: subject, object, action and environmental attributes are evaluated against policy.[15] A graph traversal adds relation and path attributes. Each candidate edge and document must be authorised before its content reaches a model. Filtering after a broad traversal is too late because candidate IDs, caches, logs and timing may already reveal protected information.
Why a giant prompt is not the join
One tempting implementation serialises rows, paths and passages into a single context and asks the language model to reconcile them. The prompt can label sections as “authoritative” or “derived”, but labels are advisory unless an external validator enforces them. A model may quote the most fluent source, merge two names, overlook a status timestamp or convert “no evidence found” into “does not exist”. Increasing context length gives it more material to arbitrate without giving it the institutional authority to arbitrate.
The join should happen in a context compiler. The compiler accepts a typed request, resolves the decision type, builds the read dependency graph and invokes adapters under a common scope token. Each adapter returns structured items rather than prose: a StateFact, RelationshipPath, DocumentSpan, DerivedAssertion, Contradiction or Unknown. The model receives a rendered view, but the original types remain available to validation and audit.
This design separates three questions that prompts often collapse. Candidate generation asks what might matter. Admission asks what is permitted and applicable. Claim support asks whether an admitted item can evidence a particular proposition. A passage can rank highly yet fail admission because it is superseded. An admitted community summary can still be ineligible for a current-status claim. A state row can be authoritative for status yet irrelevant to clause interpretation.
The compiler also owns expiry. A packet can become invalid because case status changes, a relationship is corrected, a document is superseded, policy applicability changes or the requester’s entitlement is revoked. A simple time-to-live cannot express all of these. Dependency receipts allow event-driven invalidation and targeted rebuilding. The context is a decision-specific evidence object, not a bag of retrieved text.
Party, case and facility fields with service, record version, consistency token, valid time and recorded time.
Canonical node IDs, typed edges, authority class, source receipt, effective interval and purpose permission.
Document and version ID, page or offset, applicability status, extraction release and openable source location.
supported · contradicted · unknown · ineligible
Worked example two: ownership change at Harbour Bank
Consider a fictional institution, Harbour Bank. An analyst opens case OC-8841, which already references party P-10428. The party service returns the registered entity and one controlled alias. The case service returns status evidence_pending, a proposed closing date and a snapshot token. A relationship service traverses direct owner, beneficial owner, guarantor, obligor and facility edges as of the proposed closing date. A document-derived shares_address_with edge exists, but the workflow does not permit it to expand credit scope.
Within the admitted facilities, hybrid retrieval finds an executed amendment using the phrase “change in control”, although the analyst asked about “ownership change”. Exact search finds a covenant number in the original facility letter. A GraphRAG community report surfaces a board discussion about a future group guarantee. That report is useful for discovery, but the source-status service shows that no executed guarantee exists. The packet labels it as an unfulfilled proposal rather than letting fluent summary language promote it into state.
The book of record says facility F-19 closed after settlement. The executed amendment still contains the clause, but it no longer creates an open facility action. Another facility remains active and lacks lender consent evidence. The assistant may draft: “No admissible consent evidence was found in the searched current sources for F-27 as of the packet time.” It may not conclude that consent does not exist, because search coverage is not ontological completeness.
The analyst opens the cited page, checks the path receipt and sends a recommendation through the approved workflow. An authorised officer makes the decision. The officer’s action produces a case event, which updates state and invalidates dependent packets. The model contributes language and bounded synthesis; it does not sit inside the trust root for identity, scope, authority or state transition.
Change propagation is part of correctness
Every projection introduces a disagreement window. A document is superseded, a relationship is corrected, a facility closes or access is revoked. The source changes first; vector chunks, graph edges, summaries and caches follow. The architecture must name the invalidating event, ordering key, consumer, idempotency rule, propagation bound and reconciliation check. “Near real time” is not a control unless the acceptable lag and failure behaviour are measurable.
Graph releases need lineage because rebuilding the same corpus with a new extractor can merge entities differently, split communities and alter summaries. Vector migrations need parallel evaluation because a new chunker or embedding model changes candidate recall. The graph and index should be immutable releases with dependency IDs, so a source correction can find every derived artefact and a historical packet can be replayed. W3C PROV supplies general concepts for entities, activities and agents, but a decision system must add its own authority, temporal and applicability rules.[16]
Security changes with structure. Research on GraphRAG poisoning reports that shared relations can create attack surfaces where a small injected narrative influences many queries through the graph.[17] The control is not merely prompt hardening. Derived edges need source attribution, release isolation, relation allowlists, cardinality stops, anomaly monitoring and a rule that extracted structure cannot author protected state.
A representation contract before a retrieval product
A representation contract assigns semantics at the proposition level. It prevents an application-level label such as “GraphRAG solution” from hiding several incompatible data duties. For every consequential proposition, the architect records the answer-changing variables, the authorised owner, the time basis, the required evidence form and the failure behaviour.
| Test | Question | RAG signal | Graph signal | State signal | Stop condition |
|---|---|---|---|---|---|
| Locality | Can one admitted passage support the answer? | Strong | Optional expansion | Metadata only | Answer requires missing bridge facts |
| Identity | Can two plausible entities match the wording? | Candidate only | Canonical nodes required | Exact key required | Name similarity controls a join |
| Relation | Does removing one typed edge change the answer? | Weak unless co-located | Strong when path is governed | Recursive read may suffice | Extracted edge authors scope |
| Time | Is the question current, effective-as-of or known-as-of? | Version filters | Temporal edges needed | Valid and recorded time | One timestamp stands for several meanings |
| Authority | Which source may attest the proposition? | Document status required | Edge class required | Owning service required | Derived index is called source of truth |
| Evidence | What must a reviewer open or replay? | Versioned span | Path plus source receipts | Row or event receipt | Only generated prose survives |
| Change | What event invalidates the result? | Reindex or status event | Edge or graph-release invalidation | Transaction or domain event | No bounded invalidation path exists |
The decision rule
Use RAG when source language is the answer-bearing structure. Add GraphRAG when typed connections or corpus-wide organisation materially improve candidate evidence. Use relational or event state when exact identity, invariants, current status or historical reconstruction can change the decision. Compose them only through typed proposition slots, shared scope and explicit precedence.
This instrument usually leads to fewer technologies than a feature-led design. A bounded three-hop query over modest authoritative tables may not justify a separate graph engine. A document graph may improve discovery but remain separated from an entitlement graph. A property graph may be the implementation of an authoritative relationship service, provided corrections, effective intervals, consistency and approvals are explicit. Select the smallest set of representations that removes a measured failure without creating semantic impersonation.
stateDiagram-v2 [*] --> Claim state "Thought experiment one: the<br/>namesake switch" as Claim state "A counterfactual test for<br/>representation sufficiency" as Test state "RAG is a candidate generator" as Revision Claim --> Test: expose mechanism Test --> Decision: evidence survives Test --> Revision: rival survives Revision --> Claim: narrow or rebuild Decision --> [*]
Evaluation must preserve the seams
End-to-end answer accuracy matters, but it cannot diagnose or assure this architecture. A final answer may be accidentally correct after an unauthorised read. A proper abstention may score worse than a fluent guess. The evaluation stack therefore measures each representation, each join and the final decision packet.
Start with a claim graph for the system itself. One claim says that ambiguous parties are never selected automatically above a named consequence tier. Another says that only approved edge classes can expand scope. Another says that every current-status claim is supported by a source read no older than the workflow’s freshness bound. Each claim names evidence, a test owner, a release dependency and a defeater. This turns evaluation from a leaderboard into an assurance argument that can be invalidated by change.
Measures should be conditional rather than averaged away. Retrieval recall is reported by local, multi-hop, temporal, tabular and exception slices. Graph path precision is reported by edge authority and depth. State correctness is reported around concurrency, back-dated corrections and event-order boundaries. Packet quality is reported by proposition type and consequence. An excellent average can otherwise hide a zero-recall slice for the rare clause that actually drives escalation.
For search, measure candidate recall before and after filters, source-location correctness and performance by query shape. For graphs, measure entity resolution, edge precision by authority class, temporal path validity, source attribution, scope amplification and the incremental evidence found because of expansion. For state, test lifecycle invariants, snapshot compatibility, event ordering, historical replay and correction convergence. For the packet, require every proposition to be supported, contradicted or explicitly unknown.
Failure injection provides the decisive cases. Create namesake entities, a revoked edge, a late correction, a superseded agreement, a protected node one hop away, a clause hidden in a table and a community summary that overstates a tentative source. Then inspect intermediate sets, not only prose. The system passes only when denied content never reaches candidate caches, prompts, traces or user-visible explanations.
Recent research reinforces the need for task slices rather than universal GraphRAG claims. One analysis reports gains for graph retrieval on some multi-hop tasks alongside lower performance on single-hop and time-sensitive questions, and highlights added latency.[18] Those reported values belong to its datasets and implementations, not to this article’s synthetic experiment. The transferable point is the interaction: graph benefit grows when relational structure is genuinely required, while freshness and detail can favour simpler retrieval or direct state reads.
Release evidence should make narrow, falsifiable claims. For the ownership-change workflow, a defensible claim is: canonical party and facility state come from named services; only adjudicated owner, guarantor, obligor and facility edges may expand scope; documentary search is limited to admitted identifiers; derived edges may propose leads but cannot satisfy an authoritative slot. A change to an edge type, source collection, graph release or temporal rule reruns the relevant tests.
The system boundary remains honest when uncertainty has a type. Missing evidence is not a negative fact. A conflicting relationship is not an invitation to vote among stores. A stale projection is not current because it ranks highly. The allowed outcomes include supported, contradicted, unknown, ineligible and incomplete, each with a route to a source owner or human reviewer.
Compact glossary
- RAG
- Generation conditioned on evidence retrieved from an external corpus, commonly through lexical, dense or hybrid passage search.
- GraphRAG
- A family of methods that use graph structure during indexing or query to retrieve connected evidence or support corpus-level synthesis.
- Relational state
- Identified facts whose meaning depends on keys, constraints, controlled updates, lifecycle rules and declared consistency.
- Valid time
- When a fact applied in the modelled world.
- Recorded time
- When a system stored or learned the fact, sometimes called transaction time.
- Representation contract
- A proposition-level declaration of identity, relation, time, authority, evidence and invalidation semantics.
Build the join, not the monolith
RAG is enough when the answer is local to admitted text. GraphRAG earns its cost when dispersed relations or corpus structure change what evidence can be found. Relational and event state become mandatory when identity, current status, correction history or invariants can reverse the decision.
The architecture therefore changes at the proposition boundary. Resolve identity before expansion. Let approved paths define scope. Retrieve source language inside that scope. Reconstruct mutable facts at the required time. Preserve each evidence role through generation and validate claims against it afterwards.
No representation should be allowed to impersonate another. A similar passage is not a fact table. An extracted graph is not a ledger. A relational row is not a contract quotation. The defensible system is the controlled join that knows exactly why each one is present, what it may assert and what event makes it stop being true.
sequenceDiagram participant Q as Question participant M as Mechanism participant E as Evidence participant D as Decision Q->>M: One question, three kinds of truth M->>E: Thought experiment one, the namesake switch E-->>M: compare rival explanations E->>D: References and source boundaries D-->>Q: revise scope or act
References and source boundaries
- Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Supports the original parametric and non-parametric memory formulation.
- Karpukhin et al., Dense Passage Retrieval for Open-Domain Question Answering. Supports dense passage retrieval as candidate selection.
- Thakur et al., BEIR. Supports heterogeneous retrieval evaluation and the absence of a universal retriever.
- Edge et al., From Local to Global: A Graph RAG Approach to Query-Focused Summarization. Supports graph-derived entities, communities and summaries for global corpus questions.
- Microsoft GraphRAG documentation. Supports current implementation concepts and query modes.
- Microsoft GraphRAG repository. Supports implementation inspection, cost caution and project status, not universal production fitness.
- Yang et al., HotpotQA. Supports evaluation of questions requiring multiple supporting documents.
- Trivedi et al., MuSiQue. Supports connected multi-hop question construction and shortcut-resistant evaluation.
- W3C RDF 1.1 Concepts. Supports the graph data model, not the evidential grade of an edge.
- W3C SPARQL 1.1 Query Language. Supports graph-pattern query semantics, not local decision authority.
- PostgreSQL MVCC and transaction isolation. Support concrete concurrency and snapshot semantics.
- Corbett et al., Spanner. Supports a published design for externally consistent distributed transactions, not a product recommendation.
- Han et al., RAG vs. GraphRAG: A Systematic Evaluation and Key Insights. Supports task-dependent complementarity and reported graph-construction limitations.
- Xiao et al., GraphRAG-Bench. Supports pipeline-level evaluation across construction, retrieval and generation.
- NIST SP 800-162. Supports attribute-based access-control concepts.
- W3C PROV-DM. Supports general provenance concepts for entities, activities and agents.
- Liang et al., GraphRAG under Fire. Supports the described relation-centred poisoning attack surface.
- Xiang et al., When to Use Graphs in RAG. Supports the reported interaction between query structure, performance and latency; its measurements are not treated as universal.