Home · Work · System design
Data and AI platformPlatform lifecycle and release controlfederated

Synthetic journey test factory

Production histories underrepresent rare failures and cannot be freely reused for repeated evaluation because of privacy and leakage risk.

Approach

The factory composes synthetic customer profiles, events and documents from constraint-based generators, then injects controlled contradictions and partial failures.

Organisations, systems and operating conditions are intentionally anonymised and recomposed. The design demonstrates engineering and banking-domain reasoning; it does not represent a named client estate, vendor product or measured production result.

Overview

System overview

A shared authority core coordinates independently owned capability cells, domain systems and operating evidence.

Synthetic journey test factory: federated system architecture A federated architecture separates channels, a shared authority core, independently deployed capability cells, domain systems of record, and an evidence and operations rail. Calls use scoped identity and typed contracts; every decision path emits a receipt. aSystem boundary and decision authoritySchematic architecture · illustrative scenario Channels and operatorsFederated capability cellsDomain systemsEvidence, recovery and operations Customer channelweb · mobile · voice Staff workspacereview · approve Service eventbatch · stream · API Operations consoleincident · replay Scope & authority coreidentity · purpose · policy Domain adapter meshversion · health · readback Journey StateCoordinatorWorkflow & custodyBehaviour EvaluationHarnessEvaluation & releaseOutcome RecoverySupervisorResilience & recoveryOperational Read ModelWorld state & read modelsDecision Receipt LedgerEvidence & auditAuthority Rules EngineAuthority & policyEvidence ContextCompileravailable, not admittedParty RelationshipResolveravailable, not admittedTyped Action Brokeravailable, not admittedAgentauthoritative recordtool registryauthoritative recordmodel gatewayauthoritative recordknowledge servingauthoritative record Decision receipt archiveappend-onlyTrace & cost pipelineredacted telemetryRecovery & custody queueowned exceptionsRelease & rollback registrysigned manifest
Figure 1. Independent capability cells share an authority membrane without sharing domain ownership. Source-specific behaviour remains behind adapters and every decision emits operating evidence. This is an illustrative system model, not a named deployment.

Permitted workThe platform supplies reusable controls and contracts. Domain teams retain business logic, data purpose and outcome ownership.

Consistency ruleTie deployed code, prompt, model, data, permissions, policy and evidence to one release manifest and compatibility graph.

Hard boundaryThe model is not a system of record, identity provider, policy authority or proof that an external effect occurred.

Deployment

Deployment

Agents, prompts, tools, policies and models must be built, certified, operated, changed and retired as one reachable system.

Synthetic journey test factory: deployment and trust-zone topology A production topology separates public edge, identity boundary, workload cluster, integration plane, data enclave and operating controls. Connections name transport and identity protocols, and a second region provides controlled failover. bDeployment topology and failure domainsLogical deployment · no measured estate data EDGE AND ACCESS PRIVATE WORKLOAD CLUSTER INTEGRATION AND DATA ENCLAVE OPERATIONS, EVIDENCE AND RECOVERY PLANE Web applicationfirewallTLS · bot · rate API ingressschema · quota Identity brokerOIDC · MFA Purpose gateclaims · consent HTTPStokensubject Regional traffic managerhealth · canary · failover Workload identity issuershort-lived service tokens Event & work queuepartition · DLQ · backpressure Secrets & key serviceenvelope encryption · rotation Authority RulesEnginezonal workloadJourney StateCoordinatorzonal workloadDecision ReceiptLedgerzonal workloadBehaviourEvaluation…zonal workloadOutcome RecoverySupervisorzonal workloadOperational ReadModelzonal workloadReserved workloadslotdisabledReserved workloadslotdisabled HTTPS · scoped JWT workload token async eventkey lookup IntegrationgatewaymTLSAgentsystem of recordtool registrysystem of recordmodel gatewaysystem of recordknowledge servingsystem of record Model route enclavepolicy · schema · egress Qualified data storesSQL · graph · object Trace, metric & cost pipelineOpenTelemetry · redaction Decision receipt archiveappend-only · retention Reconciliation workersreadback · replay · DLQ On-call & review queuescustody · SLA · escalation Secondary-region standbycompatible manifest telemetryreceiptsexceptions
Figure 2. Workloads remain isolated from systems of record. Short-lived identity, typed integration contracts, append-only evidence and controlled failover define the deployable boundary; the topology is schematic.
Authority checkpoint

Synthetic provenance is machine-readable; generated data cannot be mistaken for a customer record or used as outcome evidence.

Runtime

Runtime flow

A deterministic outer workflow contains model-led work inside typed, observable calls. Dashed messages remain proposals until policy or a human grants authority.

Synthetic journey test factory: build, release and retirement lifecycle A controlled lifecycle links registration, design, implementation, evaluation, approval, canary release, operation and retirement. A signed manifest binds dependencies and evidence; failed gates and operating drift return to an owned change route. cRuntime protocol and control pointsProtocol model · illustrative execution Build and releaseOperate, change and retireManifest, evidence and recovery Signed system manifestowner · code · model · knowledge · tools ·… Registerowner · purpose · data boundaryDesigntopology · authority · contractsBuildcode · prompts · rules ·…Evaluatefunction · failure · harm · NFRApprovesigned manifest · evidenceCanarylimited traffic · rollback…Operatebehaviour · cost · incidentsRetirerevoke · archive · prove closure Evaluation evidenceroute · failure · compatibilityRelease registrysignature · canary · rollbackOperating evidencetrace · cost · outcomeIncident & change queueowner · severity · deadlineArchivewithdrawal proof Gate rule: Automate evidence collection before automating approval. Expand self-service only when the paved route proves safer and faster than bespoke delivery.
Figure 3. Design, evaluation, release, operation and retirement remain bound to one signed manifest. A failed gate or operating incident returns to an owned change route. The paths are protocol semantics, not an observed production trace.
S

State model

A signed system manifest links code, prompt, model, knowledge, tools, permissions, policies, tests and owner. Any material dependency change creates a new release candidate.

I

Integration choice

Build and runtime boundaries use the same schemas. Registry metadata drives discovery, policy, telemetry attribution and deprecation.

C

Consistency semantics

The deployed artefact, registry entry and evidence pack share one immutable release identifier; partial promotion is not a valid state.

Services

Services and interfaces

These roles are deliberately vendor-neutral. Each can be independently owned, versioned and replaced.

  1. COS
    Workflow and custody

    Journey State Coordinator

    Runs a versioned state machine with explicit waits, deadlines, retries, compensations, human tasks and terminal states.

    Degraded routePause at a known checkpoint, preserve custody and resume from the last verified state rather than restarting the request.
  2. SEL
    Evaluation and release

    Behaviour Evaluation Harness

    Runs component, route, trajectory, failure, harm and outcome tests against the versioned system manifest.

    Degraded routeReduce or withdraw authority when a dependency changes or a critical suite fails; preserve the last approved route where compatible.
  3. RRS
    Resilience and recovery

    Outcome Recovery Supervisor

    Classifies unknown outcomes, replays idempotent work, runs compensations and restores from the last verified checkpoint.

    Degraded routeStop new effects, continue safe reads where possible, and expose the recovery state and owner to customers and operators.
  4. OSR
    World state and read models

    Operational Read Model

    Builds time-qualified projections from source events and reconciliations without becoming the legal system of record.

    Degraded routeExpose source lag and fall back to direct reads for defined critical facts; never present missing feeds as zero.
  5. DES
    Evidence and audit

    Decision Receipt Ledger

    Appends request, versions, policy result, model proposal, approval, action receipt, readback, correction and custody events under one correlation key.

    Degraded routeBlock a consequential close when mandatory evidence is missing; keep the case open with an explicit evidence defect.
  6. PDS
    Authority and policy

    Authority Rules Engine

    Evaluates identity, purpose, capability, amount, risk tier and policy version; returns allow, deny, step-up or human-review with reasons.

    Degraded routeFail closed for effects and restricted data. Read-only explanation may continue only from approved public or customer-visible sources.
Figure 4. Each service exposes a typed contract and a defined degraded route. Coordination stays in the control plane while domain ownership remains outside it.
Data

Data and records

Durable records carry provenance, authority, effect and custody without turning a transcript into an uncontrolled memory store.

Synthetic journey test factory: state ownership and evidence lineage Authoritative source observations feed six versioned runtime records. Each record names a single write owner. An append-only event and evidence stream builds a disposable current-state projection and supports audit replay. dState lineage, ownership and temporal validityLogical records · fields are illustrative AUTHORITATIVE INPUTS · POINT-IN-TIME QUALIFIEDAgentsource version · observed attool registrysource version · observed atmodel gatewaysource version · observed atknowledge servingsource version · observed atqualified inputqualified inputqualified inputqualified input WRITE OWNER · Evidence Context CompilerRequest enveloperequest_idpurposeactor_refsubject_refversioned · attributable · retainedWRITE OWNER · Authority Rules EngineContext manifestcontext_idsource_refsource_versionobserved_atversioned · attributable · retainedWRITE OWNER · Typed Action BrokerCase statecase_idstatestate_versioncurrent_custodianversioned · attributable · retainedWRITE OWNER · Reasoning Route ManagerAuthority decisiondecision_idpolicy_versionrequested_capabilityoutcomeversioned · attributable · retainedWRITE OWNER · Behaviour Evaluation HarnessAction and effect receiptaction_ididempotency_keyinput_hashdispatch_statusversioned · attributable · retainedWRITE OWNER · Operational Read ModelCustody eventtransfer_idfrom_ownerto_ownerreasonversioned · attributable · retained APPEND-ONLY CASE EVENTS + DECISION RECEIPTSrequest hash · source versions · policy version · proposal · authority · effect readback · custody Operational current-state viewrebuildable projection
Figure 5. The record chain preserves source version, write ownership, authority and custody. Solid records are required in this scenario; dashed records remain compatible but dormant.
Consistency contractDomain source rule

Tie deployed code, prompt, model, data, permissions, policy and evidence to one release manifest and compatibility graph.

Choices

Trade-offs

The selected design is not universally superior. It is the safer fit for this boundary and failure cost.

  1. 01

    Open-ended agent planning or a deterministic workflow shell

    Selected path

    Keep consequential state transitions deterministic and use models inside bounded steps.

    Rejected path

    Let the model choose the complete path and recovery sequence.

    Cost acceptedThe shell reduces flexibility, but makes deadlines, retries, permissions and recovery testable.

  2. 02

    Evaluate model responses alone or certify the complete reachable system

    Selected path

    Test prompts, models, tools, knowledge, policies, state transitions and human paths together.

    Rejected path

    Use a static answer-quality benchmark as the release gate.

    Cost acceptedSystem evaluation takes longer and needs synthetic environments, but detects authority and recovery failures that answer scoring misses.

  3. 03

    Distributed transaction or explicit saga with reconciliation

    Selected path

    Use local commits, idempotent steps, compensations and an unknown-outcome state.

    Rejected path

    Attempt one atomic transaction across independent banking systems.

    Cost acceptedSagas expose temporary inconsistency and require recovery logic, but fit systems that cannot share one transaction boundary.

  4. 04

    Query every source in the critical path or maintain operational read models

    Selected path

    Use event-fed projections for scale and direct readback for consequential effects.

    Rejected path

    Fan out to all systems of record for every interaction.

    Cost acceptedRead models introduce lag and reconciliation work, but reduce source load and make cross-system views feasible.

  5. 05

    Mutable current-state record or append-only decision history

    Selected path

    Use append-only events plus a rebuildable current-state projection.

    Rejected path

    Overwrite the case row with its latest status.

    Cost acceptedReplay and storage are more complex, but point-in-time reconstruction and correction lineage remain possible.

  6. 06

    Reason over policy text at runtime or compile executable rules

    Selected path

    Compile stable decision logic and retain retrieval for explanation and residual ambiguity.

    Rejected path

    Ask a model to interpret the source document for every request.

    Cost acceptedRule compilation needs controlled change, but creates repeatable decisions, regression tests and clear exceptions.

Figure 6. Teal marks the selected route; coral dotted rules preserve the rejected alternative. Each decision states the cost accepted rather than presenting one architecture as universally superior.
Recovery

Failures and recovery

Retries are bounded by knowledge of business effect; unknown outcome remains visible, owned and independently reconciled.

Synthetic journey test factory: failure classification and recovery control flow Five failure sources converge on a classifier that distinguishes known no effect, known failure, unknown outcome, material evidence gaps and control-plane failure. Each state has an explicit recovery route, service degradation and evidence receipt. eFailure classification and recovery policyControl model · thresholds set by owners DETECTION POINTSEFFECT CLASSIFIERRECOVERY DECISIONSERVICE-SPECIFIC DEGRADATION Source dependencyOver-clean synthetic cases can reward…Authority servicePolicy is unavailable, expired, or returns…Reasoning routeOutput fails schema, citation, grounding…Action pathTransport times out after dispatch & the…Evidence railReceipt or custody append fails after a… Outcome classifierattempt · idempotency · source readback ·… Known no effectSafe bounded retry with the same…Known failureNarrow, queue, compensate, or fail closed…Unknown outcomeRead back authoritative state before retry…Material evidence gapRetain custody & create an owned…Control-plane failureStop promotion or dispatch; restore the…Journey State CoordinatorPause at a known checkpoint,preserve custody & resume from thelast verified state rather than…Behaviour Evaluation HarnessReduce or withdraw authority whena dependency changes or a criticalsuite fails; preserve the last…Outcome Recovery SupervisorStop new effects, continue safereads where possible, & expose therecovery state & owner to…Operational Read ModelExpose source lag & fall back todirect reads for defined criticalfacts; never present missing feeds…Decision Receipt LedgerBlock a consequential close whenmandatory evidence is missing;keep the case open with an… RECOVERY RECEIPTfault origin · classification basis · owner · retry/compensation key · readback reference · residual risk · resolved at classifier evidencerecovery evidence Pattern rule: Canary, kill switch& rollback restore the lastcompatible manifest…
Figure 7. Recovery follows the known business-effect state, not the transport symptom. Every branch ends in a receipt preserving fault origin, custody and reconciliation basis.
Capacity

Performance and capacity

Actual thresholds belong to accountable service owners. The design exposes the equations and observables that those owners must baseline.

Dependent variablesOperating
envelope
  • deployment lead time
  • critical test coverage
  • rollback time
  • cost and trace attribution
Capacity relationships
  1. 01gate_runtime = test_cases x routes x dependency_versions
  2. 02release_capacity = available_environments / average_gate_duration
  3. 03operating_cost = model + tools + platform + human_review + incidents
Privacy lens

Enforce purpose, tenant and workload identity at every hop; telemetry stores metadata rather than unrestricted prompts or payloads.

Figure 8. The variables define what must be measured before capacity or quality claims can be accepted. Relationships are analytical scaffolds, not invented targets or production results.
Operations

Release and operations

A design is production-ready only when teams can prove what happened, recover it and change it safely.

Minimum release gates
  1. 01

    reachable-authority analysis

  2. 02

    cross-version compatibility

  3. 03

    failure and rollback rehearsal

  4. 04

    cost, latency and trace attribution

Figure 9. Promotion requires evidence at each gate and a compatible rollback route. No gate is implied to have passed; accountable owners set thresholds and sign the record.
Operating stateReleaseRecover
Evidence

Scenario seed, constraint set, coverage map, failure injects, expected invariants, run results and gap analysis.

Human authority

Domain, risk and test owners approve coverage and acceptance criteria.

Evolution

Automate evidence collection before automating approval. Expand self-service only when the paved route proves safer and faster than bespoke delivery.

Ownership

Platform, domain product, data, security, model-risk and service owners share certification decisions.

Figure 10. Evidence, human authority, ownership and controlled evolution remain coupled throughout operation. Removing any quadrant breaks the assurance case.