The arrow that changed its numbers
Two search teams encode the same phrase, “flight delay policy”. Team A obtains [0.81, -0.12, 0.55]. Team B obtains [0.04, 0.96, -0.28]. The lists scarcely resemble one another. Yet both systems return exactly the same documents in exactly the same order, and every pairwise cosine similarity agrees to floating-point precision.
The tempting diagnosis is that one model has learned the wrong representation. A second temptation is more subtle: perhaps coordinate one means “aviation”, coordinate two means “delay”, and the teams have discovered different concepts. Both conclusions can fail. Team B may simply have applied the same orthogonal rotation to every query and document vector. Its coordinate axes point elsewhere, but angles and lengths inside the space have not changed.
Imagine drawing an arrow on transparent film, then rotating the graph paper beneath it. The arrow keeps its length and direction in the room. Its horizontal and vertical readings change because the measuring frame changed. The coordinates belong jointly to the vector and the chosen basis. They are not an intrinsic name tag attached to the vector.
This distinction is the causal centre of the article. It explains why projection is useful, why cosine sometimes works, why individual embedding dimensions are hard to interpret, and why an apparently harmless model migration can corrupt retrieval. It also yields a concrete engineering rule: specify and test the geometry that must survive before comparing coordinate arrays.
Part ICoordinates are reports, not the thing
A shopping list is determined by its entries. Swap “tea” for “rice” and it becomes a different list. A vector does not behave that way. The array [3, 2] means nothing geometrically until a basis says which direction each coordinate controls. In the ordinary east-north basis, it means three units east and two units north. In a basis whose first axis points north-east, another pair describes the same displacement.
A basis is a minimal set of directions whose combinations can produce every vector in the space. That familiar definition matters because it separates three objects that software often collapses: the vector, the basis vectors, and the coordinates relative to them. MIT’s linear algebra materials state the spanning role directly.1
A minimal worked example: the wind report
Suppose a weather station records a wind displacement of three metres east and two metres north during one interval. In the ordinary orthonormal basis, the coordinate report is [3, 2]. Rotate the measuring axes anticlockwise by 45 degrees. The first new axis points north-east; the second points north-west. Projecting the same wind onto these axes gives approximately [3.54, -0.71].
Nothing blew backwards. The negative second coordinate says only that part of the arrow points opposite the new second basis direction. The length remains √13 metres, about 3.61 metres. Coordinate signs and magnitudes are statements about a measuring frame, not isolated properties of the represented event.
Direction is useful shorthand, with a boundary
In ordinary Euclidean space, describing a vector by magnitude and direction is excellent intuition. It prevents the coordinate list from becoming the object. The definition is broader, however. Functions can form a vector space: two functions can be added and multiplied by scalars even though neither is a physical arrow. Polynomials, audio signals and parameter updates can be vectors for the same structural reason. They support the required addition and scaling operations.
The distinction between a point and a displacement also matters. A point says where something is relative to an origin. A vector can describe the displacement from one point to another without selecting a privileged origin. Coordinate software often stores both as arrays, so type shape alone does not preserve the difference. Adding two displacements is meaningful. Adding two geographic locations usually is not, although subtracting them can produce a displacement after choosing an appropriate geometry.
“Direction, not list” should therefore be read as a corrective, not a replacement axiom. It says to look for the invariant object and its lawful operations before interpreting entries. In a learned model, that object might be a direction, a subspace, a linear functional, a trajectory or an equivalence class under allowed transformations. The array representation is faithful only relative to the operations the system promises to preserve.
Thought experiment 1: rename, rotate, or deform the map
Place a transparent arrow over graph paper. First, rename the horizontal axis “one” and the vertical axis “zero”. The coordinate labels change, but no geometry changes. Next, rotate the paper. Coordinates change again; lengths and angles remain. Finally, stretch the horizontal grid spacing while continuing to treat one square in either direction as one Euclidean unit. Now measured angles and lengths change.
The three interventions reveal three distinct operations. Renaming changes notation. Orthogonal rotation changes coordinates while preserving Euclidean geometry. Stretching or shearing changes the geometry unless the metric is transformed with the basis. Treating all three as “just another vector transformation” hides the precise property at stake.
Interactive basis rotation
Move the basis while the world vector v = (3, 2) remains fixed. The coordinate readout changes continuously; the vector’s length remains √13.
Change of basis is a translation between reports
Let the new basis matrix be C. The same vector satisfies v = Bx = Cy. Therefore y = C−1Bx. If the new basis is described inside the old one by C = BS, then y = S−1x. The inverse appears because moving the basis one way makes the coordinates compensate in the other direction.
This becomes operational when data crosses systems. A coordinate array without its basis and feature contract resembles a temperature without a scale. It can be stored and transmitted, but comparison may be meaningless. Versioning only the array length is insufficient. Two 1,024-dimensional outputs can inhabit incompatible spaces.
The serious counterexample: some axes are meant to be named
Consider [temperature, pressure, flow] from an industrial sensor. Each coordinate has a specified physical quantity, unit, calibration procedure and acceptable range. Rotating these three axes would create valid linear combinations, but it would destroy the measurement interface. Here the basis is not arbitrary. It is anchored by the data contract.
This counterexample narrows the thesis. “Vectors are directions” does not imply that coordinates never have meaning. It says that coordinate meaning must come from an independently established basis contract. Learned embedding axes do not inherit semantic names merely because engineers can inspect their values. Engineered feature axes may have such names because the measurement design fixed them before the vector was formed.
Part IIProjection and similarity need a metric
Projection answers a concrete question: how much of this vector lies along that direction? Stand in sunlight holding a pencil. Its shadow on a wall depends on the wall’s orientation. The pencil is the vector; the wall is the chosen subspace; the shadow is the projection. Without naming the wall, “the shadow” is incomplete.
For a unit direction u, the scalar component of v along u is the dot product uTv. Multiplying that scalar by u returns the projected vector. The residual is perpendicular to u.
Projection onto a whole subspace works the same way. If the columns of Q form an orthonormal basis for that subspace, then QQTv is the nearest vector in it. For a non-orthonormal basis A with independent columns, the projection matrix is A(ATA)−1AT. This is why least-squares fitting can be understood as projection onto the column space of a design matrix.2
A production-shaped projection: compressing a case for triage
Consider a synthetic case vector with three standardised coordinates: evidence strength, urgency and a pattern indicating possible customer harm. Let the case be v = [0.70, 0.40, 0.50]. A current triage policy defines its automated priority direction as the unit vector u = [0.80, 0.60, 0]. The scalar projection is 0.80. The projected vector is [0.64, 0.48, 0], leaving residual [0.06, -0.08, 0.50].
The calculation does not prove that the case has “risk 0.80”. It says how strongly the case aligns with one declared policy direction under one declared inner product. The residual is operationally important: the entire customer-harm coordinate lies outside the automated priority direction. A system that retains only the scalar score has compressed away that signal.
This exposes both the value and danger of projection. It can reduce a complex state to the component relevant for a bounded decision. It can also make ignored dimensions invisible. A defensible design records the projection basis, preserves the residual or its decision-relevant summary, and tests whether protected signals are systematically pushed outside the selected subspace. Projection is controlled loss, not neutral simplification.
Similarity is a compressed decision rule
The dot product combines two effects: how long the vectors are and how closely their directions align. In an orthonormal Euclidean basis, xTy = ||x|| ||y|| cos θ. A larger score can therefore come from greater alignment, greater magnitude, or both. Calling the result “semantic similarity” does not remove this ambiguity.
Cosine similarity divides out both lengths and retains only the angle. Official scikit-learn documentation describes it as the L2-normalised dot product, equivalently the dot product after projection onto the unit sphere.13
Thought experiment 2: the confident whisper and the faint shout
Take two vectors on the same ray: [1, 1] and [100, 100]. Their cosine similarity is one. Now suppose vector length encodes independently calibrated confidence, transaction size or signal energy. Collapsing them to the unit circle discards the very quantity the decision needs. If length is arbitrary scale introduced by the encoder, normalisation is helpful. If length carries valid evidence, normalisation is destructive.
Vary only one causal feature. Keep direction fixed and change magnitude. Cosine remains constant. Dot product grows. Euclidean distance grows as well. None is universally correct because each answers a different question. A similarity function is an operational hypothesis about which differences matter. It earns use through task evidence, not through familiarity.
The hidden assumption inside ordinary coordinates
In an orthonormal basis, the coordinate dot product xTy equals the geometric inner product. In a skewed or differently scaled basis, that shortcut fails. The basis itself contributes a metric matrix G = BTB. The geometric inner product is then xTGy.
An orthogonal matrix Q is the convenient special case: QTQ = I. If coordinates transform as x′ = xQ for row vectors, then x′y′T equals xyT. Norms, Euclidean distances, angles and cosine similarities survive. A shear is invertible too, but it does not preserve those quantities under the ordinary dot product.
Invertible does not mean geometry-preserving. An invertible transformation retains enough information to reconstruct the old coordinates. It may still alter the neighbourhoods used by a nearest-neighbour system. Recoverability and metric invariance are separate guarantees.
Part IIIEmbeddings inherit the geometry contract
Only now do we need the word embedding. An embedding function maps an object such as a word, sentence, image or account into coordinates in a vector space. The coordinates are useful when relations in the target space support a downstream task. Early neural word-vector work evaluated learned representations through syntactic and semantic similarity tasks.3 GloVe linked vector learning to ratios of word co-occurrence probabilities,4 while later analysis connected skip-gram with negative sampling to implicit matrix factorisation.5
These methods differ, yet the geometric lesson is shared. Training constrains relations that matter to the objective. It rarely assigns a public, stable concept to every axis. If the loss depends on dot products, then a shared orthogonal rotation of all learned vectors leaves those dot products unchanged. The objective cannot prefer the original orientation over the rotated one without another constraint.
This is an identifiability issue, not merely a visualisation nuisance. Roeder, Metz and Kingma analyse a broad family of learned representations that can be identifiable in function space only up to a linear indeterminacy under stated conditions.9 The exact invariance group depends on the model and objective, but the warning is durable: equivalent behaviour does not guarantee coordinate-by-coordinate correspondence.
What survives a rotation
If every vector in one space is multiplied by the same orthogonal matrix, several useful objects survive: pairwise dot products, norms, Euclidean distances, angles, cosine similarities, Gram matrices and nearest-neighbour rankings under those measures. Linear decision boundaries can be rotated with the representation. A scatter plot changes orientation but not shape.
What does not survive is equally important: the value of coordinate 417, the statement that “dimension 12 is legal language”, a sparse pattern tied to fixed axes, or a downstream component that reads selected positions without undergoing the same transformation. A model can therefore be geometrically equivalent yet operationally incompatible with code that has leaked coordinate assumptions.
Representation comparison needs a declared invariance
Researchers often compare layers or independently trained networks. Kornblith et al. showed why the choice of representation-similarity measure matters and proposed centred kernel alignment as a way to compare representations with useful invariance properties.8 The larger principle is that a comparison method must ignore transformations considered irrelevant while remaining sensitive to changes the scientific question cares about.
Too much invariance can hide meaningful differences. If a comparison regards every invertible linear transformation as equivalent, it may declare two spaces “the same” even though Euclidean nearest-neighbour search differs. Too little invariance can report dramatic change after a harmless rotation. The correct invariance group is part of the hypothesis.
Normalised embeddings create a spherical problem
Sentence-BERT made sentence embeddings directly comparable with cosine similarity, enabling efficient similarity search relative to pairwise cross-encoding in its experiments.6 In contrastive learning, Wang and Isola identify alignment of positive pairs and uniformity of normalised features on the hypersphere as two central properties connected to the contrastive loss.7
Normalisation is therefore more than a numeric convenience. It changes the model’s usable output from points throughout space to directions on a sphere. Radial information disappears. Retrieval becomes a question about angular neighbourhoods. That may be exactly what training prepared, but it should be recorded as a design choice rather than inferred from the availability of a cosine function.
Cosine can still be distorted inside a learned space
Contextual representations need not fill space uniformly. Ethayarajh reported that representations from the studied ELMo, BERT and GPT-2 layers were not isotropic, with geometry varying across layers and contexts.10 Timkey and van Schijndel later found that a small number of rogue dimensions, often one to three in their study, could dominate standard similarity measures while differing from dimensions important to model behaviour.11
These results do not make cosine universally invalid. They show that a familiar metric can become an unreliable instrument when the representation distribution violates its tacit assumptions. Centreing, standardisation, whitening or task-trained metrics may help in particular settings, but each changes the geometry. The remedy must be evaluated against the downstream decision and a held-out baseline.
A production-shaped worked scenario: migrating a semantic index
A document service stores six million vectors from encoder A. A new encoder B passes the organisation’s sentence-level evaluation. On a shared anchor set, B’s vectors appear to be an almost pure rotation of A’s space. The migration team updates the query service first and plans to rebuild the document index later.
Each model works correctly when queries and documents come from the same space. During the mixed period, however, a B query is compared directly with A documents. The dot product now combines coordinates relative to different bases. In the synthetic laboratory accompanying this article, the correct rank-one result, “flight delay policy”, becomes “baggage claim form” after such mixing. No encoder is individually defective. The interface failed because the coordinate systems were composed without translation.
There are two plausible remedies. Re-embed the corpus with B so both sides share the new space. Or estimate an orthogonal alignment from trusted anchor pairs and map one space into the other. Orthogonal Procrustes finds the orthogonal matrix that minimises the Frobenius distance between matched point sets; SciPy exposes this precise optimisation.12
Alignment is not a universal shortcut. It assumes the spaces differ mainly by a rotation or reflection, that anchors correspond, and that the relation generalises beyond them. Semantic drift, nonlinear distortion, changed tokenisation or different norm behaviour can violate those assumptions. A low anchor error is evidence for the anchors, not proof for all future queries.
This foundation is narrower than the operational choice between vector search and relational retrieval. The adjacent article “GraphRAG, Relational State or Vector Search?” asks when relations and governed state exceed what similarity retrieval can express. The present argument explains the geometry assumed before that architecture decision even begins.
Part IVTest what should survive
A geometry contract turns the preceding mathematics into an engineering decision. It states what the vectors represent, which coordinate system produced them, which comparison rule is authorised, which transformations should be harmless, and which observations must trigger rejection. The contract is small enough to review but strong enough to generate tests.
The practical unit of assurance is not “a 768-dimensional vector”. It is a typed relationship among object, encoder, basis version, metric, normalisation and decision. Omitting any of these invites accidental equivalence.
A five-test release protocol
First, test the identity case. Recompute scores through the release path without changing the vectors. This catches sorting, normalisation, dtype and index configuration differences before geometric transformations complicate diagnosis.
Second, test a shared orthogonal rotation. Generate an orthogonal matrix Q, multiply every query and document vector by it, and compare pairwise similarities, top-k rankings and threshold decisions. Under Euclidean distance, dot product or cosine with consistent handling, results should agree within a declared numeric tolerance. A failure reveals coordinate leakage, asymmetric preprocessing or a hidden component that reads fixed dimensions.
Third, run the mixed-basis negative case. Rotate queries but leave documents untouched. The test should fail loudly through version validation or produce materially different rankings in the laboratory. If a production path silently accepts the mixture, the interface lacks a basis compatibility check.
Fourth, apply a non-orthogonal shear. Transform both sides but keep the ordinary Euclidean metric. Pairwise angles should change. This negative control proves that the test harness can distinguish information-preserving transformations from geometry-preserving ones. A harness that declares the shear harmless is checking reconstruction rather than the search geometry.
Fifth, align and read back. Estimate an orthogonal mapping from training anchors, then evaluate on separate anchors and task examples. Check residual alignment error, ranking agreement, margin changes and threshold crossings. Read back the actual retrieved objects and downstream decisions, because a small matrix error can still matter near a business boundary.
Geometric stability is not yet decision stability
An average cosine error of 0.001 may look negligible while changing many outcomes whose scores sit near a threshold. Conversely, a larger score movement may be harmless when rankings have wide margins and the consuming policy uses only the top result. Release evidence should therefore follow the decision path rather than stop at a geometric aggregate.
For retrieval, useful measures include rank-one agreement, top-k overlap, reciprocal-rank change and the distribution of score margins between the last accepted and first rejected item. For classification through prototypes, count label flips and abstention changes. For clustering, measure membership stability and inspect entities that move across operationally named groups. Slice each measure by language, document age, source, entity rarity and other conditions that can change the representation distribution.
The strictest check is a counterfactual readback: replay the same held-out inputs through old and proposed vector paths, preserve the downstream policy, and compare the resulting objects and actions. This does not make the metric causally complete, but it binds a geometric change to the decision it can affect. Near-invariance must be judged in units of consequence, not only decimal places.
| Observation | Likely interpretation | Permitted decision | Required next evidence |
|---|---|---|---|
| Shared rotation preserves pairwise scores and rankings | Pipeline respects the declared orthogonal invariance | Do not interpret individual coordinate identities | Task-level evaluation and mixed-version guard |
| Shared rotation changes results | Coordinate leakage, asymmetric transform or numeric defect | Block release | Trace preprocessing and consumers of fixed positions |
| Mixed basis is accepted | Space version is absent or unenforced | Block mixed traffic | Typed vector metadata and compatibility check |
| Shear leaves business outcome unchanged on a small sample | Task may be insensitive locally; geometry still changed | No general equivalence claim | Broader counterexamples, margins and distribution slices |
| Procrustes fits training anchors but not held-out anchors | Relation is not a stable global orthogonal map | Re-embed or dual-run | Drift analysis and nonlinear alternatives |
| Cosine is stable but threshold outcomes move | Small score changes cross an operational boundary | Recalibrate or preserve old route | Threshold, abstention and outcome analysis |
Run the projection lab
The accompanying Python programme uses six synthetic document vectors and one query. It contains a positive shared-rotation case, two negative cases, an orthogonal Procrustes repair and a minimal projection. The programme depends only on NumPy and uses assertions so that the expected invariants are executable.
In the successful case, a three-dimensional rotation changes every coordinate yet preserves the ranking and pairwise cosine matrix to about 3.33 × 10−16. Mixing the rotated query with the old index changes rank one from “flight delay policy” to “baggage claim form”. Procrustes alignment restores the original ranking. A shared shear changes the largest pairwise cosine by about 0.735 under the unchanged Euclidean rule.
# projection_lab.py: core experiment (NumPy)
import math
import numpy as np
names = [
"flight delay policy", "storm operations notice",
"baggage claim form", "catering rota",
"runway inspection", "crew scheduling guide",
]
X = np.array([
[1.00, 0.10, 0.00], [0.78, 0.62, 0.02],
[0.42, -0.05, 0.84], [-0.10, 0.20, 0.96],
[0.66, 0.22, 0.35], [0.32, 0.88, 0.05],
], dtype=float)
q = np.array([1.00, 0.15, 0.05])
def unit_rows(values):
norms = np.linalg.norm(values, axis=1, keepdims=True)
if np.any(norms == 0):
raise ValueError("cosine is undefined for a zero vector")
return values / norms
def rank(query, documents):
scores = unit_rows(documents) @ (query / np.linalg.norm(query))
return [names[i] for i in np.argsort(-scores)]
def procrustes(source, target):
left, _, right_t = np.linalg.svd(source.T @ target)
return left @ right_t
# Deterministic orthogonal rotation.
a, b = math.radians(67), math.radians(31)
Rz = np.array([[math.cos(a), -math.sin(a), 0],
[math.sin(a), math.cos(a), 0],
[0, 0, 1]])
Ry = np.array([[ math.cos(b), 0, math.sin(b)],
[0, 1, 0],
[-math.sin(b), 0, math.cos(b)]])
Q = Rz @ Ry
base = rank(q, X)
X_rot, q_rot = X @ Q, q @ Q
rotated = rank(q_rot, X_rot)
np.testing.assert_allclose(Q.T @ Q, np.eye(3), atol=1e-12)
np.testing.assert_allclose(unit_rows(X) @ unit_rows(X).T,
unit_rows(X_rot) @ unit_rows(X_rot).T,
atol=1e-12)
assert base == rotated # positive case
assert rank(q_rot, X) != base # mixed-basis failure
R = procrustes(X_rot, X)
assert rank(q_rot @ R, X) == base # alignment repair
shear = np.array([[1, 1.4, 0], [0, 1, 0.8], [0, 0, 1]])
assert rank(q @ shear, X @ shear) != base # geometry changed
print("base rank 1:", base[0])
print("mixed rank 1:", rank(q_rot, X)[0])
print("aligned rank 1:", rank(q_rot @ R, X)[0])Copy the core programme above into projection_lab.py, install NumPy and run python projection_lab.py. Its expected output is:
base rank 1: flight delay policy
mixed rank 1: baggage claim form
aligned rank 1: flight delay policyThe synthetic assertions are the substantive artefact: they verify shared-rotation invariance, expose a mixed-basis failure, test the alignment repair and reject the shear as geometry-preserving. A release implementation should add type hints, explicit zero-vector checks, pairwise error measurements and the projection decomposition.
What would weaken the central claim?
The article’s strongest claim about learned axes would weaken if a training design established stable, reproducible coordinate semantics across seeds, rotations and equivalent parameterisations, and if interventions on those coordinates produced specific downstream changes that generalised out of sample. Mere correlation with labels would not suffice. The basis would need independent anchoring and causal validation.
The migration claim would weaken if the consumer used a comparison rule invariant to the actual cross-model transformation, or if all vectors were translated through a verified mapping before scoring. The broader point would remain: compatibility comes from the declared invariant and translation, not from equal array shape.
Evidence status
Published evidence supports the mathematical basis and projection machinery, the use of cosine in sentence embeddings, linear indeterminacy in learned representations under stated conditions, anisotropy in studied contextual models and rogue-dimension effects in the cited experiments. Design inference supplies the geometry contract and release protocol. Worked scenarios use synthetic vectors; no client deployment is implied. Open hypothesis: routine rotation and mixed-basis tests will expose a useful class of hidden coupling in embedding pipelines before task regressions reach users.
Compact glossary
- Vector
- An element of a vector space. Its coordinates depend on a basis; its abstract identity does not.
- Basis
- A linearly independent set of directions that spans the space, allowing every vector to receive a unique coordinate list.
- Inner product
- A rule that supplies lengths, angles and orthogonality. The ordinary dot product is one coordinate form in an orthonormal Euclidean basis.
- Projection
- The component of a vector within a declared direction or subspace, defined relative to an inner product.
- Orthogonal transform
- A rotation or reflection whose matrix preserves Euclidean dot products, lengths, distances and angles.
- Embedding
- A mapping from objects into vector coordinates chosen or learned so that specified relations support a task.
- Identifiability
- The extent to which observed data and objectives determine a unique parameterisation or representation, possibly only up to an allowed transformation.
- Coordinate leakage
- A practitioner term here for downstream logic that depends on fixed coordinate positions despite a contract claiming basis-invariant behaviour.
Source ledger
- Authoritative teaching source
MIT OpenCourseWare, “Independence, Basis and Dimension”. Basis, independence and dimension. - Authoritative teaching source
MIT OpenCourseWare, “Projections onto Subspaces”. Projection and least-squares intuition. - Primary research
Mikolov, Chen, Corrado and Dean, “Efficient Estimation of Word Representations in Vector Space”. Neural word-vector architectures and similarity evaluation. - Primary research
Pennington, Socher and Manning, “GloVe: Global Vectors for Word Representation”. Co-occurrence-based vector learning. - Primary research
Levy and Goldberg, “Neural Word Embedding as Implicit Matrix Factorization”. Analysis of skip-gram with negative sampling. - Primary research
Reimers and Gurevych, “Sentence-BERT”. Sentence embeddings compared with cosine similarity. - Primary research
Wang and Isola, “Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere”. Normalised representation geometry. - Primary research
Kornblith, Norouzi, Lee and Hinton, “Similarity of Neural Network Representations Revisited”. Representation comparison and centred kernel alignment. - Primary research
Roeder, Metz and Kingma, “On Linear Identifiability of Learned Representations”. Conditions for function-space identifiability up to linear indeterminacy. - Primary research
Ethayarajh, “How Contextual are Contextualized Word Representations?”. Layerwise contextual geometry and anisotropy in the studied models. - Primary research
Timkey and van Schijndel, “All Bark and No Bite”. Rogue dimensions and similarity distortion in studied transformer representations. - Official implementation
SciPy documentation,scipy.linalg.orthogonal_procrustes. Orthogonal alignment objective and API contract. - Official implementation
scikit-learn documentation,cosine_similarity. L2-normalised dot-product definition.
Optional derivation: why the metric changes with the basis
Let the columns of B be a basis in ordinary world coordinates. A vector has world form v = Bx, and another has w = Bz. Their Euclidean inner product is vTw = xTBTBz. Therefore the coordinate metric is GB = BTB.
Let a new basis be C = BS. Since v = Cy = BSy, old and new coordinates satisfy x = Sy. The new metric is GC = CTC = STGBS. Substitution gives yTGCt = xTGBz. The geometry is unchanged because both coordinates and metric changed coherently.
When B is orthonormal, GB is the identity. When S is orthogonal, GC remains the identity. This is why rotations and reflections allow the ordinary dot product to remain unchanged without carrying an explicit metric matrix.
Optional extension tests for the laboratory
Add zero vectors to verify that cosine rejects undefined inputs. Add duplicated vectors to test deterministic tie handling. Quantise coordinates to several precisions and measure ranking stability near decision margins. Fit Procrustes on one anchor subset and evaluate another. Finally, perturb only vector norms to compare cosine, dot-product and Euclidean decisions under a controlled magnitude change.
Govern the geometry, not the coordinate story
The durable intuition is the transparent arrow over rotating graph paper. The arrow is the vector. The grid supplies a basis. Coordinates report how much of the arrow lies along each grid direction. Change the grid and the report changes. Preserve the inner product and the useful Euclidean geometry can remain exactly the same.
That intuition changes how a machine-intelligence system should be designed. Projection must name its subspace and metric. Cosine must justify discarding magnitude. Embedding comparisons must declare which transformations are irrelevant. Model migrations must prevent mixed spaces, and axis-level interpretations must earn their semantics through anchoring and intervention.
The changed decision is simple: never approve a vector interface because its arrays have the same length or its coordinates look similar. Approve it when the object, space, basis, metric, normalisation and invariants are explicit, the positive and negative transformations behave as predicted, and the consuming decision survives readback on held-out cases.
A vector is powerful precisely because many coordinate stories can express one geometry. Engineering becomes reliable when it preserves the geometry that matters and rejects the stories that merely look familiar.