A matrix is a bank of directional channels

A matrix can look like a table of unrelated numbers. Multiplication reveals a more useful object. An m by n matrix accepts a direction in an n-dimensional input space and produces a direction in an m-dimensional output space. Some input directions are transmitted strongly; others barely affect the output.

The singular value decomposition chooses orthogonal coordinates that expose those transmissions one at a time:

Feed the matrix exactly the direction vᵢ and the output is σᵢuᵢ. Feed a mixture of right singular directions and linearity sends each component through its own channel before adding the outputs. The rank is the number of non-zero channels.

This geometry differs from the eigenstructure of a square operator. Eigenvectors identify directions preserved under a transformation and are especially useful for repeated dynamics. Singular vectors come in left-right pairs and exist for rectangular matrices. They answer a one-pass amplification and approximation question. The adjacent paper on eigenstructure and persistent directions treats the repeated-transformation case.

Rank-one atoms and the reason a cliff matters

Each outer product uᵢvᵢᵀ is a rank-one matrix. Multiplying it by σᵢ gives one directional channel. The complete matrix is a sum of these atoms. A low-rank approximation keeps only a prefix:

A steep spectrum creates an apparent bargain. In the running example, the singular values are 12, 5, 2, 0.1, 0.03 and 0.01. The first three channels contain almost all squared singular-value energy. The remaining directions seem negligible when the matrix is judged globally.

The cliff is useful because it says the operator’s action is concentrated in a few Euclidean channels. It does not say why. Repetition, correlated features, shared latent causes, redundant filters or a narrow adaptation can all produce concentration. Nor does it say that the tail is noise. A small channel may be a weak nuisance, a rare signal, an adversarial route or a measurement scale chosen by the modeller.

A spectrum belongs to a representation

Orthogonal changes of basis preserve singular values because they merely rotate the input or output coordinates. Arbitrary rescaling does not. Measuring one feature in pounds rather than thousands of pounds multiplies its column by one thousand. The exact rank survives an invertible rescaling, but the apparent cliff, the retained-energy percentage and the best Euclidean rank-k fit can all change.

This is not a technical nuisance. The norm declares which perturbations count as equally large. Raw amplitude may let a high-variance but irrelevant feature dominate. Standardisation may make ordinary feature changes more comparable. Whitening can flatten a spectrum by design and remove the very concentration one hoped to exploit. A singular-value plot is evidence about a matrix after representation choices, not about the world before them. Record units, centring, scaling and weighting with the rank decision. If a plausible rescaling changes the accepted rank or rotates the retained subspace, treat that sensitivity as part of the failure surface.

What truncation guarantees, and what it leaves outside

The classical result is unusually strong. The Eckart-Young theorem, later generalised by Mirsky, states that truncated SVD is a best rank-k approximation under the spectral and Frobenius norms, with analogous results across unitarily invariant norms. [1] [2] “Best” has a precise boundary: fixed matrix, fixed rank and declared norm.

The Frobenius norm squares every entry error, sums those squares and takes a square root. For truncated SVD, its squared residual is the sum of squared discarded singular values. The spectral norm asks for the largest output error over all unit-length inputs. Its residual is the first omitted singular value, σₖ₊₁.

This distinction already changes how percentages should be read. The running rank-three approximation retains 99.993642 per cent of squared energy, yet its relative Frobenius error is 0.797369 per cent because the norm takes a square root. Its spectral error is 0.1, the largest missed directional gain. All three statements are correct. They answer different questions.

Minimal worked example: a guarantee that passes while a task fails

The executable laboratory constructs a diagonal matrix with the six-value spectrum, then permutes its singular directions. A fixed input-output probe inspects coordinate four. In the top-aligned case, coordinate four carries σ₁ and survives rank three. In the tail-aligned case, it carries σ₄ and disappears.

This is stronger than showing one unfortunate reconstruction. The matrices share every singular value, so the rank-three energy, Frobenius error and spectral error remain the same. The negative task result cannot be repaired by inspecting the singular-value plot more carefully. The missing evidence is task alignment.

Negative control: the matrix has no low-rank bargain

Set the matrix to the eight-dimensional identity. Every singular value is one. No direction is more compressible than another. A rank-two approximation retains 25 per cent of squared energy and has 86.603 per cent relative Frobenius error. Any two-dimensional subspace is as good as any other under the norm, so the spectrum supplies no preferred semantic choice.

This negative control matters because a low-rank method can still produce an output when low-rank structure is absent. The factorisation will run, the factors will be thin and the code will report a rank. The failed claim is that the reduced rank preserves enough of the operator to justify use.

From matrix error to task error

A task evaluates the approximation through inputs, outputs, thresholds and consequences. One compact expression is:

This loss may agree with Frobenius reconstruction when entries are equally important, inputs are well represented by the Euclidean geometry and the downstream map is sufficiently smooth. It can diverge sharply when rare cases, asymmetric harm, thresholds or heterogeneous units dominate. Reweighting entries creates a weighted low-rank approximation problem; in general, adding arbitrary weights makes the optimisation substantially harder and truncated SVD no longer solves it directly. [5]

A decision margin can bridge the norm to a threshold

For a linear downstream score, the bridge can be written explicitly. Let s(x) = cᵀAx and issue the positive decision when s(x) ≥ τ. The approximation changes the score by δ(x) = cᵀ(A − Aₖ)x. Cauchy-Schwarz and the spectral residual give a sufficient bound:

The condition is sufficient, not necessary, and it can be conservative because it guards every input and output direction at once. Direct task probes can show a much smaller realised residual. Its value is diagnostic: it names the missing bridge. In the cancellation example, c = x = e₄, the original score is 0.1 and the threshold is 0.05. The margin is 0.05, while the global bound is 0.1, so preservation cannot be certified. The exact compressed score is zero and the decision flips. For a non-linear downstream map, an appropriate local or global Lipschitz bound can play the same role, provided its validity region is declared.

The appropriate intervention is often selective. Keep the dominant SVD subspace for common variation, then preserve a small protected subspace for known critical directions. Alternatively optimise a task-aware factorisation, retain a residual path, or decline compression for that component. Each choice changes the mathematical claim and must be evaluated against an uncompressed baseline.

The observation that would weaken this article’s preferred explanation is straightforward: if task behaviour is preserved across aggressive rank reduction even when the spectrum is flat, then singular-value concentration is not the mechanism earning the result. Regularisation, data redundancy, retraining or a forgiving task metric may be doing the work. The release evidence should name the mechanism it actually supports.

flowchart TD
  A["The cancellation mark that<br/>carried almost no energy"] --> B["A matrix is a bank of<br/>directional channels"]
  B --> C["What truncation guarantees,<br/>and what it leaves outside"]
  C --> D["When lower rank earns its cost"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#D95545,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

When lower rank earns its cost

Low rank becomes an engineering decision only after the factors make something materially cheaper. Write an approximation as A ≈ BC, where B is m by k and C is k by n. The dense matrix stores mn scalars. The factors store k(m + n). A dense matrix-vector product has nominal work proportional to mn; the two factor products have work proportional to k(m + n).

For a 4096 by 4096 matrix, rank 64 uses 524,288 factor scalars instead of 16,777,216 dense scalars, or 3.125 per cent. That is a strong algebraic signal. It still does not promise a 32-fold wall-clock speed-up.

Two thin multiplications create an intermediate vector and may invoke separate kernels. Small batches can make launch overhead visible. Hardware may already execute the dense shape efficiently. Compression can shift the bottleneck from arithmetic to memory, or from one device to communication. In training, optimiser state and activation memory may dominate. In serving, merging factors into the base weight removes extra operations but also removes the low-rank storage advantage for that merged copy.

Measure the resource that constrains the system. Parameter count is appropriate for adapter storage. Memory traffic may govern inference. End-to-end latency matters for an interactive path. Throughput matters for a batch service. Energy, device fit, checkpoint distribution or tenant switching may be the actual objective.

The saving must repay the transformation

Compression also has a time boundary. Let C_build be the one-off cost of computing, validating and distributing the factors. Let C_dense and C_fact be the measured per-use costs at matched quality. The factors break even only after more than C_build/(C_dense − C_fact) uses, assuming the denominator is positive.

That simple ratio exposes several false bargains. A matrix rebuilt every hour may never amortise its decomposition. A rarely used tenant adapter may save storage but add cold-load and switching latency. A large batch can favour thin general matrix multiplies, while a single request can be dominated by launch overhead. Merging an adapter into a base weight can improve the hot path but duplicates dense weights across variants. Report the measured crossover in requests, batches or operating hours, not only an algebraic compression ratio. The valid horizon ends when the matrix, workload shape, hardware kernel or quality gate changes materially.

Finding the subspace without paying for the full decomposition

A full dense SVD can cost more than the downstream savings when the matrix is large or changes frequently. Randomised algorithms address this by sampling a subspace that captures most of the matrix’s action, compressing the matrix to that subspace and completing a smaller deterministic factorisation. Halko, Martinsson and Tropp give a modular framework and error analysis for this approach. [6] SciPy also exposes partial sparse SVD solvers for selected singular triplets. [7]

The approximation algorithm adds another evidence boundary. Report oversampling, power iterations, random seed policy, residual estimates and repeatability. A fast approximate SVD that misses a clustered or weakly separated subspace can change the task failure even before truncation is considered.

A crowded cutoff makes the retained directions unstable

A large gap between σₖ and σₖ₊₁ separates the retained and discarded subspaces. When those values are close, a small perturbation to the matrix can rotate the estimated top-k subspace substantially even while the reconstruction error barely changes. Wedin’s perturbation analysis relates singular-subspace rotation to the size of the perturbation and the separation around the target subspace. [14]

This changes interpretation and testing. Do not assign a stable semantic story to one singular vector inside a near-tied cluster; rotations within the cluster can produce equally valid bases. Re-estimate the matrix across samples or periods, compare retained subspaces through principal angles, and increase rank past the cluster when the decision requires the whole group. A cliff can justify a cut; a crowded shelf calls for a subspace, uncertainty and sometimes no cut at all.

Neural layers: the speed-up is empirical

Low-rank decompositions have long been used to approximate expensive neural layers. Denton and colleagues exploited linear structure in convolutional filters and reported layer speed-ups with bounded accuracy change in their tested vision models. [8] Such results establish feasibility for those architectures and workloads. They do not make every learned weight matrix compressible at the same rank.

A responsible experiment compares four conditions: the original layer, the direct truncated factorisation, the factorised layer after bounded retraining, and a matched alternative such as pruning or quantisation. Direct SVD tests the low-rank structure already present. Retraining tests whether the network can reorganise around the bottleneck. Once retraining occurs, the Eckart-Young guarantee is no longer the complete causal explanation.

LoRA is a constraint on change

Low-Rank Adaptation applies the same factor geometry to model updates. A pretrained weight matrix W₀ remains frozen while training learns two smaller matrices whose product supplies an increment:

The original LoRA paper reported large reductions in trainable parameters and training memory while matching or exceeding full fine-tuning on the models and tasks it evaluated. [9] Earlier intrinsic-dimension experiments had already shown that some pretrained language models could recover much of task performance through low-dimensional reparameterisations. [10] Together, these results support a bounded claim: many useful adaptations can be reached through surprisingly small subspaces.

They do not establish a universal low-rank law. Intrinsic dimension describes the number of free optimisation coordinates needed to reach a performance level under a particular setup. Matrix rank describes the dimensionality of a specific weight update. A low-dimensional training trajectory can be distributed across many matrices; a rank-constrained update can still introduce novel singular directions. Similar benchmark scores can hide different internal solutions and different off-domain behaviour.

Positive case and representational negative case

The laboratory creates a 12 by 12 target update with true rank two. Its two non-zero singular values are 3 and 1.5. An SVD oracle splits those values across B and A, producing a rank-two LoRA update with zero numerical reconstruction error.

The negative target is the 12-dimensional identity. It requires twelve independent directions of equal strength. The best possible rank-two update has 91.287093 per cent relative Frobenius error. This is an expressivity result, not an optimisation accident. No training schedule can make a rank-two product equal the full-rank target.

What recent evidence adds to the boundary

AdaLoRA allocates rank budget unevenly across weight matrices, motivated by the observation that equal allocation can waste capacity on less important updates. [11] That design is consistent with the central mechanism: different matrices can have different useful update spectra.

Limitation studies sharpen the claim. Biderman and colleagues found that standard low-rank settings substantially underperformed full fine-tuning in their programming and mathematics regimes, while preserving more off-domain behaviour; the full fine-tuning perturbations had much higher rank than typical LoRA configurations. [12] Shuttleworth and colleagues found that LoRA and full fine-tuning could yield spectrally different weight changes, including new high-ranking singular vectors they call intruder dimensions, even when downstream performance looked similar. [13]

These results support two competing explanations for LoRA’s value. The first is efficient representation: the task-relevant update really does occupy a low-rank subspace. The second is beneficial constraint: LoRA learns less, limits drift and acts as a regulariser that is sufficient for the evaluated slice. Both can be true in different regimes. A benchmark score alone does not distinguish them.

Rank is local to a module and an objective

A transformer does not contain one adaptation matrix. Attention projections, feed-forward projections and other trainable components have different shapes, roles, gradients and opportunities for reuse. Giving every selected matrix the same rank assumes that each needs the same number of independent update directions. AdaLoRA’s uneven allocation is one response, but the more general decision is experimental rather than architectural habit.

Run a module-wise rank sweep while holding the total trainable budget and data constant. Record target-task quality, protected slices, calibration, off-domain retention, optimiser stability and serving cost. Where feasible, train a bounded full-fine-tuning reference and inspect the singular spectrum of each resulting ΔW. If quality improves steadily as rank rises and the reference update has substantial tail mass, the adapter is an expressivity bottleneck. If low rank matches the target while forgetting less, the constraint may be providing useful regularisation. If different modules saturate at different ranks, move budget rather than raising every adapter together. The relevant claim is never “rank 16 works”; it is “this rank allocation preserved these behaviours under this objective and comparison set”.

Executable SVD and LoRA laboratory

The programme below uses only NumPy and deterministic synthetic matrices. It contains a positive compression case, a task-alignment failure, a flat-spectrum negative control, a rank-two LoRA success and a full-rank LoRA failure. The assertions are part of the evidence boundary.

#!/usr/bin/env python3
"""Synthetic SVD and LoRA laboratory for Paper 21.

The programme tests three claims:
1. Truncated SVD exactly controls unweighted matrix reconstruction error.
2. High retained spectral energy does not guarantee preservation of a fixed task probe.
3. A rank-r LoRA update can reproduce a rank-r target update, but cannot reproduce an
   arbitrary higher-rank update at the same rank.

Only NumPy is required. All data are synthetic and deterministic.
"""
from __future__ import annotations

from dataclasses import dataclass
from typing import Iterable

import numpy as np
from numpy.typing import NDArray

Array = NDArray[np.float64]


@dataclass(frozen=True)
class ApproximationReport:
    rank: int
    retained_energy: float
    relative_frobenius_error: float
    spectral_error: float
    dense_parameters: int
    factor_parameters: int


def truncated_svd(matrix: Array, rank: int) -> tuple[Array, Array, Array, Array]:
    """Return the rank-r truncated SVD approximation and its factors."""
    if matrix.ndim != 2:
        raise ValueError("matrix must be two-dimensional")
    limit = min(matrix.shape)
    if not 1 <= rank <= limit:
        raise ValueError(f"rank must be in [1, {limit}]")
    u, singular_values, vh = np.linalg.svd(matrix, full_matrices=False)
    approximation = (u[:, :rank] * singular_values[:rank]) @ vh[:rank, :]
    return approximation, u[:, :rank], singular_values[:rank], vh[:rank, :]


def report(matrix: Array, rank: int) -> ApproximationReport:
    approximation, _, _, _ = truncated_svd(matrix, rank)
    singular_values = np.linalg.svd(matrix, compute_uv=False)
    total_energy = float(np.sum(singular_values**2))
    kept_energy = float(np.sum(singular_values[:rank] ** 2))
    residual = matrix - approximation
    m, n = matrix.shape
    return ApproximationReport(
        rank=rank,
        retained_energy=kept_energy / total_energy,
        relative_frobenius_error=float(
            np.linalg.norm(residual, "fro") / np.linalg.norm(matrix, "fro")
        ),
        spectral_error=float(np.linalg.norm(residual, 2)),
        dense_parameters=m * n,
        factor_parameters=rank * (m + n),
    )


def permutation_matrix(order: Iterable[int]) -> Array:
    order = list(order)
    result = np.zeros((len(order), len(order)), dtype=float)
    for column, row in enumerate(order):
        result[row, column] = 1.0
    return result


def task_value(matrix: Array, input_probe: Array, output_probe: Array) -> float:
    """A fixed, declared downstream linear measurement."""
    return float(output_probe @ matrix @ input_probe)


def best_lora_update(target_update: Array, rank: int) -> tuple[Array, Array, Array]:
    """Construct the best rank-r update as LoRA factors B @ A.

    This is an SVD oracle for representational capacity, not a training algorithm.
    Splitting each retained singular value across B and A gives B @ A equal to the
    optimal rank-r approximation under the Frobenius and spectral norms.
    """
    u, singular_values, vh = np.linalg.svd(target_update, full_matrices=False)
    roots = np.sqrt(singular_values[:rank])
    b = u[:, :rank] * roots
    a = roots[:, None] * vh[:rank, :]
    return b @ a, b, a


def relative_error(target: Array, approximation: Array) -> float:
    return float(np.linalg.norm(target - approximation, "fro") / np.linalg.norm(target, "fro"))


def main() -> None:
    # A steep spectrum: rank 3 retains nearly all squared singular-value energy.
    spectrum = np.array([12.0, 5.0, 2.0, 0.1, 0.03, 0.01])
    base = np.diag(spectrum)
    rank = 3
    base_report = report(base, rank)

    # Same singular values, same retained energy, different alignment to one task probe.
    # In TOP_ALIGNED, coordinate 4 carries sigma_1 and survives rank 3.
    # In TAIL_ALIGNED, coordinate 4 carries sigma_4 and is discarded at rank 3.
    top_permutation = permutation_matrix([3, 1, 2, 0, 4, 5])
    top_aligned = top_permutation @ base @ top_permutation.T
    tail_aligned = base.copy()
    top_rank3, *_ = truncated_svd(top_aligned, rank)
    tail_rank3, *_ = truncated_svd(tail_aligned, rank)

    input_probe = np.eye(6)[3]
    output_probe = np.eye(6)[3]
    threshold = 0.05
    top_original = task_value(top_aligned, input_probe, output_probe)
    top_compressed = task_value(top_rank3, input_probe, output_probe)
    tail_original = task_value(tail_aligned, input_probe, output_probe)
    tail_compressed = task_value(tail_rank3, input_probe, output_probe)

    # Negative control: no cliff. Every direction carries equal gain.
    flat = np.eye(8)
    flat_report = report(flat, rank=2)

    # LoRA capacity experiment. The positive update truly has rank 2.
    rng = np.random.default_rng(21)
    left, _ = np.linalg.qr(rng.normal(size=(12, 2)))
    right, _ = np.linalg.qr(rng.normal(size=(12, 2)))
    positive_update = left @ np.diag([3.0, 1.5]) @ right.T

    # The negative update distributes equal energy across all 12 directions.
    negative_update = np.eye(12)
    positive_lora, positive_b, positive_a = best_lora_update(positive_update, rank=2)
    negative_lora, negative_b, negative_a = best_lora_update(negative_update, rank=2)

    print("=== truncated SVD ===")
    print(f"rank-{rank} retained energy: {100 * base_report.retained_energy:.6f}%")
    print(
        "rank-3 relative Frobenius error: "
        f"{100 * base_report.relative_frobenius_error:.6f}%"
    )
    print(f"rank-3 spectral error: {base_report.spectral_error:.6f}")
    print(
        f"dense parameters={base_report.dense_parameters}, "
        f"factor parameters={base_report.factor_parameters}"
    )

    print("\n=== fixed task probe under the same spectrum ===")
    print(f"top-aligned original={top_original:.3f}, compressed={top_compressed:.3f}")
    print(f"tail-aligned original={tail_original:.3f}, compressed={tail_compressed:.3f}")
    print(
        "top-aligned decision preserved: "
        f"{(top_original >= threshold) == (top_compressed >= threshold)}"
    )
    print(
        "tail-aligned decision preserved: "
        f"{(tail_original >= threshold) == (tail_compressed >= threshold)}"
    )

    print("\n=== flat-spectrum negative control ===")
    print(f"rank-2 retained energy: {100 * flat_report.retained_energy:.3f}%")
    print(
        "rank-2 relative Frobenius error: "
        f"{100 * flat_report.relative_frobenius_error:.3f}%"
    )

    print("\n=== LoRA representational capacity ===")
    print(f"positive rank-2 error: {100 * relative_error(positive_update, positive_lora):.9f}%")
    print(f"negative rank-2 error: {100 * relative_error(negative_update, negative_lora):.6f}%")
    print(f"positive factor shapes: B{positive_b.shape}, A{positive_a.shape}")
    print(f"negative factor shapes: B{negative_b.shape}, A{negative_a.shape}")

    # Positive and negative assertions form the executable claim boundary.
    assert base_report.retained_energy > 0.9999
    assert np.isclose(base_report.spectral_error, spectrum[rank])
    assert (top_original >= threshold) == (top_compressed >= threshold)
    assert (tail_original >= threshold) != (tail_compressed >= threshold)
    assert np.isclose(flat_report.retained_energy, 0.25)
    assert relative_error(positive_update, positive_lora) < 1e-12
    assert relative_error(negative_update, negative_lora) > 0.90
    assert np.allclose(positive_lora, positive_b @ positive_a)
    assert np.allclose(negative_lora, negative_b @ negative_a)


if __name__ == "__main__":
    main()

Expected output

=== truncated SVD ===
rank-3 retained energy: 99.993642%
rank-3 relative Frobenius error: 0.797369%
rank-3 spectral error: 0.100000
dense parameters=36, factor parameters=36

=== fixed task probe under the same spectrum ===
top-aligned original=12.000, compressed=12.000
tail-aligned original=0.100, compressed=0.000
top-aligned decision preserved: True
tail-aligned decision preserved: False

=== flat-spectrum negative control ===
rank-2 retained energy: 25.000%
rank-2 relative Frobenius error: 86.603%

=== LoRA representational capacity ===
positive rank-2 error: 0.000000000%
negative rank-2 error: 91.287093%
positive factor shapes: B(12, 2), A(2, 12)
negative factor shapes: B(12, 2), A(2, 12)

.

The six-dimensional toy has no storage saving at rank three because 3(6 + 6) = 36, exactly the dense count. That deliberate detail prevents a common overclaim: lower mathematical rank is not automatically cheaper at a given shape. The 4096-dimensional calculation in Figure 6 supplies the separate economics case.

The rank acceptance test

A defensible rank is the intersection of three ledgers. The matrix ledger asks what the norm guarantees. The task ledger asks what the system must preserve. The economics ledger asks whether the factorisation improves the constrained resource on the target runtime.

The three ledgers diagnose different failures

When all three pass, choose the smallest rank with adequate margin and issue a bounded receipt. A matrix pass with a task failure means the norm is solving the wrong objective, so protect the tail or change the factorisation. A task pass with an economics failure means the representation is adequate but deployment gains nothing. A task pass despite a flat spectrum weakens low-rank concentration as the explanation; retraining, regularisation or data redundancy may deserve the credit. When matrix, task and economics all fail, stop tuning rank and reject the shortcut. The test is valuable because it changes the next experiment, not because it produces one universal score.

A compact test protocol

1. Define the object. State whether rank applies to raw data, an activation map, a fixed weight, a weight update or a task-specific residual. These objects support different interpretations.
2. Plot the spectrum with uncertainty. Record the norm, normalisation, estimation method and sensitivity of the retained subspace. A cliff in one scaling may disappear in another.
3. Run the norm baseline. Compute exact or bounded Frobenius and spectral residuals. Include a flat-spectrum or shuffled negative control so that the method can fail.
4. Probe the tail deliberately. Search for rare, protected or high-consequence behaviours aligned with omitted directions. Vary one alignment at a time, as in the first thought experiment.
5. Measure the actual bottleneck. Compare against dense and at least one serious alternative on target hardware, with the same quality gate and workload shape.
6. Issue a bounded receipt. Approve the rank only for the tested matrix version, input distribution, task thresholds and runtime. Name the trigger for re-evaluation.

The protocol also tells us when to stop increasing rank. More rank monotonically improves the unweighted truncated-SVD reconstruction, but deployment value can plateau. Once every protected task passes with margin and the economic objective is met, a higher rank adds cost without changing the decision. Conversely, a rank that clears an average benchmark but fails one protected tail probe remains unacceptable.

Compress the matrix, not the decision

SVD turns a matrix into an ordered set of directional channels. Keeping the first k retains the strongest Euclidean gains and gives exact reconstruction error under declared norms. That mechanism is why low rank can compress data, accelerate linear layers and create small adaptation factors.

The same mechanism defines the boundary. The tail is ordered by gain, not by semantic importance. A faint cancellation mark, a rare subgroup, an independent control direction or a high-consequence exception can occupy little global energy. When one of those directions is discarded, the theorem remains true and the system can still be wrong.

The practical unit of approval is therefore a rank plus its evidence boundary. Record the matrix and norm, the retained and omitted subspaces, the task probes, the negative control, the runtime measurement and the conditions under which the result expires. A singular-value cliff earns investigation. It does not earn release.

For fixed matrix compression, use truncated SVD as the reconstruction baseline and compare task-aware alternatives when the loss is asymmetric. For large matrices, use partial or randomised methods with residual checks. For neural layers, separate direct decomposition from retraining because the latter can reorganise the representation. For LoRA, treat the adapter rank as a hypothesis about update geometry and test it against higher-rank, sparse or full-tuning references where consequence justifies the expense.

The decision changed by this paper is precise: choose the smallest rank that passes matrix fidelity, protected task behaviour and measured economics together. If no rank passes all three, the system has not discovered a compression opportunity. It has discovered where the important complexity lives.