TLDR
- A kernel computes a similarity that makes a chosen family of functions simple. It changes the geometry used by the learner without requiring an explicit list of transformed features.
- The kernel matrix turns every training example into a relation with every other example. Learning then becomes the controlled combination of those relations, usually with a margin or smoothness penalty.
- Kernel choice is a modelling claim about invariance, locality and scale. The most important hyperparameter often decides what the learner is allowed to regard as a neighbourhood.
- Kernels lose their advantage when similarity is poorly justified, sample size makes the Gram matrix unaffordable, or a transparent linear or tree baseline already captures the useful structure.
- Use a kernel only after writing the relation it should preserve, testing a non-kernel baseline, and measuring sensitivity to scale, approximation and distribution shift.
The straight line that became a circle
Place eight observations on a sheet of paper. Four sit close to the centre. Four form a ring around them. Mark the centre points positive and the ring points negative. No single straight line on the sheet can separate the labels. Add a second line and the partition still leaves awkward corners. Yet a learner using radial similarity can separate the two groups with a smooth closed boundary.
Nothing moved on the sheet. The labels did not change. The learner received no secret rule saying that circles matter. What changed was the answer to one question: which pairs of observations should exert influence on each other? Near points were allowed to share evidence strongly; distant points were allowed to share little. A linear rule over those similarities produced a nonlinear boundary in the original coordinates.
That small trick is the entrance to kernel methods. It is also where their explanation often goes wrong. Saying that a kernel “maps data into a higher-dimensional space” is mathematically useful, but it can leave the causal mechanism hidden behind an imagined elevator. The learner does not need to manufacture and store a mysterious tower of coordinates. It needs a valid procedure for comparing two observations as if their inner product had been computed in a feature space.
The practical consequence comes early. A kernel is valuable when a defensible relation makes the target regular, and costly when the relation merely makes the training set convenient. The distinction is the difference between a model that transfers and a smooth memoriser.
Similarity is an operation
Similarity sounds like a property waiting inside the data. It is usually a decision. A bank transaction can be close to another transaction because the amounts are close, the merchants are related, the times are adjacent, the account histories have the same shape, or the two events occupy similar positions in a payment network. These definitions preserve different relations. They produce different neighbours and therefore different predictions.
Consider three short customer histories. The first and second contain the same purchases in a different order. The second and third occur in the same order but at very different amounts. A bag-of-events similarity treats the first pair as almost identical. A sequence kernel may separate them. A magnitude-sensitive radial kernel may join the first pair or the second pair depending on scale. No calculation can select among these meanings without a modelling claim.
For two observations (x) and (z), a kernel is written (k(x,z)). In the familiar geometric case it behaves as an inner product after a transformation:
Here, is a feature map and is the feature space, often a reproducing kernel Hilbert space. The equation matters because an inner product supplies length, angle and projection. It lets optimisation use the geometry while avoiding explicit construction of every coordinate in .
The phrase “kernel trick” names this substitution. An algorithm that depends on examples only through inner products can replace with . The resulting algorithm is linear in the feature space and can be nonlinear in the observed coordinates. The trick does not make every algorithm nonlinear. It applies where the objective and prediction rule can be written through pairwise inner products.
Figure 1. Similarity bends the working geometry
Thought experiment: keep the points, rotate the meaning
Imagine a dataset containing colour, mass and surface texture for pieces moving along a conveyor. Defects arise from a chemical reaction expressed through a narrow combination of colour and texture. First compare examples with ordinary Euclidean distance after standardising every column. Then compare them using a kernel invariant to changes in mass but sensitive to the colour-texture interaction. Keep the points, labels, optimiser and regularisation fixed.
If the second model transfers better, the gain did not come from a universally more powerful learner. It came from making the relevant relation cheap to express. Now reverse the environment: a supplier changes the material so mass becomes the early defect signal. The invariance that helped before becomes blindness. An invariance is a bet about which changes should not matter.
This thought experiment gives kernel selection a sharper form. Do not ask only whether the radial basis function, polynomial or string kernel scores highest. Ask what transformation of the world each treats as harmless, which distinction each compresses, and how a future change could invalidate that choice.
The kernel mechanism
Given training observations , compute the Gram matrix with entries
For a valid positive-semidefinite kernel, every real vector satisfies . This condition is more than mathematical ceremony. It says the pairwise similarities can be realised as inner products without introducing a negative squared length. It gives convex kernel objectives their stable geometry.
The matrix is the relational view of the training set. A row records how one example sees every other example. Repeated rows indicate observations the kernel cannot distinguish. Nearly block-diagonal structure suggests separated neighbourhoods. A matrix close to all ones says almost everything looks alike. A matrix close to the identity says almost nothing shares influence. Both extremes can fit badly for opposite reasons.
Figure 2. The Gram matrix is the model's relational memory
Many kernel estimators obtain a function of the form
The coefficient determines how training example contributes at the query point. In a support vector machine, only examples with active constraints carry non-zero influence in the idealised solution. They define the margin. In kernel ridge regression, regularisation distributes influence more smoothly across examples. In Gaussian process regression, the same covariance function participates in a posterior distribution and exposes predictive uncertainty under declared assumptions.
The representer theorem explains why this finite expansion appears even when the feature space is enormous or infinite-dimensional. For a broad class of regularised objectives, the solution lies in the span of kernels centred on the training observations. The search over functions collapses to the coefficients .
That result supplies the durable intuition: the trained function is assembled from shaped echoes of observed examples. The kernel decides the shape of an echo. The loss decides which discrepancies matter. The regulariser controls how violently the echoes may combine.
Margin and smoothness answer different questions
A maximum-margin classifier chooses a boundary that separates labelled examples while protecting distance to the closest cases. Its regularisation parameter trades a wider margin against training violations. Kernel ridge regression instead balances squared error against the norm of the function in the kernel space. A Gaussian process combines a covariance prior with a likelihood. These methods can share a kernel and still express different beliefs about noise, robustness and uncertainty.
Treating “the kernel model” as one object hides those distinctions. The kernel defines geometry. The surrounding estimator defines the learning and decision rule. The evaluation must vary them separately when the argument concerns mechanism.
Figure 3. A straight separator can describe a curved boundary
flowchart TD A["The straight line that became<br/>a circle"] --> B["Similarity is an operation"] B --> C["The kernel mechanism"] C --> D["One small laboratory"] D -. "re-test" .-> A classDef first fill:#0a1424,stroke:#B96D00,stroke-width:2px,color:#e6f3ff classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff class A first class B,C,D step
One small laboratory
Return to the centre and ring. Use the radial basis function kernel
The squared distance measures separation in the input coordinates. The exponential converts that distance into influence. The parameter controls how quickly influence decays. A large creates narrow neighbourhoods; a small makes distant observations look similar.
In the supplied artefact, eight points form a tight inner ring and twelve form an outer ring. Kernel ridge classification fits coefficients by solving
where is the identity matrix, contains labels, and penalises unstable coefficients. Leave-one-out evaluation removes each point in turn, fits on the remainder, and predicts the held-out label. This is intentionally small enough to inspect without a machine-learning library.
. It uses only the Python standard library and includes positive and negative cases.
#!/usr/bin/env python3
"""A dependency-free kernel laboratory with positive and negative cases."""
from __future__ import annotations
from math import cos, exp, pi, sin
def rbf(x: tuple[float, ...], y: tuple[float, ...], gamma: float) -> float:
return exp(-gamma * sum((a - b) ** 2 for a, b in zip(x, y)))
def solve(matrix: list[list[float]], vector: list[float]) -> list[float]:
"""Gauss-Jordan solve for the deliberately tiny examples in this article."""
augmented = [row[:] + [value] for row, value in zip(matrix, vector)]
size = len(augmented)
for column in range(size):
pivot = max(range(column, size), key=lambda row: abs(augmented[row][column]))
augmented[column], augmented[pivot] = augmented[pivot], augmented[column]
divisor = augmented[column][column]
if abs(divisor) < 1e-12:
raise ValueError("Singular system: change the kernel or regularisation")
augmented[column] = [value / divisor for value in augmented[column]]
for row in range(size):
if row == column:
continue
factor = augmented[row][column]
augmented[row] = [
current - factor * reference
for current, reference in zip(augmented[row], augmented[column])
]
return [row[-1] for row in augmented]
def fit_kernel_ridge(
points: list[tuple[float, ...]],
labels: list[int],
gamma: float,
regularisation: float = 0.05,
) -> list[float]:
gram = [[rbf(x, y, gamma) for y in points] for x in points]
for index in range(len(gram)):
gram[index][index] += regularisation
return solve(gram, [float(label) for label in labels])
def predict(
train: list[tuple[float, ...]],
weights: list[float],
point: tuple[float, ...],
gamma: float,
) -> int:
score = sum(weight * rbf(example, point, gamma) for weight, example in zip(weights, train))
return 1 if score >= 0 else -1
def leave_one_out_accuracy(points, labels, gamma: float) -> float:
correct = 0
for held_out in range(len(points)):
train_x = [x for index, x in enumerate(points) if index != held_out]
train_y = [y for index, y in enumerate(labels) if index != held_out]
weights = fit_kernel_ridge(train_x, train_y, gamma)
correct += predict(train_x, weights, points[held_out], gamma) == labels[held_out]
return correct / len(points)
def run() -> None:
# Two concentric classes: no single line separates them in input coordinates.
points = [
(0.22 * cos(2 * pi * index / 8), 0.22 * sin(2 * pi * index / 8))
for index in range(8)
] + [
(1.2 * cos(2 * pi * index / 12), 1.2 * sin(2 * pi * index / 12))
for index in range(12)
]
labels = [1] * 8 + [-1] * 12
useful = leave_one_out_accuracy(points, labels, gamma=2.0)
over_smoothed = leave_one_out_accuracy(points, labels, gamma=0.001)
print({"useful_gamma_accuracy": useful, "over_smoothed_accuracy": over_smoothed})
assert useful >= 0.875, "Positive case: the similarity should preserve the rings"
assert over_smoothed < useful, "Negative case: a nearly constant kernel should lose structure"
if __name__ == "__main__":
run()
The positive case uses . Nearby points on each ring preserve enough shared influence for the held-out example to inherit the correct class. The negative case uses . The similarity matrix approaches a constant matrix, so the geometry can no longer preserve the two radii. The code asserts that the useful scale outperforms the over-smoothed one.
This is not a claim that is generally correct. The numbers have meaning only relative to the coordinate scale. Multiply every coordinate by one hundred and the effective neighbourhood collapses unless changes. A release artefact must therefore record feature units, scaling and the distribution used to tune the kernel.
A negative control that looks nonlinear
Add one engineered feature: distance from the origin. A linear classifier on that single feature can separate the rings. It is simpler to train, cheaper to explain and easier to monitor. If radial distance has a stable domain meaning, the engineered baseline is the better model.
This negative control prevents a common overclaim. The kernel did not discover an irreducibly nonlinear truth. It supplied a reusable way to express a relation that an informed feature designer could make explicit. The kernel earns its complexity when the relation is defensible but inconvenient or too rich to enumerate, not when it merely rescues missing feature work.
A kernel is a prior
Every kernel favours some functions over others. The radial kernel favours smooth variation under its distance and length scale. A polynomial kernel makes interactions up to a chosen degree convenient. A string kernel can reward shared subsequences. A graph kernel can compare walks, subtrees or neighbourhood patterns. A periodic kernel makes repeating functions cheap. These preferences are structural assumptions.
Calling the kernel a prior is literal in Gaussian processes and useful elsewhere. It directs the learner towards functions smooth in the chosen geometry. Regularisation then limits how far the fit may depart from that preference. Data chooses within the permitted family; it does not erase the family’s shape.
Thought experiment: the duplicate city
Suppose a delivery model uses latitude and longitude with an RBF kernel. Duplicate the same street layout on another island one thousand kilometres away. Locally, the operational problems are identical. Globally, Euclidean coordinates treat corresponding streets as unrelated. A kernel based on local road-network position could transfer patterns; the geographic RBF cannot.
Now introduce weather fronts that move geographically across both regions. Network position alone becomes inadequate because physical distance carries causal information. A composite kernel might combine the two relations. The experiment varies one thing at a time: which transformation should preserve similarity. It shows why “more expressive” is not enough. The useful kernel reflects the mechanism through which cases share evidence.
Composition is a language of assumptions
Kernels can be added and multiplied. If and are valid kernels, then and are valid under ordinary conditions. Addition lets multiple structures contribute. Multiplication rewards joint similarity. Scaling changes their relative weight.
For a transaction, an additive kernel might combine customer-history similarity and merchant-network similarity. A product kernel says both must align. Those are different causal stories. Automatic search over compositions can help discover structure, but the resulting expression still needs a domain interpretation and a stability test. Otherwise composition becomes another hyperparameter factory.
Figure 4. Bandwidth changes the question before it changes the answer
Bandwidth is an epistemic control
At very narrow bandwidth, a radial model behaves like a collection of local memories. It can interpolate labels while having little support between observations. At very broad bandwidth, it approaches a nearly linear or constant response. Between those extremes lies a scale at which neighbourhood evidence may transfer.
Cross-validation can choose that scale for an evaluation distribution. It cannot prove that the distribution’s neighbourhoods match the future environment. Inspect performance across a band of plausible values. Plot the Gram matrix. Track the effective rank and the number of influential examples. Re-evaluate under unit changes, subgroup shifts and new operating regimes. A model that wins at one needle-thin setting carries sensitivity debt.
Read the matrix before trusting the score
The Gram matrix is observable before the labels are fitted, which makes it a useful diagnostic surface. Sort rows by known group, time or source and look for blocks. A strong block may reveal a meaningful population, or it may reveal duplicated records, acquisition batches or source artefacts. A bright diagonal with almost no off-diagonal mass indicates neighbourhoods too narrow to share evidence. A nearly uniform matrix indicates that the kernel has erased distinctions.
Eigenvalues show how similarity energy is distributed across directions. A few dominant values imply a low effective dimension; a long flat tail implies many weak directions. Regularisation suppresses directions whose small eigenvalues would otherwise require large, unstable coefficients. Plotting validation performance beside the eigenvalue spectrum can reveal whether a gain depends on weak directions likely to move under shift.
Effective rank is one summary, but inspect the corresponding cases. Which examples become support vectors? Which training points exert the largest influence on a consequential prediction? Do influential examples have verified labels and provenance? If one unusual record controls a region, removing it is a targeted stability test. Influence is not a causal explanation, but it tells the reviewer where the fitted relation is fragile.
Condition number matters operationally. A nearly singular kernel system can magnify numerical error. Adding improves conditioning and changes the learned function. Solver tolerances, floating-point precision and approximation therefore belong to the versioned artefact. Reproducing a kernel name and bandwidth is not enough if the numerical route changes the coefficients materially.
Similarity across structured objects
The kernel idea is not restricted to fixed-length vectors. A string kernel can count shared subsequences while discounting gaps. A tree kernel can compare substructures. A graph kernel can compare walks, neighbourhoods or spectral properties. A set kernel can compare distributions of members. These methods make structured relations available to a convex estimator without flattening everything into an arbitrary table.
The difficulty moves into validity. Shared subsequences may reflect authorship or boilerplate rather than meaning. Graph walks may be dominated by degree. Tree fragments may reward syntax while missing semantic equivalence. For each structured kernel, construct pairs that should be close and pairs that look superficially close but should remain separate. Pairwise challenge sets are often more revealing than end-task accuracy.
Positive semidefiniteness can also constrain intuitive similarities. A domain expert may propose a pairwise score that violates the kernel condition. One option is to redesign it as an explicit feature inner product. Another is to use methods built for indefinite similarity, accepting different optimisation properties. Silently clipping negative eigenvalues creates a new relation; evaluate the change rather than describing it as harmless repair.
Calibration and uncertainty remain estimator questions
A large margin is not a probability. Support-vector outputs can be calibrated on held-out data, but the calibration inherits that data and may shift. Gaussian processes express a posterior under their covariance, likelihood and approximation assumptions; sparse coverage can produce uncertainty, while model misspecification can still produce confidence in the wrong family.
When the decision needs probability, evaluate reliability and sharpness by region. Compare uncertainty with neighbour density, label noise and out-of-distribution tests. A kernel can report low influence far from training points, yet a constant mean or biased calibration may still yield an actionable-looking number. Build abstention from a validated uncertainty rule rather than distance alone.
The decisive test is behavioural. As support weakens, does the system ask for evidence, defer or reduce authority? If uncertainty is displayed but never changes a decision, it is explanatory furniture.
Scale changes the design
Exact kernels move work from explicit feature construction into relationships among training observations. That trade is attractive for modest datasets and expensive for large ones. Storing a dense Gram matrix requires entries. A direct solve can require time, although decomposition, sparsity, iterative solvers and problem structure can reduce practical cost. Prediction may also depend on many training examples.
Figure 5. Exact similarity accumulates a quadratic storage burden
The burden is not only runtime. Every retained training example can become part of the deployed predictor. Data deletion, provenance and privacy obligations may therefore affect model state. A request to forget one record can require more than deleting a row from a source table. The operational design should state whether examples, support vectors, inducing points or derived features persist.
Random features make the approximation explicit
For shift-invariant kernels, random Fourier features sample a finite transformation whose inner product approximates the kernel. The learner then trains a linear model on those features. If random features are used, storage and prediction can scale with rather than the number of training examples.
The approximation introduces another random object. Different feature draws produce different geometric error. Increasing tends to reduce that error at additional cost. The right evaluation therefore measures task performance and kernel approximation across seeds. Reporting only the best draw disguises the approximation as deterministic.
Nyström methods take another route. They approximate the Gram matrix using a subset of columns or landmark points. Their success depends on how those landmarks cover the matrix structure. Uniform sampling can miss rare but important regions; leverage-aware or stratified choices can help while adding estimation work.
Figure 6. Random features converge as a cloud, not a promise
Approximation is not merely a speed switch. It can regularise by removing weak directions, or harm by erasing a rare subgroup. Treat exact and approximate kernels as different models. Re-run calibration, subgroup, shift and failure tests. Preserve the approximation seed and basis as versioned model artefacts.
When deep representations enter
A learned representation can precede a kernel. The kernel then operates on embeddings rather than raw inputs. This hybrid may give a convex last-stage estimator, data-efficient adaptation or calibrated uncertainty. It also inherits every ambiguity of the representation. If the embedding collapses a protected distinction or moves after an upstream model update, a stable kernel cannot recover what disappeared.
Neural tangent kernels offer another connection. In a particular infinite-width and small-update regime, gradient-trained neural networks behave like kernel machines governed by their tangent features. The result clarifies one training regime; it does not reduce all finite network learning to a fixed kernel. Feature learning outside that regime remains a material difference.
The baselines that can win
Kernel methods are often compared with a weak linear rule and declared successful when the target is nonlinear. A serious baseline set is broader.
First, improve the transparent representation. Ratios, splines, interactions, domain distances and monotonic transformations may expose the structure directly. Second, test trees and boosting when local thresholds and heterogeneous interactions fit the task. Third, test nearest-neighbour methods when locality itself is the claim. Fourth, consider a compact neural model when representation learning across abundant data is essential. Fifth, test a constant or policy baseline to learn whether prediction changes the decision at all.
The comparison must hold information and budget steady. A kernel with extensive tuning should not face a default linear model. A deep model using external pretraining has received information the kernel did not. A tree with post-outcome features has received leakage. The model family is only one experimental factor.
Worked production example: irregular equipment traces
An engineering team must predict whether a pump needs inspection. Each case contains an irregular vibration trace collected under different rotational speeds. A standard feature table loses phase and alignment. A sequence kernel based on dynamic alignment gives a plausible relation: traces may be similar after small time warps.
The team begins with four baselines. A policy rule based on maximum vibration establishes operational value. A linear model on spectral summaries tests explicit features. Gradient-boosted trees test threshold interactions. One-dimensional convolution tests learned local patterns. The sequence kernel is evaluated with the same train and temporal holdout periods.
It wins on the historical holdout, but the result is not enough. A new sensor changes sampling density. Alignment cost rises and the learned scale becomes unstable. The convolution tolerates the change after resampling; the spectral model remains competitive. The team chooses the spectral model for the first release and keeps the kernel as an investigation tool. This is a kernel success in understanding and a kernel rejection in operation. Good evaluation allows that outcome.
stateDiagram-v2 [*] --> Claim state "Figure 1. Similarity bends the<br/>working geometry" as Claim state "Thought experiment: keep the<br/>points, rotate the meaning" as Test state "Figure 2. The Gram matrix is<br/>the model's relational memory" as Revision Claim --> Test: expose mechanism Test --> Decision: evidence survives Test --> Revision: rival survives Revision --> Claim: narrow or rebuild Decision --> [*]
A kernel decision record
The decision record below turns kernel enthusiasm into a sequence of falsifiable commitments.
Figure 7. Choose the kernel by invariance, not fashion
| Question | Required evidence | Stop condition |
|---|---|---|
| What relation should be preserved? | Domain examples of pairs that should and should not share influence | Experts cannot agree on pairwise meaning |
| What invariance follows? | A controlled transformation that should leave the output stable | The transformation changes the target mechanism |
| Can a simpler representation expose it? | Matched linear, tree, neighbour and policy baselines | A simpler model is within the decision tolerance |
| Is the scale stable? | Sensitivity across bandwidth, units, seeds and subgroups | Performance exists only in a narrow setting |
| Is the matrix affordable? | Memory, fit, prediction, deletion and retraining budgets | Cost violates the operating envelope |
| Does approximation preserve rare cases? | Exact versus approximate comparison by subgroup | Approximation erases consequential regions |
| Does the score change a decision? | Utility, calibration, abstention and recovery tests | No improvement over the policy baseline |
The record should live beside the model artefact. It records the similarity function, preprocessing, kernel parameters, regularisation, solver, training examples or basis, random seeds, validation splits, decision threshold and expiry conditions. These fields are part of the model, not incidental notebook state.
Where the method breaks
Three failures deserve separate names.
Semantic mismatch occurs when the kernel’s similarity does not follow the target mechanism. The model can be smooth, stable and wrong. Scale sensitivity occurs when small changes in units, bandwidth or density reorganise neighbourhoods. The model can look accurate and lack a durable operating region. Compute debt occurs when the relational representation cannot be stored, updated, deleted or served within the system’s obligations.
Figure 8. The failure surface has three independent axes
There is also a boundary no kernel crosses. A similarity function cannot create missing causal variables, repair a shifted label policy or authorise a consequential decision. It cannot tell whether a false negative is acceptable, whether a record may be used, or whether a prediction is fresh enough to act on. Those are surrounding measurement and governance questions.
A valid kernel can still be dangerous under selective labels. Suppose loan outcomes are observed only for approved applicants. Similarity among historical approvals says little about rejected cases without assumptions about selection. A beautifully regular function can reproduce the boundary of past decisions rather than the risk construct. The remedy is not a more exotic kernel. It is an evaluation design that models missing outcomes and policy effects.
Monitor relations, not only predictions
Deployment monitoring should follow the kernel’s mechanism. Track distances and similarity distributions between new and reference cases. Measure effective neighbourhood size, support-vector activation, leverage and kernel-matrix summaries on a rolling sample. A stable prediction distribution can hide a geometric shift when a downstream threshold absorbs it.
Pair monitoring with semantic checks. A change in units can inflate distance immediately. A new category code can create an artificial island. A source-system migration can duplicate records and produce dense similarity blocks. These faults are easier to diagnose in the relational representation than in aggregate accuracy months later.
Set responses. An unfamiliar region may trigger abstention, a broader evidence request or sampled review. A sudden collapse in effective rank may block release. A gradual drift may schedule recalibration. Store representative cases with privacy controls so reviewers can see which relation changed.
When labels arrive, test whether errors cluster by similarity region. If nearby cases receive conflicting outcomes, the kernel assumption may be too smooth or the labels may be unstable. If isolated cases fail, coverage rather than boundary shape may be the issue. The monitoring plan should help choose the next intervention, not merely announce that a metric moved.
Keep a small library of sentinel pairs whose relationship has clear domain meaning. Recompute their similarities after every preprocessing, feature or kernel change. A sentinel that suddenly becomes near or far can block release before end-task metrics conceal the semantic break.
Add at least one sentinel that ought to stay dissimilar despite sharing obvious surface features. It is a negative control for the geometry. If that pair converges while accuracy rises, the representation may be learning a shortcut rather than the relation the task needs. Record the expected direction, tolerance and owner beside each sentinel so the check remains operational after the original modeller leaves.
The strongest objection
Modern representation learning can discover useful similarity from large data rather than asking a practitioner to encode it. Fixed kernels may appear to freeze the very relation the model should learn. In vision, language and multimodal work, pretrained networks often dominate hand-selected kernels because their representations reuse enormous external evidence.
The objection is strong. The reply is bounded. Kernel methods remain useful when data is modest, relations have domain structure, convexity and uncertainty matter, or a learned representation needs a controlled final stage. They also serve as scientific instruments: a carefully chosen kernel can test whether a hypothesised invariance explains performance. The choice is empirical, not nostalgic.
sequenceDiagram participant Q as Question participant M as Mechanism participant E as Evidence participant D as Decision Q->>M: The straight line that became a circle M->>E: Figure 1. Similarity bends the working geometry E-->>M: compare rival explanations E->>D: Sources D-->>Q: revise scope or act
Sources
- Primary Cortes and Vapnik, Support-Vector Networks, 1995.
- Primary Schölkopf, Herbrich and Smola, A Generalized Representer Theorem, 2001.
- Primary Rahimi and Recht, Random Features for Large-Scale Kernel Machines, 2007.
- Primary Williams and Seeger, Using the Nyström Method to Speed Up Kernel Machines, 2001.
- Primary Rasmussen and Williams, Gaussian Processes for Machine Learning, 2006.
- Primary Jacot, Gabriel and Hongler, Neural Tangent Kernel, 2018.
- Reference Hofmann, Schölkopf and Smola, Kernel Methods in Machine Learning, 2008.
- Reference Smola and Schölkopf, A Tutorial on Support Vector Regression, 2004.