t-SNE and UMAP Visualization

Your manager sends a 784-dimensional digits dataset and asks for "a picture of what's in there." PCA gives you a scatter plot of the first two components — and the classes sit in overlapping wedges, because PCA preserves variance, not neighborhoods. t-SNE is the tool built for this exact job: produce a 2D map in which points that were neighbors in high dimensions remain neighbors, with essentially no other guarantee. Understanding what it optimizes is the difference between reading an embedding and being fooled by one.

t-SNE works in four steps. First, for every point x_i it places a Gaussian around it and measures the probability that x_j is drawn from it — the conditional similarity:

p_j|i = exp(-||x_i - x_j||^2 / (2 sigma_i^2)) / SUM_{k != i} exp(-||x_i - x_k||^2 / (2 sigma_i^2))

Second, it symmetrizes: p_ij = (p_j|i + p_i|j) / (2n), fixing the asymmetry so every pair has one shared probability. Third, in the 2D map it uses a Student-t distribution with one degree of freedom instead of a Gaussian: q_ij = (1 + ||y_i - y_j||^2)^-1 / SUM. The heavy tails are deliberate — in two dimensions, a Gaussian's probabilities for mid-distance points shrink so much that the optimizer has no gradient room to separate them (the "crowding problem"); the t-distribution's fatter tails give those points somewhere to go. Fourth, it descends the gradient of KL(P || Q), matching the two probability distributions.

Perplexity is the only hyperparameter that matters, and it is not a mystery: it is the effective number of neighbors each point considers. For each point, sigma_i is found by binary search such that the entropy of P_i satisfies H(P_i) = log2(perplexity) — larger perplexity forces a wider Gaussian, pulling more distant points into the similarity distribution. Everything else follows from it:

Run it yourself and watch the two properties that matter — structure is stable, coordinates are not:

from sklearn.manifold import TSNE
from sklearn.datasets import load_digits

X, y = load_digits(return_X_y=True)

for seed in [0, 42]:
    emb = TSNE(n_components=2, perplexity=30, init="pca",
               learning_rate="auto", random_state=seed).fit_transform(X)
    print(seed, emb[:3])       # completely different coordinates, same neighbor graph

Two seeds give maps that look nothing alike — rotations, reflections, even cluster spacing differ — while the neighborhoods agree. That is expected: t-SNE preserves local probability structure, which has no canonical orientation, so the axes carry no meaning and you must never read coordinates off the plot. Its O(n^2) pairwise-distance cost (Barnes-Hut approximation: O(n log n), practical to ~50k points) is the other reason nobody promises a canonical layout.

Now the trap that ends up in interview postmortems: clusters in a t-SNE plot are hypotheses, not findings. KL(P || Q) is asymmetric — it punishes separating true neighbors (p large, q small) far more than creating extra neighbors — so the optimizer happily invents white space between groups, equalizes apparent densities, and can render one continuous Gaussian blob as several islands. "We found 8 clusters in t-SNE, so the data has 8 clusters" is a claim you cannot make from this plot; verify in the original space (silhouette, stability across seeds and perplexities, or a downstream probe). If the structure survives three perplexities and two seeds, you have evidence. That discipline — local neighborhoods trustworthy, cluster existence provisional, global distances meaningless — is exactly what the UMAP chapter extends, and the reason both belong after the feature-level thinking of Feature Selection Methods: scale and denoise (standardize, then PCA to ~50 dims) before embedding, or your 2D map will faithfully visualize the noise.

t-SNE answers "show me the neighborhoods," but it scales awkwardly beyond ~50,000 points and its global arrangement is close to arbitrary. UMAP (Uniform Manifold Approximation and Projection) is the tool most teams reach for when the dataset is bigger or when the rough global layout should also mean something. Mechanistically it differs from t-SNE — it builds a fuzzy k-nearest-neighbor graph in high dimensions, then optimizes a 2D layout whose fuzzy graph matches it — but practically you drive it through a handful of hyperparameters whose effects you should be able to predict before running anything.

The high-dimensional side connects every point to its n_neighbors nearest neighbors with weights that decay with distance (the actual math: a fuzzy simplicial set built from locally adaptive radii, so each point's neighborhood scale adapts to local density). The low-dimensional side packs points using an exponential-family kernel whose minimum separation is bounded by min_dist. Two knobs, two questions: how local is "local", and how tightly may points clump inside a cluster.

import umap  # pip install umap-learn
from sklearn.datasets import load_digits

X, y = load_digits(return_X_y=True)

reducer = umap.UMAP(
    n_neighbors=15,        # local vs global structure trade-off
    min_dist=0.1,          # how tightly points pack inside a cluster
    n_components=2,        # output dimensions
    metric="euclidean",    # neighborhood definition
    random_state=42,       # determinism (forces single-threaded)
)
emb = reducer.fit_transform(X)     # 1797 x 2; use reducer.transform() for NEW points

The hyperparameter map, as an interview-ready table:

The most-tuned pair is n_neighbors and min_dist. n_neighbors=2 makes the graph so local that every point lands in a tiny isolated group; n_neighbors=200 averages neighborhoods so aggressively that distinct clusters merge into one continent. min_dist=0 renders clusters as crisp dots — beautiful for a slide, but distances within a cluster become meaningless; min_dist=0.8 keeps a natural scatter. The metric choice is semantic, not cosmetic: for normalized text or sentence embeddings, cosine measures the right thing and euclidean quietly measures length instead. And one UMAP-specific lever t-SNE lacks: supervised mode. Passing the labels (UMAP(..., y=y)) pulls same-class neighbors together during layout, sharpening class geometry for visualization or pretraining — with the obvious caveat that you can no longer present the plot as unsupervised evidence of class separation; you fed the answer in.

Why UMAP handles datasets where t-SNE chokes: t-SNE needs all n^2 pairs (or a Barnes-Hut tree at O(n log n) with heavy constants), while UMAP's neighbor-graph construction uses nearest-neighbor descent that scales roughly linearly in n. In practice t-SNE tops out around 50k points on a workstation; UMAP routinely plots a million. UMAP also tends to preserve more global arrangement — relative positions and sizes of distant groups come out more faithful — though the honest phrasing is "empirically better, not guaranteed," and a small n_neighbors degrades global fidelity by design.

The 80% workflow, worth memorizing verbatim:

from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
import umap

Xs = StandardScaler().fit_transform(X)
Xp = PCA(n_components=50, random_state=0).fit_transform(Xs)   # denoise + speed
emb = umap.UMAP(n_neighbors=15, min_dist=0.1, random_state=42,
                metric="cosine").fit_transform(Xp)

PCA-to-50 first is standard practice: it removes noise directions the neighbor graph would otherwise chase, and makes the expensive step cheaper. Then sweep n_neighbors over {5, 15, 50} and keep only structure that persists across the sweep — same stability discipline as t-SNE, one chapter prior. The final discipline, reading embeddings without lying to yourself, is the subject of the next chapter — and the last stop in this phase before Perceptron and Multilayer Networks.

An interviewer shows you a t-SNE plot with eight well-separated blobs and asks: "So the data has eight clusters — agree or disagree?" This is not a trick question; it is the industry's favorite test of whether you understand what 2D embeddings actually promise. The correct answer: disagree — the plot is a hypothesis generator, and every claim on it needs verification back in the original space.

It helps to have the trust list organized before you walk in:

Three known illusions produce most bad readings. Cluster multiplication: t-SNE's per-point adaptive sigma and its KL asymmetry let one continuous Gaussian render as several islands; UMAP with n_neighbors=5 does the same to any smooth manifold. Density equalization: a 30-point class and a 3,000-point class occupy similar area, so you cannot compare prevalence from the plot. The size illusion: because min_dist controls packing (previous chapter), apparent cluster size reflects a hyperparameter, not data volume. Any slide that argues "segment B is bigger than segment A" from an embedding is wrong no matter how good the plot looks.

So what converts a pretty picture into evidence? Three practices:

1. Verify structure in the original space. Labels you "found" in 2D: check cluster silhouette and class separation on the pre-embedding representation (ideally the 50-dim PCA space, not the 784 raw features); check that a simple probe (logistic regression on the same representation) agrees the groups are separable.

2. Quantify embedding fidelity. Trustworthiness, from sklearn, scores how many of the 2D neighbors were genuine high-dimensional neighbors, penalizing "intrusions" (points placed next to strangers):

from sklearn.manifold import trustworthiness
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA

Xs = PCA(n_components=50).fit_transform(StandardScaler().fit_transform(X))
for perp in [5, 30, 100]:
    emb = TSNE(perplexity=perp, random_state=0).fit_transform(Xs)
    print(perp, round(trustworthiness(Xs, emb, n_neighbors=12), 3))

Pair it with stability: rerun with two or three seeds and two perplexities (or n_neighbors values) and keep only the structure that survives all of them. Real geometry persists; optimizer artifacts move around.

3. Never ship the map as a feature. t-SNE has no transform for unseen points — its optimization is per-dataset, so there is literally no way to embed a new user's vector into an existing map (UMAP does have .transform, but out-of-sample points land by interpolation of the existing layout, with its own distortions). And a 2D coordinate pair throws away nearly all information the model could use. If you want embedding features for a downstream model, use the 5-50 dim UMAP output — validated for lift — or skip embeddings and feed the PCA representation to a regularized model. The 2D map exists for human eyes and slide decks.

The reporting standard that keeps you honest: fix random_state, state the metric, show more than one hyperparameter setting, and phrase findings as "consistent evidence of k groups" only after the original-space checks pass. With that, you can close any interview loop that starts with "explain this plot": what t-SNE optimizes (KL between neighbor distributions, perplexity as effective neighbor count), what UMAP changes (fuzzy k-NN graph, n_neighbors/min_dist, linear scaling, supervised mode), and what neither of them buys you — a license to read global distances off a screen. Those same neighborhood ideas return, in a trainable form, in the next topic, Perceptron and Multilayer Networks, and the preprocessing discipline here extends the leakage lessons from Feature Selection Methods.