Supervised vs Unsupervised Learning

Your support organization has 50,000 historical tickets, each tagged by a human agent with one of twelve categories. Leadership wants those tags assigned automatically the moment a ticket arrives. Every historical example already pairs an input (the ticket text) with a desired output (the tag). That pairing is the whole definition of supervised learning: you fit a model to labeled examples, then ask it to predict outcomes on data it has never seen.

Formally you are learning a function f from a dataset of pairs (x1, y1) through (xn, yn) so that f(x) is close to y on new draws from the same population. The learning algorithm searches a hypothesis space (a linear model, a tree ensemble, a neural network) for the parameters that minimize a loss measured on the labeled training set. What you are buying is a transfer of human judgment: a person labeled 50,000 tickets once, and the model reproduces that judgment at request time for free.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline

X_train = ['refund delayed three weeks', 'cannot log in to portal']
y_train = ['billing', 'access']

model = make_pipeline(TfidfVectorizer(min_df=5), LogisticRegression(max_iter=1000))
model.fit(X_train, y_train)                     # fit sees both X and y
print(model.predict(['where is my invoice?']))  # ['billing']

Two families cover most supervised work: regression, when y is a continuous number, and classification, when y is a discrete label. Regression vs Classification Targets takes that split apart, and the decision comes from the shape of y, not from the model you happen to like. Spam detection, fraud scoring, demand forecasting, medical diagnosis, and price estimation are all supervised; what they share is a labeled history and a decision you want made on the next input.

The label column is also where projects die. Tagging 200,000 support tickets at 30 seconds each is roughly 1,700 person-hours, so real teams reach for cheaper label sources: an existing operational trace (the user clicked the spam button), a rule written by a domain expert (orders over $10,000 are flagged for review), a paid annotation vendor, or a small hand-labeled set expanded through pseudo-labeling. Whatever the source, measure inter-annotator agreement before training. If two humans disagree on 20% of tickets, your model cannot score above that ceiling, and no architecture change will fix it.

Supervised learning also carries two assumptions you should say out loud in an interview. Training data must be drawn from roughly the same distribution as deployment data, and labels must be correct enough to learn from. Violate the first and you get a model that scores well offline and fails in production once traffic shifts. Violate the second and the model faithfully learns the labeling noise, including the shortcuts the annotators used when they were tired.

A question interviewers like: with unlimited budget, would you always choose supervised learning? The honest answer is that supervised learning wins whenever labels exist and match the deployment decision, but label acquisition, drift, and the ongoing maintenance of the label pipeline belong in the trade-off. A perfectly accurate model trained on last year's labels can still be the wrong system for this quarter's traffic.

Now change the situation. You have 3 million users, their purchase histories, and no segment definitions anywhere in the company. The CEO asks: who are our customers? There is no label column because nobody has ever written one down. This is unsupervised learning, and it does not fit a function from inputs to outputs. It searches for structure in the data itself: groups, densities, low-dimensional layouts, or associations that were implicit all along.

The workhorse methods split into three jobs. Clustering assigns each point to a group: k-means minimizes within-cluster variance around k centroids, DBSCAN declares a cluster wherever points pack together above a density threshold, and hierarchical clustering builds a dendrogram you can cut at any granularity. Density and outlier methods flag points that sit where nothing else does, which is how anomaly detection on transactions and server metrics is usually framed. Projection compresses many correlated features into a few components so you can plot and inspect the data (PCA), or lay it out for human eyes (t-SNE, UMAP).

from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

# rows = users scaled to zero mean and unit variance
X = scaled_features                # shape (n_users, n_features)
km = KMeans(n_clusters=5, n_init=10, random_state=42).fit(X)
print(km.labels_[:10])             # [2 0 0 3 1 ...]
print(silhouette_score(X, km.labels_))   # -1.0 to 1.0, higher is tighter

The painful difference from supervised work is evaluation. With labels you report accuracy or MAE and move on. Without labels there is no ground truth, so you validate structure three ways: an internal metric (silhouette score, Davies-Bouldin index, DBSCAN's count of noise points), stability (rerun on a bootstrap sample and check that the same groups reappear), and domain inspection (export twenty users per cluster and ask whether a marketer recognizes them as distinct segments). Clusters that score well on inertia but mean nothing to the business are useless; elbow curves are a hint, not proof.

This is also where the classic interview question lands: how do you know k equals 5? The strong answer has three parts. You cannot know from the data alone; you use the elbow method on inertia and the silhouette score to narrow the range, then let downstream utility decide (do the segments change marketing decisions, and does a supervised churn model trained on cluster features beat the baseline?).

Choosing among the unsupervised families themselves follows the data geometry. K-means assumes roughly spherical, similarly sized clusters and needs k declared up front, which makes it fast but fragile on elongated groups. DBSCAN needs no k and recovers arbitrarily shaped clusters while marking sparse points as noise, but its two parameters (neighborhood radius and minimum points) have to be set with knowledge of the feature scale. Hierarchical clustering trades speed for a dendrogram that shows how groups nest, which suits taxonomy-style business questions but scales poorly past tens of thousands of rows.

Unsupervised methods also feed supervised ones. You can cluster customers first and hand the cluster id to a churn model as a feature, or fit PCA to collapse 300 correlated sensor readings into 20 components before training a classifier. The unsupervised step discovers structure; the supervised step still needs labels to learn a decision.

Most projects that fail at the learning-type decision fail upstream of any code. The stakeholder said cluster our users, so the team clusters users, and three weeks later nobody can connect the segments to a business action. Choosing the wrong learning type usually stems from framing the business question incorrectly, and the fix is a translation step you run before touching a notebook.

The translation has two moves. First, write the target variable: what exactly will the model output for one input? Second, ask whether labeled examples of that target already exist or can be created at reasonable cost. The answer picks the learning type.

When labels exist for only a sliver of the data, semi-supervised learning is the middle road: train on the labeled subset, predict confident labels on the unlabeled mass, and retrain on the combined set. Self-supervised learning pushes this further by manufacturing labels from the data itself (predict a masked word, predict the next frame), which is how large language and vision models are pre-trained before any human annotation touches them. Both variants sit on the same spectrum: they differ only in where the training signal comes from, not in whether a target variable exists. If you cannot name the target at all, you are unsupervised; if you can name it but lack full labels, you are semi-supervised or self-supervised.

Framing errors have recognizable mid-project symptoms, and spotting them early saves weeks. If your clustering demo keeps producing segments nobody can act on, the underlying question was probably a supervised one (which users will churn) that was misquoted as an exploratory one. If a classification project stalls because the label column keeps changing definition between annotators, the target variable was never pinned down. If a regression model predicts an average of everything, the target may have discrete levels that belong in an ordinal or classification formulation. Each of these is a framing bug: the algorithm was never the problem.

Real pipelines also mix the two legitimately. A fraud system might fit an autoencoder on normal transactions (unsupervised) and use its reconstruction error as one feature in a supervised gradient-boosted classifier trained on confirmed fraud cases. An NLP pipeline might fit TF-IDF or topic models across all documents (unsupervised) and feed those into a supervised intent classifier. Keep the boundary crisp during evaluation: anything fit on the full dataset, including PCA and scalers, must be fit inside the training fold only, or the test set leaks into training.

A useful interview script for this topic: restate the business question, name the target variable, state where the labels come from and what they cost, then name the learning type and its evaluation metric. That sequence shows the interviewer you frame problems before you reach for algorithms, which is exactly the skill this phase of the roadmap is building. Next you will pin down the shape of the target itself in Regression vs Classification Targets.