Class Imbalance Remedies

Your dataset has 1,000 rows: 950 legitimate transactions, 50 fraudulent. You train a logistic classifier, and it predictably learns that "legit" is almost always right. The fix is not to touch the data — it is to change what a mistake costs during training. Class weights do exactly that: the loss becomes a weighted sum where misclassifying the minority class is charged more.

scikit-learn's class_weight='balanced' computes the weight for class k as w_k = n / (K × n_k) — total rows divided by the number of classes times that class's size:

n = 1000, K = 2
w_legit = 1000 / (2 × 950) = 0.526
w_fraud = 1000 / (2 ×  50) = 10.0      # ratio 19:1

# equivalently, by hand in Python
from sklearn.utils.class_weight import compute_class_weight
import numpy as np
compute_class_weight("balanced", classes=np.array([0, 1]), y=y_train)
# array([0.526, 10.0])

What does the 19:1 ratio buy? During gradient descent, each fraud row's loss term is multiplied by 10, so its gradient pushes the boundary hard while each legit row nudges with weight 0.526. The optimizer now needs the model to be genuinely wrong on many legit rows before a fraud miss stops being worthwhile — the decision boundary slides toward the majority class, and fraud recall climbs. You can verify the intuition: with class_weight='balanced', the model starts flagging transactions it would previously have cleared, which is precisely what you wanted and precisely what threshold tuning would have done from the other direction.

That last observation is the deep connection to ROC Curve and Threshold Selection, and it is worth deriving because interviewers love it. Suppose the true posterior is p and you weight the positive class by w. The weighted loss is minimized by a model whose output is q = wp / (wp + (1 − p)) — the weighted probability, not the true one. Predict positive when q > 0.5, which rearranges to p > 1 / (1 + w). With w = 19, the effective threshold on the true probability is 1/20 = 0.05 — exactly the cost-optimal threshold for C_FN/C_FP = 19. So class weighting and threshold tuning are two routes to the same operating point: weighting changes the model, the threshold changes the decision rule on a fixed model.

The costs of the weighting route are real, though. First, the probabilities come out miscalibrated: q is warped away from p by construction, so predict_proba from a weighted model over-reports the minority probability and must be recalibrated (Platt or isotonic on a validation split) before anyone reads it as a chance. Second, weighting does not add information — 50 fraud rows are still 50 distinct patterns, so if those rows are noisy or narrow, no weight fixes coverage; you may still need resampling (next chapter). Third, weights are tunable, not sacred: 'balanced' is a starting point, and for custom cost ratios you pass a dict, or per-row costs via sample_weight in fit() — the right hook when the cost depends on the transaction amount rather than the class:

model = LogisticRegression(class_weight={0: 1.0, 1: 19.0}, max_iter=1000)
# or per-row costs, e.g. proportional to transaction amount
model.fit(X_train, y_train, sample_weight=per_row_weights)

Use 'balanced' when you want a defensible default, and the cost matrix when the business can state what a miss and an alarm are actually worth. The same knob exists across the toolbox — RandomForest, SVM, and XGBoost all accept class_weight or per-row sample_weight — so treat the weight ratio as a hyperparameter to cross-validate rather than a constant handed down from the formula: grid-search ratios around balanced (say 5, 10, 19, 40) and score by PR AUC or F1, because the theoretically balanced ratio is optimal only if the loss and the deployment costs actually agree.

Weights change the loss; resampling changes the data. All three standard techniques are quick to implement and each has a characteristic way to hurt you.

Random oversampling duplicates minority rows until the classes balance. It costs nothing and leaks nothing, but the duplicates add zero information: gradient-based models simply see the same 50 rows more often, and a decision tree that splits on them can reach pure leaves over duplicated points and memorize them. Expected symptom — training metrics jump, validation metrics barely move.

Random undersampling drops majority rows. It is fast, it genuinely decorrelates redundant negatives, and it throws away information: a model trained on 50 versus 50 never sees the diversity of the 900 discarded legit rows, so variance climbs and borderline cases get misjudged. On large datasets where the majority is repetitive, it can still be the cheapest effective fix.

SMOTE (Synthetic Minority Over-sampling Technique) targets the gap between duplicating and discarding: it invents new minority examples by interpolation. For each minority row x, find its k nearest minority neighbors (default k = 5), pick one at random, pick λ ~ Uniform(0, 1), and create:

x_new = x + λ × (x_neighbor − x)
# x = (2, 3), neighbor = (4, 1), λ = 0.5  ->  x_new = (3, 2)

That synthetic point lies on the segment between two real minority points, so the decision boundary gets gradient signal in territory that previously held only real examples. In practice, SMOTE plus a logistic model or a tree typically lifts minority recall more than plain duplication, because it thickens the minority region instead of stacking points on top of each other.

Now the traps, which are the part interviews actually care about.

# WRONG: resample before splitting -> synthetic copies of test rows leak into training
from imblearn.over_sampling import SMOTE
from imblearn.pipeline import make_pipeline as imb_make_pipeline
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier

pipe = imb_make_pipeline(
    SMOTE(k_neighbors=5, random_state=42),     # runs inside each CV fold only
    RandomForestClassifier(n_estimators=300, class_weight="balanced", random_state=42),
)
cross_val_score(pipe, X, y, cv=5, scoring="average_precision")

Trap one, leakage: if you SMOTE the whole dataset and then split, synthetic rows built from points near validation examples sit in training, and your score is fiction. Use an imblearn Pipeline so resampling executes inside each fold only; scikit-learn's plain Pipeline does not know how to resample. Trap two, smearing across boundaries: interpolation assumes the segment between two minority points stays minority, which is false when the minority has two disconnected subclusters or sits beside the majority — SMOTE then manufactures mislabeled points in the majority's territory. Clean afterwards (Tomek links or EditedNearestNeighbours remove borderline mess), or use SMOTE with a smaller k. Trap three, categorical features: interpolating one-hot columns produces fractional categories that mean nothing — use SMOTENC, which interpolates continuous features and picks neighbors' categories outright. Trap four, the evaluation set: validation and test data must stay at their natural distribution, always, because that is the world the model will face.

A practical recipe: start with class weights (no synthetic data, no leakage surface), escalate to random oversampling, then SMOTE + cleaning when coverage — not just loss balance — is the problem. If you reach for cleaning, know what the two standard tools do: Tomek links find a minority/majority pair that are each other's nearest neighbors and remove (or relabel) the majority side — they sharpen a blurred boundary rather than deleting bulk data; EditedNearestNeighbours drops any majority row whose label disagrees with the majority of its k neighbors, deleting whole noisy regions. Undersamplers such as RandomUnderSampler(strategy='majority') and combined pipelines (SMOTETomek, SMOTEENN) compose these steps, and every one of them belongs on the training side of the split — the evaluation rules below never bend.

A stakeholder shows you a dashboard: the churn model is 95% accurate. You ask one question — what would a constant "no churn" prediction score? It is also 95%. On a dataset that is 95% negative, accuracy certifies a model that never finds a single positive, and this is the environment in which every remedy from this topic earns its keep. The lazy model's confusion matrix is 950 TN, 0 FP, 50 FN, 0 TP: recall 0, precision undefined, and a product that flags nobody.

Start the fix at evaluation, because a metric that cannot see the failure will let every later mistake through. Three rules. Rule 1: stratified splits. With 50 positives, a random 80/20 split can strand 40 in training and 10 in test; train_test_split(..., stratify=y) and StratifiedKFold keep the positive rate stable across folds so your recall estimate does not depend on luck. Rule 2: leave validation and test at the natural distribution — resampling and weighting belong to training only. Synthetic fraud in the test set is like an exam whose questions you wrote. Rule 3: pick metrics whose denominator is the minority class. Recall, precision, F1, PR AUC, or a constraint like "recall ≥ 0.8 at precision ≥ 0.5"; accuracy and ROC AUC alone will understate the problem (see ROC Curve and Threshold Selection for the FPR-denominator arithmetic).

Then assemble the training pipeline in the order that composes cleanly:

from sklearn.model_selection import train_test_split, StratifiedKFold, GridSearchCV
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import average_precision_score, classification_report

X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2,
                                           stratify=y, random_state=42)
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
grid = GridSearchCV(
    LogisticRegression(max_iter=2000),
    {"class_weight": [{0: 1, 1: r} for r in (5, 10, 19, 40)]},
    scoring="average_precision", cv=cv)
grid.fit(X_tr, y_tr)
# validate on natural-distribution X_tr via CV; test ONCE on X_te at the end

Notice scoring="average_precision" in the grid: it ranks examples by score (threshold-free), its random baseline is the positive rate (0.05 here, not 0.5), and it cannot be gamed by predicting the majority class — which is exactly what makes it safe as an objective. Related deployment pattern: many imbalance problems are really top-k problems. A fraud team reviews the 200 highest-scoring transactions a day, so what matters operationally is precision@200 and recall@200, not performance at any probability cutoff; compute the confusion matrix at exactly k flagged rows and you have a number the business can staff against.

Here is the subtlety that separates competent from convincing work: training with weights and evaluating at threshold 0.5 are two different decisions. The weighting route already moved your effective threshold to roughly 1/(1+w); you can and should still calibrate the weighted model's probabilities and tune the final cutoff on validation against the actual cost ratio, then freeze it. Common overcorrections to avoid: aggressive oversampling until the classes are 50/50 (you have changed the training prior so far from deployment that probabilities become fiction and boundaries drift), accuracy as the tuning score (it will select the lazy model), and resampling before the train/test split (leakage, as covered above).

One bridge to keep in mind going forward: these remedies are model-agnostic, and the next phase's learners have their own relationship with imbalance — a decision tree chooses splits by impurity reduction weighted by sample counts, and its leaves predict by majority vote, so an unconstrained tree on 95:5 data grows leaves that all say "no churn". Sample weights flow into trees through the same sample_weight/class_weight path, which is why Decision Tree Learning starts from purity rather than from metrics: the tree's objective is where the bias enters, and pruning — the topic after — is where part of it gets corrected.