Python ML Data Stack

Every classical ML library you will use speaks one data language underneath: the NumPy ndarray. A DataFrame becomes .values before scikit-learn touches it, an image batch becomes a float32 tensor of shape (batch, channels, height, width), and a text dataset becomes a sparse matrix of TF-IDF weights. Understanding why that array exists, and why loops over it are the cardinal sin of scientific Python, makes the rest of the stack legible.

An ndarray is a homogeneous block of typed memory plus three attributes: shape, dtype, and strides. Homogeneous means every element has the same size, which lets the machine execute one compiled operation over millions of elements with a tight loop written in C, indexed by arithmetic on strides. A Python list of floats is instead a block of pointers to scattered PyObject boxes, each requiring type checks and reference counting per access. The practical difference is two to four orders of magnitude:

import numpy as np
import time

a = np.random.default_rng(0).standard_normal(1_000_000)

start = time.perf_counter()
b1 = a * 2 + 1                      # one vectorized pass
print('vectorized', time.perf_counter() - start)   # ~0.003 s

lst = a.tolist()
start = time.perf_counter()
b2 = [x * 2 + 1 for x in lst]       # Python-level loop
print('list comp', time.perf_counter() - start)    # ~0.05 s, still 15x slower
# an explicit for loop with indexing would be slower still

Broadcasting is the rule that lets expressions like a * 2 + 1 work without allocating scalars, and it generalizes: when two arrays have mismatched shapes, NumPy compares shapes from the trailing dimension and stretches any dimension of size 1. That means an (n, d) feature matrix minus a (d,) mean vector yields centered features in one expression, with zero Python-level work per row. Predicting the result shape before you run it is a genuine interview skill: (n, d) times (d, 1) broadcasts to (n, d), while (n, d) times (k, d) fails unless k equals n or 1.

Randomness deserves its own habit. Use the new numpy.random.default_rng(seed) API, pass the Generator everywhere reproducibility matters, and remember that model seeds in scikit-learn do not seed NumPy: a pipeline that seeds LogisticRegression but not the train_test_split shuffle is not reproducible. dtype discipline matters too: float64 by default doubles memory and bandwidth against float32 with no accuracy gain for most features; storing categories as integers with a mapping beats object arrays of strings for both memory and speed. Indexing deserves a mention as well: NumPy slices are views into the same buffer, so b = a[:100] then b *= 0 mutates the original, while pandas .loc assignment and boolean masks return copies by default. Knowing which operations share memory is what keeps debugging sane.

Where NumPy stops, the next layer begins. Rows of real business data arrive with mixed types, missing values, and labels, which is exactly the territory pandas was built for. The two interoperate at the boundary: df.to_numpy() crosses to NumPy for modeling, and pd.DataFrame(array, columns=...) crosses back for reporting, and both crossings copy, which you should do once per pipeline stage rather than per expression.

A DataFrame is a labeled 2-D table where each column is a typed Series, and the four operations that consume most real project time are inspection, missing-data handling, joins, and group-by aggregation. The rest of pandas is variations on these.

Inspection is df.info() (dtypes and null counts per column), df.describe() (numeric summary), and df.head(). The single most important habit: check whether numeric columns silently hold strings. A CSV where 3% of the amount column contains 'N/A' parses as object dtype, your regression then fails (or worse, sorts lexicographically) deep inside the pipeline, and the fix was one pd.to_numeric(col, errors='coerce') at ingestion.

import pandas as pd
import numpy as np

df = pd.read_csv('orders.csv', parse_dates=['ts'])
print(df.dtypes.value_counts())                    # spot object columns early

# missing data: count, then decide per column, never silently
print(df.isna().mean().sort_values(ascending=False).head())

median_amt = df['amount'].median()
df['amount'] = df['amount'].fillna(median_amt)      # impute with train data in prod

df = df.assign(log_amount=np.log1p(df['amount']))

# group-by: split-apply-combine, any aggregation pandas knows
weekly = (df.groupby(df['ts'].dt.isocalendar().week)
            .agg(total=('amount', 'sum'),
                 n=('amount', 'size'),
                 avg=('amount', 'mean')))

# joins: validate cardinality instead of trusting the docs
enriched = df.merge(users, on='user_id', how='left', validate='many_to_one')

Group-by is pandas' split-apply-combine: it groups rows by key, applies an aggregation per group, and recombines into a table. The named-aggregation form shown above (agg(total=('amount', 'sum'))) keeps output columns explicit, which matters when the result feeds a feature pipeline. Two traps live here. Grouping by a column that includes NaN silently drops those rows unless dropna=False, and computing a group mean over the whole dataset before splitting train/test leaks the validation distribution into training features; in production code, aggregate on the training window only and join the result back.

Time-aware columns close many cleaning loops. parse_dates at read time turns string timestamps into datetime64, after which .dt accessor methods (dt.hour, dt.dayofweek, dt.isocalendar().week) generate calendar features in vectorized form instead of per-row Python. Filtering a time window becomes an unambiguous comparison on datetime64 rather than a string sort that breaks across year boundaries ('2024-12-31' sorts after '2025-01-01' as text).

Joins are where data engineering bugs enter ML tables. validate='many_to_one' turns a silent fan-out duplication (each order joined to N user rows, doubling your dataset and your accuracy estimate) into a loud error. Always eyeball len(enriched) against len(df) after a merge, and prefer how='left' from the table whose row count must survive, so a missing match shows up as NaN in the joined columns rather than silently shrinking the dataset the way an inner join does.

Finally, avoid chained assignment (df[df.x > 0]['y'] = 1), which either raises SettingWithCopyWarning or does nothing; use df.loc[mask, 'y'] = 1 instead, which assigns on a single labeled index operation and reports how many rows were touched. And know when to leave pandas: once a column-wise Python-level loop appears in a hot path (string parsing per row), either vectorize with .str accessor methods or move the step to NumPy array space, because pandas rows iterate at Python speed. Crossing to NumPy for modeling is one .to_numpy() call, and the matrix that comes out is the subject of the next phases of this roadmap.

scikit-learn's real contribution is not any single algorithm; it is a uniform object contract that lets you swap model families without rewriting the pipeline around them. Every estimator fits on data and exposes learned parameters, every transformer additionally transforms data, and every supervised model answers predict (and usually predict_proba). Because the contract is uniform, cross-validation, grid search, and pipelines compose with any estimator you drop in.

The contract in practice: fit(X, y) learns parameters and returns self; predict(X) produces outputs; transform(X) (transformers only) produces a new matrix; predict_proba(X) (where available) returns class probabilities in model.classes_ order. Two rules keep the contract honest. fit must never see validation or test rows, and the same fitted object must serve both training-time transforms and inference-time transforms, which is what prevents a model trained on scaled features from receiving unscaled ones at serving time. X is always a 2-D array-like of shape (n_samples, n_features), and y a 1-D array of length n_samples (or 2-D for multi-output), a shape discipline that surfaces in every error message you will ever read from the library.

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.model_selection import cross_val_score

num = Pipeline([('impute', SimpleImputer(strategy='median')),
                ('scale', StandardScaler())])
cat = Pipeline([('impute', SimpleImputer(strategy='most_frequent')),
                ('oh', OneHotEncoder(handle_unknown='ignore'))])

prep = ColumnTransformer([('num', num, ['age', 'amount']),
                          ('cat', cat, ['country', 'channel'])])
model = Pipeline([('prep', prep),
                  ('clf', HistGradientBoostingClassifier())])

scores = cross_val_score(model, X_train, y_train, cv=5, scoring='roc_auc')
model.fit(X_train, y_train)                    # whole pipeline fitted once
model.predict_proba(X_test)[:, 1]              # preprocessing applied for you

The Pipeline earns its place by making leakage structurally impossible. When preprocessing lives inside the pipeline, cross-validation refits the scaler, imputer, and encoder per fold automatically. Outside the pipeline, the classic beginner bug fits a StandardScaler on all data before splitting, quietly leaking each validation fold's statistics into training; your scores improve by a point or two and production suffers. Grid search stays consistent with the same idea: GridSearchCV(model, {'clf__max_depth': [3, 5], 'prep__num__scale__with_mean': [True]}) addresses steps by step__param names, and refits the winning configuration on the full training set at the end.

Contract details that surface in interviews and in bugs: fit mutates the estimator and returns self, so best = grid.fit(...) and grid.fit(...); best = grid are the same object; transform must be pure with respect to the fitted state; get_params() and set_params() underpin every search; and classes appear in predict_proba columns ordered by classes_, which is not the numeric order of your encoded labels unless you arranged it. Sparse matrices from OneHotEncoder or TF-IDF stay sparse through the pipeline, and some estimators (k-means, PCA with centering, any model calling .toarray()) force densification, which is a memory decision worth checking on wide categorical data before you ship.

One more uniformity worth knowing for interviews: cloning. GridSearchCV and cross_val_score do not reuse your estimator object across folds; they clone it via sklearn.base.clone, which constructs a fresh unfitted copy from get_params(). That is why a pipeline defined once serves both cross-validation and final fitting, and why learned state never leaks between folds through the estimator object itself.

With the stack in place, the roadmap moves to the structures these libraries pass between them: Scalars, Vectors, and Matrices.