Feature Engineering Recipes

Plot your revenue, claim_amount, or followers column and you will usually see a wall of small values with a thin tail stretching far right. Ordinary least squares treats a $50,000 residual as 1,000 times the problem of a $50 one; when a handful of rows live in the tail, the tail quietly owns the fit. A log transform fixes the asymmetry, and it fixes the story: on log-revenue, a doubling costs the same at every scale, which is how these quantities actually behave.

The transform you want is log1p — log(1 + x) — not log:

import numpy as np
import pandas as pd

df['log_revenue'] = np.log1p(df['revenue'])   # defined at 0; log(0) is not
df['log_revenue'].describe()
# mean 8.2   std 1.9   skew 0.4     <- was: mean 5400  std 41000  skew 7.8

Two properties matter downstream. The residual distribution stops being dominated by tail rows, so the fitted line represents the majority without pretending the tail does not exist. And leverage collapses: a point at 8.2 versus 9.9 on the log scale is strong signal, not a hijacked gradient.

Where the recipe applies — and where it does not:

Put the transform in the pipeline rather than in a notebook cell, so it applies identically on every fold and at serving time:

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import FunctionTransformer

log_step = FunctionTransformer(np.log1p, feature_names_out='one-to-one')
pre = ColumnTransformer([('log', log_step, ['revenue', 'claims'])],
                        remainder='passthrough')

This composes with the scaling discipline from Feature Scaling and Normalization: log first, then standardize, both inside the same ColumnTransformer — the standard recipe for monetary features.

Interpretation shifts with the transform, and you should be able to state the change cleanly. In the raw model, a coefficient on revenue is dollars-of-target per extra dollar of revenue. In the log-log model it is an elasticity: a 1% rise in revenue moves the target by β%. Elasticity is often the truer business statement anyway — nobody at a pricing review talks in absolute dollars across a range spanning three orders of magnitude.

One caution on inverse predictions, because it is a reliable interview differentiator: predicting on a log target and reporting np.expm1(pred) recovers the median of the conditional distribution, not the mean. Recovering means requires a smearing correction (Duan's smearing, or a fitted bias factor). Notice it, name it, and you have separated yourself from most candidates.

Next, the columns most people ignore completely: raw timestamps, which hide day-of-week, seasonality, and recency signal that no amount of scaling will manufacture.

For a log transform, inspect zeros and negatives before applying log1p. The transformed coefficient describes a change on the transformed scale, so communicate predictions in the original units when that is what the user needs.

For a log transform, inspect zeros and negatives before applying log1p. The transformed coefficient describes a change on the transformed scale, so communicate predictions in the original units when that is what the user needs.

For a log transform, inspect zeros and negatives before applying log1p. The transformed coefficient describes a change on the transformed scale, so communicate predictions in the original units when that is what the user needs.

A raw signup_ts column is refused by every sklearn estimator, but the moment you decompose it you usually find three or four of the strongest predictors in the dataset. Retail, banking, and ride-hailing models all lean on when as much as what.

The first recipe is mechanical decomposition:

ts = pd.to_datetime(df['signup_ts'], utc=True)

df['dow']        = ts.dt.dayofweek        # 0=Mon ... 6=Sun
df['hour']       = ts.dt.hour
df['month']      = ts.dt.month
df['is_weekend'] = df['dow'].isin([5, 6]).astype(int)
df['is_month_end'] = ts.dt.is_month_end.astype(int)   # payday effects

Day-of-week and hour routinely beat hand-built ratios in behavioral targets — support tickets spike Monday morning, fraud spikes at 2 a.m., retail peaks Thursday evening. This is the cheapest feature engineering that exists: the column was already there.

The second recipe handles the trap hidden inside calendar parts: day-of-week 6 sits next to day-of-week 0, and an integer code tells no model that. Cyclic encoding says it with trigonometry:

df['dow_sin'] = np.sin(2 * np.pi * df['dow'] / 7)
df['dow_cos'] = np.cos(2 * np.pi * df['dow'] / 7)
# Sunday (6) and Monday (0) land adjacent on the circle

Use it for genuinely circular variables (hour, day-of-week, month, compass direction); skip it for variables that only look cyclic — day-of-month restarts, but businesses rarely treat the 31st as adjacent to the 1st, so a plain integer or bins do better.

The third recipe is time-as-distance: how long ago something happened. This is where signal and leakage meet, so the reference moment matters — "now" must be the scoring timestamp, never the training run's date:

now = pd.Timestamp('2026-01-01', tz='UTC')
df['acct_age_days']   = (now - ts).dt.days
df['days_since_last'] = (now - df.groupby('user_id')['event_ts']
                              .transform('max')).dt.days

Lag and rolling features carry one hard rule: shift before you aggregate. A 'orders in the last 7 days' column whose window includes the current row's own day has already peeked at the answer:

hist = (df.sort_values('event_ts')
          .groupby('user_id')['order_amount']
          .rolling('7D', closed='left')   # excludes the current timestamp
          .sum().reset_index())

closed='left' — or an explicit .shift(1) after daily groupby aggregation — guarantees the feature only sees strictly earlier rows. The same discipline applies to 'mean demand per product' style features: computed over the whole table, they average in the future. Get this wrong and you have built a model that scores 0.98 locally and 0.62 in production; the full anatomy of that failure is Data Leakage Prevention, the next topic.

One operational detail to keep honest: recency features drift with calendar time. A days_since_last column computed against a fixed 'now' during training will be wrong for every row scored later unless the serving code recomputes it against the live scoring timestamp — a feature that rots silently in production.

Create interaction features only when the combined effect is meaningful. Centering numeric inputs can make lower-order coefficients easier to interpret, but it does not eliminate confounding or replace validation.

Create interaction features only when the combined effect is meaningful. Centering numeric inputs can make lower-order coefficients easier to interpret, but it does not eliminate confounding or replace validation.

Create interaction features only when the combined effect is meaningful. Centering numeric inputs can make lower-order coefficients easier to interpret, but it does not eliminate confounding or replace validation.

A gradient-boosted tree will discover on its own that price / square_meter matters, that very high tenure combined with low balance flags churn, and that amount behaves differently above $500. A linear model discovers none of these — it sees columns, not relationships between them — and this is where feature engineering closes the gap.

Ratios and differences. The most durable recipe: divide two features that share a unit, or subtract one from another.

df['price_per_sqm']    = df['price'] / df['area'].replace(0, np.nan)
df['balance_per_year'] = df['balance'] / df['tenure_months'].clip(lower=1)
df['capacity_used']    = df['used_gb'] / df['quota_gb'].clip(lower=1e-9)

Guard every denominator — a single zero produces inf and poisons an entire fold, and SimpleImputer downstream will not save you if the column arrives as infinity. Prefer ratios whose meaning you can state in one sentence; a ratio nobody can interpret is usually noise with a division sign.

Binning. Turn a continuous feature into intervals when the effect is stepped rather than smooth — insurance deductibles, tax brackets, and 'first purchase over $500 gets a coupon' all have cliffs:

from sklearn.preprocessing import KBinsDiscretizer
kbd = KBinsDiscretizer(n_bins=5, encode='ordinal', strategy='quantile')
df['amount_bin'] = kbd.fit_transform(df[['amount']])

Quantile strategy gives equal-count bins — robust to the skew you failed to log away — and the estimator stores the edges, so serving-time values bin by the same boundaries. The cost is real: binning discards within-bin resolution, so keep the raw column alongside and let the model weight both. (The same trap from ordinal encoding applies: the bin code is an order, the bin edges are the information.)

Interactions and polynomials. The explicit version for linear models:

from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2, interaction_only=True, include_bias=False)
# [a, b, c] -> [a, b, c, ab, ac, bc]

interaction_only=True is the setting that matters: products of distinct features without the squares, keeping column count at k(k−1)/2 instead of the full quadratic blow-up. Two engagement columns multiplied can be powerful (visits x minutes-on-site), but every interaction is another coefficient to estimate — with 5,000 rows, a 40-feature interaction set already overfits. Generate them inside cross-validation and keep them only if folds improve, not the training score.

The theme across every recipe here: engineering is hypothesis work. State what the feature asserts about the world, then let fold scores judge the assertion. What the folds cannot judge, however, is a feature that leaked its answer in — computing any of these transforms on the full dataset before splitting is exactly the crime dissected in Data Leakage Prevention.

At serving time, use the same calendar, timezone, and missing-date policy as training. Monitor feature distributions and null rates so a change in upstream data does not silently alter the model's behavior.

At serving time, use the same calendar, timezone, and missing-date policy as training. Monitor feature distributions and null rates so a change in upstream data does not silently alter the model's behavior.

At serving time, use the same calendar, timezone, and missing-date policy as training. Monitor feature distributions and null rates so a change in upstream data does not silently alter the model's behavior.