Rolling and Expanding Windows

Rolling Windows for Time Series

Time series data is noisy. Daily sales spike and dip due to promotions, weekends, and random variation. A rolling window smooths this noise by computing statistics over a sliding window of consecutive observations.

The most common application is the moving average — the average of the last N observations. It reveals the underlying trend by averaging out short-term fluctuations.

import pandas as pd
import numpy as np

dates = pd.date_range('2024-01-01', periods=30, freq='D')
np.random.seed(42)
sales = pd.Series(
    np.random.randint(80, 130, size=30),
    index=dates,
    name='daily_sales'
)

# 7-day moving average
sales_ma7 = sales.rolling(window=7).mean()
print(sales_ma7.tail(10))
# 2024-01-21    108.142857
# 2024-01-22    109.285714
# 2024-01-23    110.428571
# 2024-01-24    111.571429
# 2024-01-25    112.714857
# 2024-01-26    113.857143
# 2024-01-27    115.000000
# 2024-01-28    116.142857
# 2024-01-29    117.285714
# 2024-01-30    118.428571

The rolling(window=7) creates a rolling window of 7 observations. The .mean() computes the average within each window. The first 6 values are NaN because the window is not full — there are not enough preceding observations to fill it.

The window size controls the tradeoff between smoothness and responsiveness. A small window (e.g., 3) tracks changes closely but remains noisy. A large window (e.g., 30) is smooth but lags behind real trends. Choose based on the frequency of your data and the question you are answering. For daily data with weekly seasonality, a 7-day window is natural because it covers one full cycle.

# Compare different window sizes
sales_df = pd.DataFrame({
    'sales': sales,
    'ma_3': sales.rolling(3).mean(),
    'ma_7': sales.rolling(7).mean(),
    'ma_14': sales.rolling(14).mean()
})
print(sales_df.tail(10))

Rolling windows are not limited to the mean. Any aggregation works: .sum(), .std(), .min(), .max(), .median(), .var(). The choice depends on what you need to measure. Rolling standard deviation measures volatility. Rolling min/max identifies extremes within a recent window.

Rolling Aggregations and Custom Functions

Beyond the built-in aggregations, rolling windows accept custom functions via .apply(). The function receives a NumPy array of the window's values and must return a scalar.

import pandas as pd
import numpy as np

dates = pd.date_range('2024-01-01', periods=30, freq='D')
np.random.seed(42)
sales = pd.Series(np.random.randint(80, 130, size=30), index=dates)

# Rolling range (max - min) within each window
def rolling_range(window):
    return window.max() - window.min()

range_7 = sales.rolling(7).apply(rolling_range)
print(range_7.tail(10))

Custom functions are powerful but slow. The .apply() method calls your Python function once per window, which is orders of magnitude slower than the C-optimized built-in aggregations. When a built-in exists, use it. When it does not, consider whether you can express the computation using built-in operations on the rolling object.

# Instead of a custom function for range:
range_7_fast = sales.rolling(7).max() - sales.rolling(7).min()

This produces the same result but runs entirely in C. The pattern — decompose a custom statistic into built-in rolling operations — is a key performance technique.

Rolling windows also support the min_periods parameter, which controls how many observations are required to produce a result. By default, min_periods equals the window size, so partial windows produce NaN. Setting min_periods=1 computes results even for partial windows.

# Compute mean even for partial windows (first 6 values will not be NaN)
ma7_partial = sales.rolling(7, min_periods=1).mean()
print(ma7_partial.head(10))
# 2024-01-01    102.0
# 2024-01-02    105.5
# 2024-01-03    108.0
# 2024-01-04    109.25
# 2024-01-05    110.6
# 2024-01-06    111.5
# 2024-01-07    112.142857
# 2024-01-08    113.285714

Use min_periods=1 when you need a value for every observation, such as when computing a rolling feature for a machine learning model. Use the default when you want to ensure the statistic is computed over a full window.

The center parameter shifts the window so the result is centered on the current observation rather than trailing. This is useful for visualization but not for forecasting, where you must not use future data.

# Centered window: uses 3 observations before and 3 after
ma7_centered = sales.rolling(7, center=True).mean()

For forecasting and anomaly detection, always use trailing windows (the default). A centered window leaks future information into the current value, which invalidates any forward-looking analysis.

Expanding Windows for Cumulative Statistics

An expanding window grows with the data. It starts at the first observation and includes all preceding observations at each step. The result is a cumulative statistic — a running total, running average, or running maximum.

import pandas as pd
import numpy as np

dates = pd.date_range('2024-01-01', periods=10, freq='D')
np.random.seed(42)
sales = pd.Series(np.random.randint(80, 130, size=10), index=dates)

# Cumulative sum
cumsum = sales.expanding().sum()
print(cumsum)
# 2024-01-01    102
# 2024-01-02    211
# 2024-01-03    320
# 2024-01-04    429
# 2024-01-05    538
# 2024-01-06    647
# 2024-01-07    756
# 2024-01-08    865
# 2024-01-09    974
# 2024-01-10   1083

The expanding window answers questions like: what is the total revenue so far? What is the average sale price to date? What is the highest single-day sale we have seen?

# Cumulative statistics
print("Running total:", sales.expanding().sum())
print("Running average:", sales.expanding().mean())
print("Running max:", sales.expanding().max())
print("Running min:", sales.expanding().min())

The cumulative average is particularly important. It shows how the average evolves as more data arrives. Early values are volatile because they are based on few observations. Later values stabilize as the sample size grows.

running_avg = sales.expanding().mean()
print(running_avg)
# 2024-01-01    102.000000
# 2024-01-02    106.500000
# 2024-01-03    109.666667
# 2024-01-04    112.250000
# 2024-01-05    114.600000
# 2024-01-06    116.500000
# 2024-01-07    118.142857
# 2024-01-08    119.625000
# 2024-01-09    120.888889
# 2024-01-10    122.100000

Expanding windows also support min_periods, which defaults to 1. This means the first observation produces a result (itself), the second produces a result based on two observations, and so on. You can raise min_periods to require a minimum sample size before producing a value.

# Require at least 3 observations before computing
running_avg_3 = sales.expanding(min_periods=3).mean()
print(running_avg_3.head(5))
# 2024-01-01          NaN
# 2024-01-02          NaN
# 2024-01-03    109.666667
# 2024-01-04    112.250000
# 2024-01-05    114.600000

The key difference between rolling and expanding: rolling windows have a fixed size and slide forward, discarding old observations. Expanding windows grow, incorporating all history. Use rolling for local trends and recent behavior. Use expanding for cumulative metrics and running totals.

Window Operations on Grouped Data

Real-world time series often has multiple groups — sales per store, temperature per city, stock prices per ticker. Window operations must respect group boundaries. A rolling average for store A must not include observations from store B.

The pattern is to group first, then apply the rolling operation within each group.

import pandas as pd
import numpy as np

np.random.seed(42)
dates = pd.date_range('2024-01-01', periods=10, freq='D')
df = pd.DataFrame({
    'store': ['A'] * 10 + ['B'] * 10,
    'date': list(dates) * 2,
    'sales': np.random.randint(80, 130, size=20)
})

# Rolling mean within each store
df['ma_3'] = df.groupby('store')['sales'].transform(
    lambda x: x.rolling(3).mean()
)
print(df)

The groupby('store')['sales'] selects the sales column grouped by store. The .transform() applies the rolling mean within each group and broadcasts the result back to the original row positions. Without .transform(), the result would be a grouped Series that does not align with the original DataFrame.

An alternative approach uses .rolling() directly on the grouped object (available in pandas 1.3+):

df['ma_3'] = df.groupby('store')['sales'].rolling(3).mean().reset_index(level=0, drop=True)

This produces the same result but requires .reset_index(level=0, drop=True) to align the MultiIndex result with the original DataFrame. The .transform() approach is more concise and is generally preferred.

Expanding windows work the same way:

# Cumulative sum within each store
df['cumsum'] = df.groupby('store')['sales'].transform(
    lambda x: x.expanding().sum()
)

A critical requirement for window operations: the data must be sorted by time within each group. If the dates are out of order, the rolling window will compute over the wrong observations.

# Sort by store and date before applying window operations
df = df.sort_values(['store', 'date'])
df['ma_3'] = df.groupby('store')['sales'].transform(
    lambda x: x.rolling(3).mean()
)

This sorting step is easy to forget and produces silently wrong results. Always verify that your data is sorted chronologically within groups before applying rolling or expanding operations. A quick check: plot the raw data for one group and confirm the x-axis is monotonic.