Stationarity & Differencing

Interview-grade notes for the technical (it-ml) track.

The unit-root framing. Model the series as y(t) = ρ·y(t−1) + ε(t). If ρ < 1, shocks decay geometrically and the series is stationary; if ρ = 1, the model is a random walk — shocks accumulate forever, variance grows linearly with t, and the series has a unit root. The Dickey–Fuller null hypothesis is ρ = 1; the alternative is ρ < 1. This is why "high p-value" means "could not reject non-stationarity".

The test regression. Subtract y(t−1) from both sides to get Δy(t) = γ·y(t−1) + ε(t), with γ = ρ − 1. The null becomes γ = 0. The test runs this regression and computes the t-statistic of γ̂. The catch: under the null the regressor y(t−1) is itself a random walk, so that t-statistic does not follow a t-distribution — it follows the non-standard Dickey–Fuller distribution, whose critical values (tabulated by simulation, roughly −2.86 at 5% for the constant-only variant) are far more negative than the −1.96 intuition from ordinary regression. Using normal critical values would reject stationarity's absence far too often.

The "Augmented" part. The plain DF regression assumes ε(t) is uncorrelated. Real series have short-run dynamics, so ADF augments the regression with lagged difference terms — Δy(t−1), …, Δy(t−p) — to soak up autocorrelation in the residual. The lag order p is chosen by information criterion (statsmodels' adfuller defaults to AIC); too few lags leave residual autocorrelation that invalidates the distribution, too many burn power.

Deterministic terms. The regression variant matters: no constant, constant ("c", the default — appropriate for series with a non-zero level), or constant plus linear trend ("ct" — use when the alternative you care about is trend-stationarity, a stationary wiggle around a deterministic line). Each variant has its own critical values, and testing a clearly trending series with the wrong variant produces nonsense verdicts in both directions.

Power, and the KPSS complement. ADF has notoriously low power near ρ = 1: with a few hundred points it often cannot distinguish ρ = 0.97 from ρ = 1. Standard practice pairs it with the KPSS test, whose hypotheses point the other way (null: stationary). ADF-rejects + KPSS-accepts → confidently stationary; ADF-accepts + KPSS-rejects → confidently unit root; both reject or both accept → the data is not answering, and pragmatism (difference once, per the module) wins.