Time Series & Data Issues

Interview-depth applied ML: time-series stationarity and autocorrelation, class imbalance and SMOTE pitfalls, and missing-data mechanisms — with the assumptions, the leakage traps, and the 'why it matters' framing.

Back to the foundations: probability and statistics basics live in the P1 statistics-core track, and Advanced Probability (P1.statistics-advanced.01) covers the estimation and sampling machinery.

1 — Stationarity and autocorrelation (s50)

A series is (weakly) stationary when its statistical properties are time-invariant: a constant mean, a constant variance, and an autocovariance γ(h) = Cov(Xₜ, Xₜ₊ₕ) depending only on the lag h, never on absolute time t. Most classical time-series models assume stationarity, because a shifting mean or variance makes yesterday's fitted parameters useless tomorrow.

Autocorrelation quantifies self-dependence across lags. The ACF is the normalised autocovariance ρ(h) = γ(h)/γ(0); the PACF removes the influence of the intermediate lags, isolating the direct lag-h effect. Together they diagnose the order of a model — a sharp PACF cut-off suggests AR order p, a sharp ACF cut-off suggests MA order q.

Testing stationarity. Use the ADF test (null: a unit root / non-stationary) alongside the KPSS test (null: stationary) — they are complementary, so reading both guards against a misleading single verdict. When a series is non-stationary, restore stationarity by differencing (∇Xₜ = Xₜ − Xₜ₋₁), de-trending, or a variance-stabilising transform (log, Box–Cox).

ARIMA(p, d, q) ties this together: an AR(p) term (regress on p past values), d orders of differencing to reach stationarity, and an MA(q) term (regress on q past errors). Seasonal SARIMA(p, d, q)(P, D, Q)ₛ adds seasonal AR/MA/differencing at period s.

Why it matters — validation. Time-ordered data must be split by time, never shuffled. Use forward-chaining (expanding- or rolling-window) cross-validation: always train on the past and test on the future. Random shuffling leaks future information backward (look-ahead leakage), inflating offline scores that then collapse in production.

2 — Class imbalance and SMOTE pitfalls (s28)

When positives are rare (fraud, disease, defaults), accuracy is misleading: a model that predicts "negative" for everyone scores 99% on a 1%-positive dataset while catching nothing. The optimiser, minimising average loss, is happy to ignore the minority class entirely.

Remedies.

• Class weights / cost-sensitive learning — reweight the loss so a minority error costs more; the cleanest fix as it changes no data.
• Threshold tuning — the default 0.5 cutoff is rarely optimal; pick the operating point from a PR or ROC curve to match the business cost of false positives vs false negatives.
• Resampling — random over-sampling the minority (risks overfitting duplicates) or random under-sampling the majority (discards data).
• SMOTE (Synthetic Minority Over-sampling Technique) — instead of duplicating, synthesise new minority points by interpolating between a minority example and its k nearest minority neighbours, placing a synthetic point along the connecting segment.

Pitfalls of SMOTE.

• Leakage — resample inside each CV fold, on the training data only. Applying SMOTE before the split lets synthetic points derived from validation neighbours leak into training, producing optimistic, unreproducible scores.
• Boundary blurring / noise amplification — interpolating across an overlapping region invents points in the wrong class's territory, and interpolating from a noisy or outlier minority point propagates that noise.
• Metrics — never judge with accuracy. Use PR-AUC, F1, and recall (plus the confusion matrix), which stay honest under heavy imbalance because they focus on the positive class rather than the dominant negatives.

3 — Missing-data mechanisms MCAR/MAR/MNAR (s32)

The right way to handle missing values depends entirely on why they are missing — Rubin's three mechanisms.

• MCAR (missing completely at random) — missingness is independent of everything, observed and unobserved alike (a sensor that drops readings by pure chance). Listwise deletion is unbiased here, merely wasteful of data.
• MAR (missing at random) — missingness depends only on observed variables (older respondents skip an income question, but age is recorded). Conditioning on the observed data removes the bias, so multiple imputation and model-based (e.g. EM, likelihood) methods are valid.
• MNAR (missing not at random) — missingness depends on the unobserved value itself (high earners refuse to state their income). No amount of observed data fixes this; you must explicitly model the missingness mechanism (selection/pattern-mixture models), and the distinction from MAR is generally untestable from the data alone.

Imputation cautions. Mean imputation understates variance (every gap collapses to one value, deflating standard errors and correlations). Prefer multiple imputation — impute several times to propagate the uncertainty — and add missingness indicator flags so a model can learn whether the fact of being missing is itself predictive.

Interview one-liners

• Stationary = time-invariant mean/variance/autocovariance; ACF/PACF diagnose the AR/MA orders and ADF+KPSS test the assumption.
• ARIMA(p, d, q) = AR + d differencing + MA; SARIMA adds a seasonal block at period s.
• Never shuffle time series — forward-chaining/time-based splits avoid look-ahead leakage.
• Imbalance breaks accuracy — use class weights/threshold tuning/SMOTE and judge with PR-AUC / F1 / recall.
• SMOTE only inside the training fold — resampling before the split leaks and blurs the class boundary.
• MCAR delete-safe, MAR impute-safe, MNAR needs the mechanism modelled — and mean imputation understates variance.

Rigorous references for applied time-series and data hygiene.