Model Evaluation, Selection & Interpretability

Interview-depth model evaluation for the ml-engineer track: metric choice under imbalance and calibration, feature engineering and leakage, nested cross-validation, generalization and double descent, and interpretability — with the assumptions and the 'why it matters' framing.

Back to the foundations: evaluation statistics live in the P1 statistics-core track (P1.statistics-core.05 — Evaluation Statistics); the regularization and capacity-control story is covered by the AI/ML foundations pillar.

1 — Metric choice under imbalance: ROC-AUC vs PR-AUC, calibration (s27)

Accuracy misleads under imbalance. With 99% negatives, a model that always predicts the majority class scores 99% accuracy while catching zero positives — the metric rewards the base rate, not the skill.

ROC-AUC plots the true-positive rate against the false-positive rate across thresholds:

Because FPR normalises by the (large) negative count, ROC-AUC is insensitive to the base rate and can look optimistic when negatives dominate — a handful of false positives barely moves it.

PR-AUC plots precision against recall:

Precision does depend on the positive base rate, so PR-AUC focuses squarely on the positive/minority class and is preferred under heavy imbalance (fraud, disease, rare-event detection), where the cost of false positives on a small positive class is what actually matters.

Calibration matters whenever probabilities — not just rankings — drive decisions (expected value, thresholds, cost-sensitive actions). A calibration (reliability) diagram bins predicted probabilities and plots them against observed frequencies; the expected calibration error (ECE) summarises the gap. Post-hoc fixes: Platt scaling (a logistic fit on the scores) and isotonic regression (a non-parametric monotone map). ROC-AUC and PR-AUC are rank metrics and are blind to miscalibration — a model can rank perfectly yet output badly-scaled probabilities.

2 — Feature engineering, selection & data leakage (s29)

Feature engineering — encodings (one-hot, target/ordinal), interaction terms, non-linear transforms, scaling/normalisation — increases the signal a model can exploit. Feature selection prunes noise to reduce variance and overfitting:

• Filter methods rank features by a univariate statistic (correlation, mutual information, χ²) independently of the model.
• Wrapper methods (forward/backward selection, RFE) search subsets by retraining the model.
• Embedded methods select during fitting — e.g. L1 (Lasso) drives coefficients to zero, and tree-based importances.

The cardinal sin is data leakage — letting information that would be unavailable at prediction time seep into training. It comes in two flavours:

1. Preprocessing leakage: fitting any transform on the full dataset before the split — a scaler's mean/variance, target encoding, imputation statistics, or feature selection — so the validation fold has already influenced training.
2. Target leakage: using future- or target-derived features (a column populated after the label is known, an ID that encodes the outcome).

Data leakage inflates offline metrics and then fails in production because the leaked signal is not really there at inference. The fix: fit every transform inside the CV fold, never on held-out data — wire preprocessing and the estimator into a single pipeline so fit sees only the training partition of each fold.

3 — Hyperparameter tuning and nested cross-validation (s31)

Hyperparameters (regularisation strength, tree depth, learning rate, k) are not learned by fitting — they are searched:

• Grid search — exhaustive over a discretised lattice; expensive, curse-of-dimensionality.
• Random search — samples configurations; often finds good points faster when only a few hyperparameters matter.
• Bayesian optimisation — builds a surrogate (e.g. a Gaussian process) over the validation score and picks the next trial by an acquisition function; sample-efficient.

The trap: tuning hyperparameters and estimating generalisation on the same cross-validation reuses the validation folds for two jobs, so the model selection leaks into the performance estimate and the reported score is optimistically biased.

Nested cross-validation separates the two:

• an inner loop performs model selection — it searches hyperparameters on its own train/validation splits;
• an outer loop estimates performance — for each outer fold it takes the inner-selected model and scores it on outer test data the inner loop never saw.

Because selection happens strictly inside the outer training partition, nested cross-validation gives an approximately unbiased estimate of generalisation — at the cost of O(outer × inner) more fits.

4 — Generalization, capacity & double descent (s33)

The classical bias-variance story is a U-curve: as capacity grows, training error falls monotonically while test error first drops (bias down) then rises (variance up), with an optimal sweet spot in the middle. This underlies "don't over-parameterise."

Double descent revises that picture. Plot test error against capacity and continue past the interpolation threshold — the point where the model has just enough capacity to fit the training set exactly (zero training error):

• Up to the threshold you see the classical U — test error rises as variance explodes near the interpolation point.
• At the threshold test error can peak.
• Beyond it, adding more parameters makes test error fall again, so heavily over-parameterised models (modern deep nets, wide random-feature models) can generalise well despite interpolating the data.

This ties to VC dimension / effective capacity — raw parameter count overstates the effective degrees of freedom — and to implicit regularisation: gradient descent among the many zero-training-error solutions is biased toward low-norm / minimum-complexity interpolants that generalise. Double descent is why "bigger than needed to fit the data" is not automatically worse.

5 — Interpretability: global vs local, SHAP vs LIME (s59)

Interpretability answers two different questions:

• Global — how does the model behave overall? Feature importances (gain, permutation importance) and partial-dependence plots (marginal effect of a feature averaged over the data) describe the whole model.
• Local — why did the model make this prediction? Per-instance attributions for a single decision (a loan denial, a flagged transaction).

Two model-agnostic local methods:

• LIME (Local Interpretable Model-agnostic Explanations) perturbs the instance, weights the perturbations by proximity, and fits a simple local surrogate (usually sparse linear) whose coefficients explain that one prediction.
• SHAP (SHapley Additive exPlanations) grounds attribution in Shapley values from cooperative game theory: each feature's contribution is its average marginal effect over all orderings. This gives additive, consistent local explanations (f(x) = φ₀ + Σᵢ φᵢ) that also aggregate to a global view (mean |φᵢ|).

Trade-offs. SHAP has strong theoretical guarantees (local accuracy, consistency, missingness) but exact Shapley values are exponential — practical use relies on approximations (KernelSHAP, TreeSHAP). LIME is cheaper and intuitive but its surrogate can be unstable — small changes in perturbation sampling or the neighbourhood kernel shift the explanation, and local fidelity is only as good as the surrogate. The axes to weigh: fidelity, stability, and compute.

Interview one-liners

• Accuracy lies under imbalance: ROC-AUC is base-rate-insensitive (optimistic when negatives dominate); PR-AUC targets the minority class — and both are blind to calibration.
• Data leakage = fitting any transform (scaler, target encoding, imputation, selection) before the split or using target-derived features; fix by fitting inside the CV fold via a pipeline.
• Nested cross-validation: inner loop selects hyperparameters, outer loop estimates performance — the only way to avoid an optimistically biased score.
• Double descent: past the interpolation threshold test error can fall again, so over-parameterised models can still generalise.
• SHAP vs LIME: SHAP = Shapley values, additive/consistent, aggregates globally but costly; LIME = cheap local linear surrogate but less stable.

Rigorous references for model evaluation and interpretability.