ML in Production (method view)

P5.ml-methods.07 · Audience: guest, it-ml, language-pro · Prerequisites: Model Evaluation, Selection & Interpretability

Model Evaluation, Selection & Interpretability Evaluation Metrics for NLP

A model that passed every offline check can rot in production without a single line of code changing, because the world it scores keeps moving while the model stands still. This module takes the five failures behind that sentence one at a time — the inputs drift away from the training distribution, the training and serving pipelines quietly compute the same feature differently, the proxy metric being optimised detaches from the business outcome it stood for, statistical rigour erodes under the pressure to find effects, and a recommender meets users and items it has never seen. Everything builds on the offline evaluation discipline of the previous module, and each failure is made concrete enough, in numbers or a few lines of code, to reconstruct in a data-scientist interview.

Framing. In production the model is the easy part — the data moves, the offline/online paths diverge, and the proxy metric gets gamed. Each step turns one production failure into numbers.

ⓘ Concept: Method view here — case studies in the P13 pillar

Read this module for the concepts and the interview framing; head to the P13 pillar (Production ML & MLOps) when you want to build the infrastructure — deployment pipelines, feature stores, eval harnesses, and monitoring dashboards — around them.

Why it matters — This module is the method-level view of production concerns: what drift, skew, and Goodhart failures are and how to reason about them in an interview. The Production ML & MLOps pillar (P13) goes deep on the same topics with hands-on case studies — packaging a model, serving it, and wiring up drift monitoring — and its modules reference this one rather than re-teaching it.

Step 1 / 5 — Drift & Feedback Loops (s35)

1 — Monitoring: data drift, concept drift, feedback loops (s35). Data drift = the input $p(x)$ shifts while $p(y \mid x)$ holds; concept drift = the relationship $p(y \mid x)$ itself changes (the dangerous one — only delayed-label performance catches it). Watch inputs with the population stability index (PSI) or KS test; watch the relationship with performance on labels as they arrive.

Worked example — a PSI alert on an input feature. Compare a feature's live distribution against the training reference by bucket:

PSI = sum( (live% - ref%) * ln(live% / ref%) )  over buckets

rule of thumb:  < 0.10      no shift
              0.10-0.25   moderate shift -> investigate
              > 0.25      major shift -> retrain

measured PSI = 0.25  -> major drift, trigger a retrain

A PSI of 0.25 trips the major-shift threshold, so the input distribution has moved far enough to retrain. But PSI only sees inputs — if fraud tactics evolve (concept drift) with $p(x)$ unchanged, PSI stays quiet and only delayed-label AUC reveals it. Beware feedback loops too: a recommender that only shows popular items starves the tail, and future training data confirms the bias.

Note. Monitoring is the online half of a loop whose offline half is the eval harness: drift here is the trigger to refresh the eval set and re-run evaluation before retraining. The evaluation infrastructure for that loop — eval harnesses and experiment tracking — lives in the P13 pillar, Production ML & MLOps.

⚡ Interview Ref — the quick-scan Reference face

Interview-depth production ML for the data-scientist track: drift monitoring and feedback loops, the MLOps stack and training-serving skew, aligning metrics with business impact, statistical-rigour failures, and recommenders — with the assumptions and the 'why' framing. Back to the foundations: ML6 — Evaluation & Model Selection, and Advanced Probability.

1 — Monitoring: data drift, concept drift, feedback loops (s35)

Once a model is live the world moves underneath it. Distinguish the failure modes:

Data drift (covariate shift) — the input distribution shifts, p(x) changes while the learned relationship p(y | x) stays fixed (new user segment, seasonality, a changed upstream sensor). The model is still "correct" but now extrapolates outside its training support.
Concept drift — the relationship itself changes: p(y | x) moves even if p(x) looks stable (fraud tactics evolve, a pricing policy changes buyer behaviour). This is the dangerous one — the mapping the model encodes is now wrong. Watching only inputs misses it, so concept drift demands label-based performance monitoring.
Label / prediction drift — the marginal p(y) or the distribution of the model's own outputs shifts; a cheap early proxy when true labels are delayed.

Detection. For inputs, compare live vs reference distributions with the population stability index (PSI), KS test, or per-feature histograms/embeddings. For the relationship, track performance on delayed labels (AUC, error) as ground truth arrives — the only direct signal for concept drift.

Feedback loops. Beware when the model's own actions change future data: a recommender that only shows popular items starves the tail of impressions, so future training data confirms the bias (a self-fulfilling loop). Left unmonitored these degrade silently and entrench bias.

Respond with alerting thresholds on drift metrics, scheduled or trigger-based retraining, shadow evaluation before promotion, and logging exploration to break feedback loops.

2 — What MLOps covers and training-serving skew (s62)

MLOps is DevOps adapted to the fact that an ML system is code + data + model, all three of which drift. It covers:

Versioning — code, data, and model artefacts all versioned so any prediction is reproducible (DVC, model registries, dataset snapshots).
Reproducible pipelines — orchestrated, parameterised training runs with tracked lineage.
CI/CD for models — automated tests (data validation, model quality gates), then automated deployment.
A feature store — a central, versioned place to define and serve features, giving the same transform to training (offline/batch) and serving (online) code.
Deployment patterns — batch (precompute), online (real-time), shadow (mirror live traffic, don't act), and canary (small % rollout with automatic rollback).
Monitoring — drift, latency, and quality (section 1).
Governance — access control, audit trails, lineage, and model documentation.

Training-serving skew. The classic silent killer: training-serving skew is any difference between how features are computed offline for training vs online for serving — a subtly different transform, a unit mismatch, or stale features (training on fresh data, serving on an hours-old cache). Metrics look fine offline yet production accuracy quietly degrades because the model sees inputs it was never trained on. The structural fix is a shared feature store or a single library of feature transforms invoked identically in both paths, so offline and online computations are provably the same.

3 — Aligning metrics with business impact; Goodhart's law (s36)

A model metric is only ever a proxy for the value the business actually wants. Optimising AUC, log-loss, or click-through rate is a stand-in for revenue, retention, or user well-being — and the two can diverge sharply.

Goodhart's law — "when a measure becomes a target, it ceases to be a good measure." The moment you optimise a proxy hard, the system finds ways to raise the number without delivering the underlying value: a CTR-maximising feed learns clickbait; a "time-on-site" target breeds addictive dark patterns. Goodhart is the reason a metric that correlated with value in observation stops correlating once it is the optimisation target.

Defences:

Pick a single north-star metric tied to real value, plus guardrail metrics (latency, complaint rate, churn, fairness) that must not regress while you optimise the proxy.
Validate causally with online experiments (A/B tests) — offline gains often don't materialise online.
Watch for proxy gaming and guardrail regressions explicitly; treat a proxy win with a guardrail loss as a net loss.

4 — p-hacking, pre-registration & the replication crisis (s52)

Running experiments analytically honestly is harder than it looks.

p-hacking — trying many analyses, subgroups, or stopping rules and reporting only the ones that cross p < 0.05. The garden of forking paths captures the subtler version: even without conscious cheating, data-dependent choices at each fork inflate the false-positive rate far above the nominal α.
HARKing ("Hypothesising After the Results are Known") — presenting a post-hoc pattern as if it were a pre-planned prediction, which converts exploratory noise into a spurious "confirmed" hypothesis.
The replication crisis — the empirical result: large swaths of published findings fail to replicate, because the literature is filtered for significance rather than truth (publication bias + the above).

Defences:

Pre-registration — commit the hypothesis, metric, sample size, and analysis plan before seeing outcomes, separating confirmatory from exploratory claims.
A locked holdout / confirmatory test that is touched only once.
Multiple-comparison correction (Bonferroni, Benjamini–Hochberg FDR) when many hypotheses are tested.
Report effect sizes and confidence intervals, not just p-values — significance is not magnitude.
Reproducible analysis (versioned code + data + seeds) so a result can be re-run exactly.

5 — Recommendation systems and the cold-start problem (s60)

Three families of recommender:

Collaborative filtering — learn from the user–item interaction matrix alone. Matrix factorisation decomposes it into latent user and item vectors, r̂ᵤᵢ = pᵤᵀ qᵢ, so "users who liked what you liked" is captured geometrically. Powerful, but needs interaction history.
Content-based — recommend items whose features (text, genre, embeddings) resemble what a user already engaged with. Works from item attributes, so it survives sparse interactions.
Hybrid — combine both (and context) to get collaborative signal where data is dense and content signal where it is sparse.

The cold-start problem. A pure collaborative system is helpless when there is no interaction history — a brand-new user or a just-added item has no rows/columns to factorise. Mitigations:

Content features — fall back to item/user attributes and embeddings so a new item can be matched before it has any clicks.
Popularity fallbacks — serve trending/top items until personal signal accrues.
Onboarding — explicitly elicit preferences (pick genres, follow topics) to seed the profile.
Contextual bandits — treat recommendation as exploration vs exploitation, deliberately probing uncertain new items (with a Hoeffding/UCB-style bonus) to gather the interaction data the model lacks.

Interview one-liners

Data drift = inputs p(x) shift; concept drift = the p(y | x) mapping changes — the latter needs delayed-label performance monitoring, not just input tests.
Training-serving skew is offline vs online feature computation diverging (or stale features); a shared feature store / identical transforms is the structural fix.
Goodhart's law: optimise a proxy hard and it stops tracking value — use a north-star plus guardrail metrics and validate online.
p-hacking / the garden of forking paths inflate false positives — pre-registration and a locked holdout are the antidotes; the replication crisis is what happens without them.
Cold-start: no history for new users/items — lean on content features, popularity fallbacks, onboarding, and bandit exploration.

📚 Go Further

Practical references for production ML and MLOps.

Type	Resource
Book	Huyen, Designing Machine Learning Systems — drift, feature stores, deploy
Book	Kleppmann, Designing Data-Intensive Applications — data pipelines
Paper	Sculley et al., Hidden Technical Debt in Machine Learning Systems
In-app	ML4 — Causal Inference (P5.ml-methods.04), for the online experiments that validate these metrics
In-app	ML6 — Evaluation & Model Selection (P5.ml-methods.06), for the offline metrics these systems monitor
In-app	P13 — Production ML & MLOps pillar, for the case-study deep dives (packaging, serving, drift monitoring)

✅ Key takeaways

Data drift means the inputs p(x) shift; concept drift means the mapping p(y | x) itself changes — PSI and KS tests watch the inputs, but only performance on delayed labels catches concept drift, and feedback loops silently entrench bias.
MLOps versions code + data + model with reproducible pipelines, CI/CD gates, and shadow/canary deployment; training-serving skew — features computed differently offline vs online — is the classic silent killer, and a shared feature store is the structural fix.
Every model metric is a proxy for business value, and Goodhart's law says a proxy optimised hard stops tracking value — defend with a north-star metric, guardrails that must not regress, and online A/B validation.
p-hacking and the garden of forking paths inflate false positives far above α (20 null tests give a 64% chance of a false 'discovery') — pre-registration, a locked holdout, and multiple-comparison correction are the antidotes.
Collaborative filtering needs interaction history, so new users and items hit cold-start — mitigate with content features, popularity fallbacks, onboarding elicitation, and bandit exploration. This module closes the ml-methods track: from algorithms to what survives contact with production.

Next: P5.evaluation.01 — Evaluation Metrics for NLP

Ask the mentor about this module

Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.

Images, PDF or text. Kept on this device only.

Keeping your files on this device

Keep a copy of the files I attach, in this browser

Off by default. The mentor always gets your file; this only decides whether your own copy stays here. Copies live in this browser only - they do not follow you to another device, and clearing site data removes them.

Ctrl/Cmd + Enter to send

Try it yourself

A scratch console for this page's ideas — ungraded, nothing you run here is recorded.

Scratch console

A scratch console with the scientific stack (pandas, numpy, scikit-learn). Runs on the server — no network, resource-limited and measured.

Output appears here.

My notes on this module

Loading your notes...

Where next?

Continue

Evaluation Metrics for NLP →

Evaluation · P5

Go up a level