ML Lifecycle
A track of P13 · Production ML & MLOps.
Experiment tracking to model registry: the reproducibility spine of production ML, exercised on real MLflow.
The model worked in March. It is June, the numbers look different, and nobody can reproduce March: which rows was it trained on, what seed, which version of the preprocessing, which eval set produced that 0.91 everyone signed off on? None of it was written down — it lived in a notebook's memory and one engineer's recollection, and the engineer has moved teams. Nothing "broke"; the knowledge simply evaporated. This track is the discipline that makes March reconstructible on demand.
The first move is to treat evaluation itself as infrastructure rather than a notebook cell. Eval datasets get versions and names; every training and eval run is logged to an experiment tracker — the modules exercise real MLflow, with Weights & Biases and Braintrust as the same idea in other clothes — and a regression gate blocks any candidate that scores worse than the model it would replace. A metric in a notebook is a fact about one afternoon; a tracked run is evidence you can cite in June. The same module builds a model-graded-eval harness, with the human-calibration and leakage checks that keep LLM-as-judge honest.
The second move makes the model itself reproducible. A fitted model in memory is a fact about your machine; packaging turns it into a byte-exact artefact — serialised, checksum-pinned, carrying a manifest and a model card — that a serving container, a colleague, or you in six months can reload and get the same predictions from, verifiably. Registry promotion then becomes a recorded decision rather than a Slack message. This is the least glamorous track in the pillar and the one everything else depends on: Operations serves the artefact you package here, and the drift decisions in monitoring are only meaningful against the eval baseline you recorded here.