Documenting Models — Cards & Datasheets

P15.fairness-privacy.04 · Audience: guest, it-ml, language-pro · Prerequisites: Fairness Metrics — and Why They Conflict

Privacy — What Aggregation Cannot Hide The Governance Landscape

LLM grading (this page):Real gradingOff — no LLM calls

A team built a model to predict which patients were at high risk and needed extra care. It worked well and was widely adopted. Years later, researchers discovered it was steering care away from Black patients — not because it used race, but because it had been trained to predict healthcare cost as a stand-in for healthcare need, and less had historically been spent on Black patients for the same illness. The proxy was in the objective, exactly the failure the first module named. But here is the part this module is about: the people deploying the tool had no way to know. There was no document that said what the model predicted, what it was validated for, which groups it had been checked on, or where it should not be trusted. The model's most important limitation lived only in the heads of the few who built it — and they had moved on.

That is the failure documentation prevents. A model is not just its weights; it is a set of claims about what it does and where it works, and if those claims are not written down, every downstream user is guessing. Model cards and datasheets are the artefacts that write them down — short, structured documents that travel with a model and a dataset so the next person inherits the knowledge instead of the blind spot. This module reads one, names the sections that matter, and then has you write one for the biased screener from earlier in the track.

🗺️ So far you have built: that bias enters at four points and that fairness is a per-group measurement with a built-in conflict (Where Bias Enters; Fairness Metrics), and that 'anonymous' data can still leak (Privacy). Every one of those findings is only useful if it reaches the person deploying the model. This module is how it reaches them.

In this module you will: the documentation that keeps a model accountable after it ships. You will learn what a model card records — intended use, out-of-scope use, per-group metrics, limitations — and what a datasheet records for the dataset upstream of it; read a worked card carrying the fairness numbers from module two; and see why the discipline is less about a template than about the honesty to write down where your model should not be used.

Step 1 / 5 — A model is a set of claims — write them down

When a model leaves the team that built it, it carries its weights but not its context. The people who deploy it, integrate it, or rely on its output rarely know what data it saw, what it was optimised for, or the conditions under which it was validated. In that vacuum they assume the reasonable thing — that it works for their case — and the assumption is often wrong. The care-risk model failed precisely here: nothing told its users that "risk" meant "predicted cost", so nobody thought to question it.

ⓘ Concept: Undocumented is unaccountable

model = weights + claims about what it does and where it works · claims unwritten → downstream users guess → the guess is often wrong

Why it matters — You cannot hold a model, or its makers, accountable for a claim that was never written down. Documentation converts private knowledge — the trade-offs the team debated, the groups they checked, the uses they never intended — into a public record the next person can read, cite and challenge. Without it, every finding from this pillar evaporates the moment the original team disperses: the fairness gap they measured, the population they trained on, the purpose they scoped all vanish, and the model floats free of the caveats that made it responsible to release. A model card is how a team's honesty about its own model survives contact with the wider world.

🗣️ From a linguist's perspective: The translator's note that travels with the text

A careful translator attaches a note: this rendering targets a lay audience, not a legal one; these terms were localised, these left in the source language; here is where the original was ambiguous and how the choice was made. Strip the note and a reader may use a plain-language translation as if it were a certified legal one — the same document, deployed out of scope, causing harm. A model card is that translator's note for a model: not the artefact itself, but the indispensable statement of what it is for and where it must not go.

✅ Key takeaways

A model is weights plus claims about what it does and where it works; if the claims are not written down, every downstream user guesses. A care-risk model steered care from Black patients because nothing told deployers that 'risk' secretly meant 'predicted cost'.
A model card is a short structured document — intended use, out-of-scope use, training data, per-group metrics, limitations. The two sections that prevent harm are the uncomfortable ones: where the model must NOT be used, and how it performs broken out by group rather than on average.
A responsible card surfaces flaws on purpose: putting a demographic-parity gap of 0.5 in a box a deployer must read turns shipping-anyway into a documented, attributable decision instead of an accident of the average.
A datasheet is the model card's upstream companion — it documents the dataset (why it was collected, who is represented and missing, how labels were assigned, what not to reuse), where most bias originates. Together they trace accountability from data in to model out.
Reading a card is easy; writing an honest one is the skill, because the load-bearing sections are the ones a proud team wants to soften. A card that only flatters is worse than none — it lends false confidence.

Next: P15.governance-safety.01 — The Governance Landscape

Ask the mentor about this module

Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.

Ctrl/Cmd + Enter to send

🎓 Practice ladder

1 graded rung · ~15 min

Now write one. The rung below is a mentor-graded craft exercise: draft the model card for the biased screener as it should have existed before it shipped. There are no hidden tests — a mentor reads your card and grades it on whether it covers the sections that do the work: intended use, out-of-scope use, the per-group metrics from module two, and honest limitations.

Write the model card

Loading exercise…

Try it yourself

A scratch console for this page's ideas — ungraded, nothing you run here is recorded.

Scratch console

A scratch console with the scientific stack (pandas, numpy, scikit-learn). Runs on the server — no network, resource-limited and measured.

Output appears here.

Where next?

Continue

The Governance Landscape →

Governance & Safety · P15

Go up a level