Fairness Metrics — and Why They Conflict

P15.fairness-privacy.02 · Audience: guest, it-ml, language-pro · Prerequisites: Where Bias Enters

Where Bias Enters Privacy — What Aggregation Cannot Hide

LLM grading (this page):Real gradingOff — no LLM calls

"The model was unfair" is an accusation, not a measurement. Before you can fix unfairness — or defend against the charge of it — you have to say precisely what you mean, in numbers a second person could check. That turns out to be harder than it sounds, because there is more than one reasonable way to define a fair decision, and this module's central, unsettling result is that two of the most natural definitions cannot both be satisfied at once when the groups differ in how often the good outcome actually occurs. Fairness is not one target you can hit if you try hard enough; it is a set of targets that pull against each other, and choosing among them is a value judgement no formula can make for you. We build the two definitions on a tiny, hand-checkable cohort, watch them disagree, and see why the disagreement is not a bug you can engineer away.

🗺️ So far you have built: that bias enters a model at four points — data, labels, objective and deployment (Where Bias Enters) — and that per-group evaluation is the only way to see the harm an overall accuracy hides. This module makes 'per-group evaluation' concrete.

In this module you will: how to put numbers on fairness, and why the numbers refuse to agree. You will meet two definitions — demographic parity (equal approval rates) and equalized odds (equal true- and false-positive rates) — build per-group confusion matrices on an eight-person cohort, read a demographic-parity gap of 0.5 and an equalized-odds gap of 0.5 straight off the counts, and then see the impossibility: when two groups have different base rates, you cannot drive both gaps to zero without a useless classifier.

Step 1 / 5 — Two reasonable definitions of a fair decision

Picture a model that approves or rejects applicants, and split the people it judges into two groups, A and B. What would make its decisions fair? Two answers both sound obviously right — and they are different answers.

Definition	Requires	The intuition
Demographic parity	equal approval rate across groups	"the same share of each group should be approved"
Equalized odds	equal true-positive rate AND equal false-positive rate	"the model should be equally accurate, and equally wrong, for each group"

ⓘ Concept: Parity looks at outcomes; odds look at accuracy

demographic parity: approval_rate(A) = approval_rate(B) · equalized odds: TPR(A) = TPR(B) AND FPR(A) = FPR(B)

Why it matters — These two definitions answer different moral questions, which is why they can conflict. Demographic parity cares only about the bottom line — did each group get approved at the same rate? — regardless of who was actually qualified. Equalized odds cares about the model's error behaviour — among genuinely qualified people, is it as likely to say yes for either group; among unqualified people, is it as likely to wrongly say yes? One asks 'are the outcomes equal?'; the other asks 'are the mistakes equal?'. A given classifier can honour one and violate the other, so you cannot say a model is 'fair' without saying which fairness you mean.

The vocabulary of true- and false-positive rates comes straight from the confusion matrix you met in evaluation — see Evaluation Metrics (P5) for the mechanics. The new move here is to compute those rates once per group and compare.

✅ Key takeaways

'Unfair' must become a measurement before it can be fixed. Two natural definitions disagree: demographic parity (equal approval rates) looks at outcomes; equalized odds (equal TPR and FPR) looks at whether the model's mistakes are equal across groups.
Every fairness number is a per-group rate you can count by hand: approval rate = approved / group, TPR = true positives / qualified, FPR = false positives / unqualified. On the eight-person cohort, group A reads 0.75 / 1.0 / 0.5 and group B reads 0.25 / 0.5 / 0.0.
The demographic-parity gap is max−min approval (0.75 − 0.25 = 0.5); the equalized-odds gap is the larger of the TPR and FPR gaps (max(0.5, 0.5) = 0.5). Equalized odds takes the worst of two, which is why it is strict.
The conflict is arithmetic, not a flaw: when base rates differ, a group's approval rate is a base-rate-weighted blend of its TPR and FPR — so equal error rates force unequal approval rates, and vice versa. The threshold-sweep frontier never reaches (0, 0).
Because no model satisfies every definition, 'make it fair' is a value decision, not a technical one: choose the fairness your context demands, and honestly document the residual gap you accepted on the other.

Next: P15.fairness-privacy.03 — Privacy — What Aggregation Cannot Hide

Going deeper (technical) — the impossibility theorem sketch

For the it-ml track: the conflict above is not folklore, it is a theorem. Kleinberg, Mullainathan and Raghavan (2017) and, in a parallel result, Chouldechova (2017) prove that three intuitive fairness conditions cannot simultaneously hold whenever the base rates differ and the classifier is imperfect.

Fix a group with base rate p (the fraction actually positive). Consider three conditions on a risk score:

Calibration within groups — among people the model assigns risk r, a fraction r really are positive, in every group.
Balance for the positive class — the average score of genuine positives is the same across groups (the score-level analogue of equal TPR).
Balance for the negative class — the average score of genuine negatives is the same across groups (the analogue of equal FPR).

The theorem: if two groups have different base rates (p_A ≠ p_B) and the model is not a perfect predictor, then no score can satisfy all three at once. The only escape hatches are the degenerate ones — perfect prediction (never available on real data) or equal base rates (not true of the populations we actually screen).

The intuition tracks the base-rate blend from the main text. Calibration ties a score's meaning to the group's base rate; the two balance conditions demand the score distributions line up across groups regardless of base rate. When base rates differ, these pull apart, and something has to give. This is why the threshold sweep in the senior rung finds a strictly-positive residual gap: the frontier's floor is the theorem made numerical. The practical upshot is unchanged — pick the condition your context most needs, quantify the violation of the others, and put that number in the model card.

Ask the mentor about this module

Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.

Ctrl/Cmd + Enter to send

🎓 Practice ladder

4 graded rungs · ~40 min

Now measure fairness yourself. Each rung is a three-panel workspace: instructions on the left, a code editor in the middle, output and test results on the right. Run checks the visible tests; Submit grades against hidden cohorts. The explorer rung reads the parity gap off per-group approval rates; the practitioner rungs build per-group confusion matrices and compute both gaps on a cohort where they disagree; the senior rung sweeps per-group thresholds to find the frontier and the residual gap that proves you cannot zero both.

Rung 1 — Read the gap (explorer)

Loading exercise…

Rung 2 — Group confusion matrices

Loading exercise…

Rung 3 — Parity vs odds

Loading exercise…

Rung 4 — The impossible threshold (senior)

Loading exercise…

Try it yourself

A scratch console for this page's ideas — ungraded, nothing you run here is recorded.

Scratch console

A scratch console with the scientific stack (pandas, numpy, scikit-learn). Runs on the server — no network, resource-limited and measured.

Output appears here.

Where next?

Continue

Privacy — What Aggregation Cannot Hide →

Fairness & Privacy · P15

Later in Fairness & Privacy

This module unlocks

Documenting Models — Cards & Datasheets

Go up a level