Governance, Ethics & Honest Data
P16.ml-literacy.04 · Audience: guest, it-ml, language-pro · Prerequisites: Metric Strategy
The track closes with the four questions that decide whether an organisation can be trusted with data at all: where bias actually comes from, what it means to hold customer data in trust, how a chart is made to mislead without lying, and what separates a team that uses numbers from a team that hides behind them.
One mental model. A model and a chart both reflect the choices behind them — which data went in, who was represented, and what was left out. Most governance and ethics questions come down to asking those questions before you trust the result.
Model bias usually does not come from the algorithm — it comes from the data: unrepresentative samples, historical human bias baked into labels, and proxies for protected traits. A model learns and can amplify these patterns.
Worked example — the postcode that stood in for ethnicity. A lending model has no ethnicity field — yet still discriminates. How? A proxy slipped in:
| Field in the model | What it secretly encodes | Effect |
|---|---|---|
| Postcode / ZIP | neighbourhood → strongly correlated with ethnicity | the model recreates a biased pattern it was never explicitly given |
Removing the sensitive field is not enough when a proxy carries the same information. Mitigate by checking representativeness of the data, testing outcomes across groups, and monitoring in production — bias can drift back in as the world and inputs change.
ⓘ Concept: Proxy variable
Why it matters — It defeats the most common compliance answer in the room. 'We removed gender and ethnicity from the model' sounds like a complete response and is not one: correlated fields reconstruct the removed attribute, so fairness has to be verified in the outcomes, never assumed from the input list.
⚡ Interview Ref — the quick-scan Reference face
In plain language: where bias sneaks into models and data, how to use customer data ethically and protect privacy, how to spot a chart that lies, and how to lead with numbers without being fooled by them. No maths required.
1 — Bias in a model or data; where it comes from (b21)
Model bias usually doesn't come from the algorithm — it comes from the data. Three common sources: unrepresentative samples (a hiring model trained mostly on one region or age group), historical human bias baked into labels (past decisions that were themselves unfair), and proxies for protected traits (a postcode or first name standing in for ethnicity or gender, even when those fields are removed). A model learns and can amplify these patterns rather than correcting them. Mitigate by checking representativeness of the data, testing outcomes across groups, and monitoring in production — bias can drift back in as the world and the inputs change.
2 — Data privacy and the ethics of using customer data (b28)
Customer data is held in trust, not owned outright. Ethical use rests on a few principles: consent (use data for what customers agreed to), purpose limitation (don't quietly repurpose it for something new), and minimisation (collect and keep only what you actually need). Watch for re-identification: "anonymised" data can often be linked back to individuals when combined with other datasets. And comply with regulations such as GDPR, which give people rights over their data. Strong privacy and ethical use protect both the customer and the business's reputation — a breach or a creepy use of data can cost far more than the analysis was ever worth.
3 — Spotting a misleading chart or statistic (b29)
The same honest numbers can be dressed up to mislead. Common tricks:
- A truncated axis — a y-axis that doesn't start at zero, exaggerating a tiny change into a dramatic cliff,
- Cherry-picked timeframes — starting the chart at a convenient low or high point,
- Missing baselines — a big number with nothing to compare it against,
- Confusing correlation with causation — two lines move together, so one is claimed to cause the other,
- Percentages without the underlying counts — "100% increase" when it went from 1 case to 2.
Before trusting a misleading chart, ask what's not shown: where does the axis start, what timeframe was chosen, and where are the raw counts?
4 — Building a data-informed (not data-blinded) culture (b31)
A data-informed culture uses numbers alongside judgment and context — data is one input into a decision, not the whole decision. It welcomes questions about where a figure came from and treats a challenged number as healthy, not disloyal. The opposite is data-blinded: cherry-picking the metrics that confirm a decision already made, or hiding behind a dashboard to avoid owning a judgment call. To build the healthy version: encourage people to ask how a number was produced, reward the question "what would change your mind?" before a decision is locked in, and treat context, ethics, and common sense as part of the analysis, not an afterthought.
Interview one-liners
- Bias lives in the data — unrepresentative samples, biased labels, and proxies for protected traits — and the model can amplify it.
- Customer data is held in trust: respect consent, purpose limitation, and minimisation, and protect privacy to protect the business's reputation.
- Spot a misleading chart by asking what's not shown — watch for a truncated axis, cherry-picked timeframes, and percentages without counts.
- Lead data-informed, not data-blinded: use numbers with judgment, welcome questions, and ask what would change your mind.
📚 Go Further
Plain-language explainers on fairness, privacy, and honest data.
| Type | Resource |
|---|---|
| Book | How to Lie with Statistics — Darrell Huff, on spotting misleading charts |
| Reference | GDPR — the EU rules on consent, purpose limitation, and data minimisation |
| In-app | Metric Strategy — for baselines, proxies, and reading numbers well |
Ask the mentor about this module
Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.