Uplift & Heterogeneous Effects

P8.causal-methods.04 · Audience: guest, it-ml, language-pro · Prerequisites: Matching & Stratification

Difference-in-Differences Learning from Reward

Every method so far chased a single number: the effect of a treatment, averaged over everyone. But an average hides a question that is often the one that pays the bills — who does the treatment help, whom does it not touch, and whom does it quietly harm? This closing module of the pillar is about heterogeneous treatment effects: the effect varies from person to person, and the goal shifts from measuring an effect to targeting the people it actually moves. That is uplift, and it will reorganise how you think about a coupon, a reminder, or a retention call.

Step 1 / 5 — The average hides the people

A retention email lifts renewals by 4 percentage points on average. Reassuring — until you learn how that 4 is built. Maybe every customer is nudged a little. Or maybe the email does nothing for most people, works wonders on a small persuadable slice, and backfires on another slice who did not want to be reminded their subscription was auto-renewing. All three stories share the same average and demand completely different actions.

ⓘ Concept: Heterogeneous treatment effects

the effect τ(x) depends on the individual's features x; the average effect is just its population mean and can mask both zeros and negatives

Why it matters — The average treatment effect answers 'should we do this at all?'. It cannot answer 'to whom?' — and when treatment is costly or occasionally harmful, 'to whom' is the decision that determines whether the programme makes or loses money. Two interventions with identical average lift can have opposite optimal targeting policies, so an average alone can send you spending on exactly the wrong people.

The move of this module is to stop averaging and start segmenting: group people by how the treatment affects them, and treat only the groups worth treating.

✅ Key takeaways

An average treatment effect hides who the treatment moves; heterogeneous effects vary person to person, and the goal becomes targeting, not just measuring.
Two potential outcomes sort people into four types: persuadables (uplift +1, treat them), sure things and lost causes (uplift 0, immune), and sleeping dogs (uplift −1, treatment harms them).
Uplift = P(action | treated, segment) − P(action | control, segment). On the worked table: new users +0.20 (email), loyal +0.02 (skip), churn-risk −0.07 (never email — treatment backfires). An uplift programme needs a genuine holdout control.
Outcome modeling (target argmax P(action)) targets sure things and wastes budget; uplift modeling (target argmax change in P(action)) targets persuadables. Predicting the outcome is not predicting the effect — the pillar's correlation-vs-causation lesson applied to a budget.
Classifier metrics reward finding people who act; a Qini curve scores incremental effect — rank by predicted uplift, plot cumulative extra actions vs fraction treated, and read the area over the random diagonal as the model's worth.

Next: Reinforcement Learning (P9) is where the Machine Learning domain closes — the pillar that drops "the data is given" and lets an agent's own actions generate its training signal. It is planned, and its sprint is next (not yet live). For now, step back to the Causal Inference & Experimentation overview to see how experimentation and causal methods fit together.

Ask the mentor about this module

Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.

Ctrl/Cmd + Enter to send

Try it yourself

A scratch console for this page's ideas — ungraded, nothing you run here is recorded.

Scratch console

A scratch console with the scientific stack (pandas, numpy, scikit-learn). Runs on the server — no network, resource-limited and measured.

Output appears here.

Where next?

Continue

Learning from Reward →

RL Foundations · P9

Go up a level