Matching & Stratification

P8.causal-methods.02 · Audience: guest, it-ml, language-pro · Prerequisites: Confounders & Causal Diagrams

Confounders & Causal Diagrams Difference-in-Differences

The previous module diagnosed the disease — a confounder opens a backdoor path and can flip a result's sign — and hinted at the cure: read the effect within each stratum. This module turns that hint into a working method. Matching and stratification are two faces of one idea: instead of hoping the treated and control groups are comparable, force the comparison to be like-with-like by hand. We will match a toy cohort exactly, watch a confounded estimate shrink to its honest value, and end on the method's hard limit — the confounder you did not measure.

Step 1 / 5 — Compare like with like

Randomization works because the coin flip, applied to a big enough group, hands the treated and control arms populations that look alike in every respect — measured or not — so any outcome gap must be the treatment. Observational data denies you the coin. Matching is the manual substitute: for each treated unit, find control units that look the same on the confounders, and compare only those. You are rebuilding, by hand, the balance the coin would have given you.

ⓘ Concept: Matching

for each treated unit, pair it with control unit(s) sharing its confounder values; the effect is the average within-pair outcome difference

Why it matters — Comparing the raw treated average to the raw control average answers the wrong question when the groups differ to begin with — you measure the treatment plus every way the groups were already unalike. Matching restricts the comparison to look-alike units, so the confounder is held roughly constant across the comparison and the outcome gap is attributable to treatment instead.

The catch that shapes everything below: you can only match on what you have measured. Matching balances the columns in your table. Randomization balances the columns you never thought to record. That gap is the last step of this module.

✅ Key takeaways

Matching rebuilds by hand the like-with-like balance that randomization gives for free: compare each treated unit only against controls sharing its confounder values.
On the worked cohort, the naive effect +4.8 was inflated by confounding; exact matching within engagement strata recovered the honest effect of +2 (both strata agreed at +2).
Stratification is matching in bulk — effect within each stratum, then a population-weighted average; the weighting decides whose effect you are reporting.
The propensity score (P(treated | covariates)) collapses many confounders into one number so matching scales past one or two columns — but it only balances the confounders it was given.
Every matching method balances MEASURED confounders only. An unmeasured confounder keeps its backdoor open, which is the wall randomization clears and matching cannot — state your assumptions and test their sensitivity.

Next: P8.causal-methods.03 — Difference-in-Differences

Ask the mentor about this module

Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.

Ctrl/Cmd + Enter to send

Try it yourself

A scratch console for this page's ideas — ungraded, nothing you run here is recorded.

Scratch console

A scratch console with the scientific stack (pandas, numpy, scikit-learn). Runs on the server — no network, resource-limited and measured.

Output appears here.

Where next?

Continue

Difference-in-Differences →

Causal Methods · P8

Later in Causal Methods

This module unlocks

Uplift & Heterogeneous Effects

Go up a level