Instrumental variables

Setup#

pip install numpy

Verified with numpy 1.26.4.

Recovering the truth without measuring the confounder#

Diligence drives class attendance and marks — and stays out of the dataset for the analysis. The voucher is random. True effect: +3 marks per class.

import numpy as np

rng = np.random.default_rng(18)
n = 20_000
diligence = rng.normal(0, 1, n)                  # unobserved: drives classes AND marks
voucher = (rng.random(n) < 0.5).astype(float)    # a randomly posted discount coupon
classes = 2 + 1.5 * diligence + 2.0 * voucher + rng.normal(0, 1, n)
marks = 40 + 3.0 * classes + 8.0 * diligence + rng.normal(0, 5, n)   # truth: +3 per class

ols = np.polyfit(classes, marks, 1)[0]
v = voucher == 1
wald = (marks[v].mean() - marks[~v].mean()) / (classes[v].mean() - classes[~v].mean())
print(f"plain regression:    {ols:+.2f} marks per class")
print(f"instrumented (Wald): {wald:+.2f} marks per class  (truth: +3.00)")

plain regression:    +5.84 marks per class
instrumented (Wald): +2.79 marks per class  (truth: +3.00)

The walkthrough#

Plain regression nearly doubles the effect: +5.84. Diligent students take more classes and score higher regardless; linear regression hands attendance the credit for diligence's work. No amount of data repairs this — the bias is structural.

The Wald estimator divides two clean differences. Voucher randomness makes both the marks gap (numerator) and the classes gap (denominator) causal effects of the voucher. Their ratio, +2.79, is the per-class effect — within noise of +3.00, with diligence never entering the code after data generation.

Beyond two groups: two-stage least squares (2SLS). Stage one regresses treatment on the instrument and keeps the predicted treatment — the portion of attendance the voucher explains, scrubbed of diligence. Stage two regresses the outcome on that prediction. With a single binary instrument, 2SLS collapses to the Wald ratio above. Production users reach for linearmodels.iv.IV2SLS (the standard-error bookkeeping is genuinely fiddly by hand).

Watch the denominator. The voucher moved attendance by 2 classes — a strong nudge. Had it moved attendance by 0.05, the ratio would divide by a whisper: tiny denominators amplify noise and bias catastrophically. This is the weak instrument problem, and checking the nudge's strength comes before believing any IV number.

Common mistakes#

Falling in love with a weak instrument. A nudge that barely moves treatment produces estimates that are wild and biased toward the plain regression — the worst of both worlds. Convention demands a strong first stage (the classic bar: F-statistic above 10; modern practice asks for far more).

Exclusion violations by side channel. If vouchers made families feel wealthier and buy books, marks moved through a second road, and the ratio silently absorbs it. Exclusion cannot be tested from this data — argue it, and stress-test the argument.

Forgetting whose effect you estimated. The ratio describes compliers — students whose attendance the voucher changed. The always-attenders and never-attenders contributed nothing. If compliers benefit differently from others, the IV answer is theirs, not everyone's.

Instrumenting with something the confounder touches. "Distance to the coaching centre" fails if diligent families chose houses near coaching centres. An instrument correlated with the confounder is no instrument.

Try it yourself#

Weaken the voucher to 0.1 * voucher and rerun with several seeds — watch the Wald estimate swing wildly around (and away from) the truth. Then break exclusion: add + 2.0 * voucher directly into marks and see the estimate absorb it. Two failure modes, both invisible from the output alone — which is the entire IV cautionary tale.

What to learn next#

• Uplift modelling — from average effects to per-person targeting.
• Regression discontinuity — the cutoff design that is secretly an IV.
• Double machine learning — ML-strength nuisance models under orthogonal moments.