Robust, accurate confidence intervals with a weak instrument: quarter of birth and education

An instrument or instrumental variable manipulates a treatment and affects the outcome only indirectly through its manipulation of the treatment. For instance, encouragement to exercise might increase cardiovascular fitness, but only indirectly to the extent that it increases exercise. If instrument levels are randomly assigned to individuals, then the instrument may permit consistent estimation of the effects caused by the treatment, even though the treatment assignment itself is far from random. For instance, one can conduct a randomized experiment assigning some subjects to 'encouragement to exercise' and others to 'no encouragement' but, for reasons of habit or taste, some subjects will not exercise when encouraged and others will exercise without encouragement; none-the-less, such an instrument aids in estimating the effect of exercise. Instruments that are weak, i.e. instruments that have only a slight effect on the treatment, present inferential problems. We evaluate a recent proposal for permutation inference with an instrumental variable in four ways: using Angrist and Krueger's data on the effects of education on earnings using quarter of birth as an instrument, following Bound, Jaeger and Baker in using simulated independent observations in place of the instrument in Angrist and Krueger's data, using entirely simulated data in which correct answers are known and finally using statistical theory to show that "only" permutation inferences maintain correct coverage rates. The permutation inferences perform well in both easy and hard cases, with weak instruments, as well as with long-tailed responses. Copyright 2005 Royal Statistical Society.