Machine Learning and Prediction Errors in Causal Inference

Machine learning is a growing method for causal inference. In machine learning settings, prediction errors are a commonly overlooked problem that can bias results and lead to arbitrarily incorrect parameter estimates. We consider a two-stage model where (1) machine learning is used to predict variables of interest, and (2) these predictions are used in a regression model for causal inference. Even when the model specification is otherwise correct, traditional metrics such as p-values and first-stage model accuracy are not good signals of correct second-stage estimates when prediction error exists. We show that these problems are substantial and persist across simulations and an empirical dataset. We provide consistent corrections for the case where unbiased training data is available for the machine learning dataset