Demand Estimation with Machine Learning and Model Combination

We survey and apply several techniques from the statistical and computer science literature to the problem of demand estimation. We derive novel asymptotic properties for several of these models. To improve out-of-sample prediction accuracy and obtain parametric rates of convergence, we propose a method of combining the underlying models via linear regression. Our method has several appealing features: it is robust to a large number of potentially-collinear regressors; it scales easily to very large data sets; the machine learning methods combine model selection and estimation; and the method can flexibly approximate arbitrary non-linear functions, even when the set of regressors is high dimensional and we also allow for fixed effects. We illustrate our method using a standard scanner panel data set to estimate promotional lift and find that our estimates are considerably more accurate in out of sample predictions of demand than some commonly used alternatives. While demand estimation is our motivating application, these methods are likely to be useful in other microeconometric problems.