When Good Instruments Go Bad

This note examines the instrumental variables method used by Neumark, Zhang, and Ciccarella (2005) to analyze Wal-Mart's effect on retail labor markets, and exposes major flaws in that methodology. Neumark, Zhang, and Ciccarella use an interaction between distance from Wal-Mart's headquarters and time effects to predict Wal-Mart's presence in a county, and find that each Wal-Mart store destroys, on average, approximately 200 retail jobs. These findings are in stark contrast to Basker (2005) who found a small, but positive and statistically significant, effect on jobs. I show that the IV estimates obtained by Neumark, Zhang, and Ciccarella confound Wal-Mart's causal effect with other factors. To illustrate the problem, I show that their methodology implies a large impact of Wal-Mart not only on retail employment but also on county manufacturing employment. Reduced-form estimates of the regressions show statistically and economically indistinguishable effects in counties with and without Wal-Mart presence, implying that other factors are most likely driving the results.