Theory Competition and Selectivity: Are All Economic Facts Greatly Exaggerated?

There is growing concern and mounting evidence of selectivity in empirical economics. Most empirical economic literatures have a skewed (or truncated) distribution of results. The aim of this paper is to explore the links between publication selectivity and theory competition. In research areas where theory supports a wide range of outcomes, empirical evidence is less likely to be affected by selectivity. However, in those areas where theory is consistent with only one qualitative effect (e.g, a negative effect of price on quantity demanded), selectivity is more likely and its bias, more severe. This hypothesis is supported through the analysis of 65 distinct empirical economics literatures, involving approximately two thousand separate empirical studies, which in turn collectively contain many more thousands of estimates. Our meta-meta-analysis shows that publication selection is widespread, but not universal. It distorts scientific inference with potentially adverse effects on policy making, but competition and debate between rival theories reduces this selectivity and thereby improves economic inference. All literature reviews, whether traditional or quantitative (meta-analysis), need to adopt explicit selection correction methods.