Testing for Heterogeneous Treatment Effects in Experimental Data:False Discovery Risks and Correction Procedures

Randomization has emerged as preferred empirical strategy for researchers in a variety of fieldsover the past years. While the advantages of RCTs in terms of identification are obvious, thestatistical analysis of experimental data is not without challenges. In this paper we focus onmultiple hypothesis testing as one statistical issue commonly encountered in economic research.In many cases, researchers are not only interested in the main treatment effect, but also want toinvestigate the degree to which the impact of a given treatment varies across specific geographicor socio-demographic groups of interest. In order to test for such heterogeneous treatmenteffects, researchers generally either use subsample analysis or interaction terms. While bothapproaches have been widely applied in the empirical literature, they are generally not validstatistically, and, as we demonstrate in this paper, lead to an almost linear increase in thelikelihood of false discoveries...