Technical Methods Report The Estimation of Average Treatment Effects for Clustered RCTs of Education Interventions

This paper examines the estimation of two-stage clustered randomized controlled trial designs (RCTs) in education research using the Neyman causal inference framework that underlies experiments. The key distinction between the considered causal models is whether potential treatment and control group outcomes are considered to be fixed for the study population (the finite-population model) or randomly selected from a vaguely defined universe (the super-population model). Appropriate estimators are derived and discussed for each model. Using data from five large-scale clustered RCTs in the education area, the empirical analysis estimates impacts and their standard errors using the considered estimators. For all studies, the estimators yield identical findings concerning statistical significance. However, standard errors sometimes differ, suggesting that policy conclusions from RCTs could be sensitive to the choice of estimator. A key recommendation is that analysts test the sensitivity of their impact findings using different estimation methods and cluster-level weighting schemes.