Statistical Power for Random Assignment Evaluations of Education Programs

This paper examines theoretical and empirical issues related to the statistical power of impact estimates for experimental evaluations of education programs. It considers designs in which random assignment is conducted at the school, classroom, or student level, using a unified analytic framework based on statistical methods from the literature. Focusing on standardized test scores of elementary school students, the author discusses appropriate precision standards, and, for each design, the required number of schools to achieve these standards. Clustering effects vary by design but are typically large; consequently, large school samples are required, suggesting that most impact studies will only be able to address broad questions rigorously.