The Late Pretest Problem in Randomized Control Trials of Education Interventions

Pretest and post-test experimental designs are often used in randomized control trials (RCTs) in the education field to improve the precision of the estimated treatment effects. For logistic reasons, however, pretest data are often collected after random assignment, so that including them in the analysis could bias the post-test impact estimates. Deciding whether to collect and use late pretest data in RCTs involves a variance-bias tradeoff. This paper addresses this issue both theoretically and empirically for several commonly used impact estimators, using a loss function approach grounded in the causal inference literature. For RCTs of interventions to improve student test scores, estimators that include late pretests will typically be preferred to estimators that exclude them or that instead include uncontaminated baseline test score data from other sources. This result holds as long as test score impacts do not grow very quickly early in the school year.