"Try to balance the baseline" : a comment on "Parent-teacher meetings and student outcomes: evidence from a developing country" by Islam (2019)

Islam (2019) reports results from a randomized field experiment in Bangladesh that examines the effects of parent-teacher meetings on student test scores in primary schools. The reported findings suggest strong positive effects across multiple subjects. In this report, we demonstrate that the school-level randomization cannot have been conducted as the author claims. Specifically, we show that the nine included Bangladeshi unions all have a share of either 0% or 100% treated or control schools. Additionally, we uncover irregularities in baseline scores, which for the same students and subjects vary systematically across the author’s data files in ways that are unique to either the treatment or control group. We also discovered data on two unreported outcomes and data collected from the year before the study began. Results using these data cast further doubt on the validity of the original study. Moreover, in a survey asking parents to evaluate the parent-teacher meetings, we find that parents in the control schools were more positive about this intervention than those in the treated schools. We also find undisclosed connections to two additional RCTs.