What can we learn from student performance measures? : identifying treatment in the presence of curves and letter grades

Grade-based performance measures are often relied on when considering the efficacy of education-related policy interventions. Yet, it is common for measures of student performance to be subjected to curves and discretized through letter-grade transformations. We show how transformed grades systematically challenge causal identification. Even without explicit curving, transformations to letter grade are particularly problematic and yield treatment estimates that are weighted combinations of inflated responsiveness around letter thresholds and "zeros" away from these thresholds. Curving practices can also introduce false patterns of treatment heterogeneity, attenuating measured responses to treatment among high-performing students, for example, or inflating measured responses among low-performing students.