Cultural Context in Standardized Tests

We report results from a field experiment on cultural context in standardized tests among 6th- and 8th-grade school students in Australia. The National Assessment Program Literacy and Numeracy (NAPLAN) is a series of basic-skills tests given to Australian students. In our experiment, 1135 students in Dubbo – a regional area in the North-Western part of the state of New South Wales – were randomly assigned to either a regular NAPLAN test or a contextualized test designed specifically for this experiment by the NSW Aboriginal Education Consultative Group — a not-for-profit Aboriginal organization. The contextualized test was specifically designed to mimic the regular test, but adapted to the local context of Dubbo. We evaluate effects on tests scores in numeracy for grades 6 and 8, and reading for grade 6. In numeracy, we do not find robust evidence of an impact on test scores. In reading, we find qualitatively large effects. The average treatment effect for reading is 0.27 s.d., with higher effects for Indigenous students (0.30 s.d.) than non-Indigenous students (0.24 s.d.) Together these results imply that cultural context may be important for performance on certain types of basic-skills tests