Are Vaccination Campaigns Misinformed? : Experimental Evidence from COVID-19 in Low- and Middle-Income Countries

Routine immunization coverage estimated in surveys often substantially differs from figures reported in administrative records, presenting a dilemma for researchers and policy makers. Using high-frequency phone surveys and administrative records from government sources in 36 low- and middle-income countries, this paper shows that such misalignment has also been common in the case of COVID-19. Across the sample, survey estimates exceed administrative figures by 47 percent on average, at times suggesting markedly different policy conclusions depending on the data source consulted. This pattern is particularly stark and consistent in Sub-Saharan Africa. To investigate the sources of this discrepancy, the paper presents results from six methodological experiments that vary survey design choices and documents their effect on estimated COVID-19 vaccine coverage. The results show that design choices matter, in particular the selection of respondents to be interviewed. However, phone survey estimates prove remarkably robust to several commonly claimed biases. After accounting for observed errors of representation and measurement in the survey data, there remains a nonnegligible, unexplained residual gap with administrative records. The paper provides indicative evidence of flaws and weaknesses in administrative data recording and reporting that affect reported vaccination rates and could contribute to this gap. The findings matter for past research on COVID-19 vaccination, future immunization efforts, and the design of robust data production systems on health topics