Estimation of mode effects in the Health and Retirement Study using measurement models

Using multiple modes to collect data is becoming a standard practice in survey agencies. While this should save costs and decrease non-response error it may have detrimental effects on measurement quality. This can happen because different modes have distinct measurement biases which, when combined with selection effects, can increase the total survey error of a mixed-mode survey relative to a single mode approach. In this paper we use a quasi-experimental design from the Health and Retirement Study to compare the measurement quality of a number of scales between face-to-face, telephone and Web modes. Panel members were randomly assigned to receive a telephone survey or enhanced face-to-face survey in the 2010 core wave, while this was reversed in the 2012 core wave. In 2011, panelists with Internet access completed a Web survey containing selected questions from the core waves. We examine the responses from 3251 respondents who participated in all three waves, using latent models to identify measurement mode effects. Two of the scales, depression and physical activity, show systematic differences between interviewer administered modes (i.e., face-to-face and telephone) and the self-administered one (i.e., Web) while religiosity shows no differences of measurement between modes. Possible explanations are discussed.