Evaluating Relative Mode Effects in Mixed-Mode Surveys:

In order to investigate the advantage of mixed-mode (MM) surveys, selection effects between the modes should be evaluated. Selection effects refer to differences in respondent compositions on the target variables between the modes. However, estimation of selection effects is not an easy task because they may be completely confounded with measurement effects between the modes (differences in measurement error). Publications concerning the estimation of these mode effects are scarce. This article presents and compares three methods that allow measurement effects and selection effects to be evaluated separately. The first method starts from existing publications that avoid the confounding problem by introducing a set of mode-insensitive variables into the analysis model. However, this article will show that this method involves unrealistic assumptions in most practical research. The second and the third methods make use of an MM sample extended by comparable single-mode data. The assumptions, advantages, and disadvantages of all three methods are discussed. Each method will further be illustrated using a set of six variables relating to opinions about surveys among the Flemish population. The results show large differences between the methods.