Automated Chat Application Surveys Using Whatsapp: Evidence from Panel Surveys and a Mode Experiment

We present a method to conduct automated surveys over WhatsApp, a globally popular messaging service. WhatsApp surveys offer potential advantages since they incur relatively low costs to respondents and researchers, are easy to use for respondents who are already familiar with WhatsApp chat features, and facilitate continued engagement with mobile populations as users can retain their WhatsApp number even if they change SIM cards and phone numbers. Yet, there is limited knowledge on how well WhatsApp surveys perform empirically. We test the WhatsApp method using two original panel surveys of refugees in Colombia and the U.S. and find satisfactory response rates and retention over a nine-month follow-up period in these mobile populations. We also conduct a mode experiment in Colombia comparing WhatsApp to short message service (SMS) and interactive voice response (IVR) surveys. We find that WhatsApp had a 12 and 27 percentage points higher response rate than IVR and SMS, respectively, resulting from higher initial engagement and lower break-off. We conclude by discussing advantages and limitations of the WhatsApp method and offer documentation and a public code repository to support researchers and practitioners in applying the method in other contexts.