On the Generalizability of Using Mobile Devices to Conduct Economic Experiments

Recent technological advances enable the implementation of online, field and hybrid experiments using mobile devices. Mobile devices enable sampling of incentivized decisions in more representative samples, consequently increasing the generalizability of results. Generalizability might be compromised, however, if the device is a relevant behavioural confound. This paper reports on a battery of classic economic games and tasks in which we randomize the decision-making device (computer versus mobile) and the laboratory setup (physical versus remote). Our results offer broad support for conducting decision experiments using mobile devices. For six out of eight tasks, we find robust null results in terms of average treatment effects and variability. This should give researchers confidence to conduct studies via mobile devices. However, we find two caveats. First, with respect to decisions, subjects using a mobile device are significantly more risk averse and offer less during bargaining. Second, decision response times and the time taken to read instructions are significantly shorter for the remote-mobile treatment. These qualifications suggest the importance of ensuring device consistency across treatments