Interactive experiments in Toloka

Even before the COVID-19 pandemic, the popularity of online behavioral experiments grew steadily. Due to lockdowns, online studies often became the only available option for behavioral economists, sociologists, and political scientists. The use of the most well-known platforms, such as mTurk, was so intensive that the quality of data was harmed. However, even before the pandemic-induced quality crisis, online studies were limited in scope; real-time interactions between participants were hard to achieve due to the large proportion of drop-outs and issues with creating stable groups. Using the relatively unknown crowdsourcing platform, Toloka, we successfully ran several multi-round interactive experiments. Toloka’s sizeable online population, fairly low exposure of participants to sociological surveys and behavioral studies, and convenient application programming interface can make it a useful addition to the toolbox of an experimentalist who needs to run behavioral studies that require real-time interactions between participants.