Stochastic choice and imperfect judgments of line lengths: What is hiding in the noise?

Noise is a pervasive feature of economic choice. However, standard economics experiments are not well equipped to study the noise because experiments are constrained: preferences are either unknown or only imperfectly measured by experimenters. As a result of these designs--where the optimal choice is not observable to the analyst--many important questions about the noise in apparently random choice cannot be addressed. We design an experiment to better understand stochastic choice by directing subjects to make incentivized binary choices between lines. Subjects are paid a function of the length of the selected line, so subjects will attempt to select the longer of the lines. We find a gradual (not sudden) relationship between the difference in the lengths of the lines and the optimal choice. Our analysis suggests that the errors are better described as having a Gumbel distribution rather than a normal distribution, and our simulated data increase our confidence in this inference. We find evidence that suboptimal choices are associated with longer response times than optimal choices, which appears to be consistent with the predictions of Fudenberg, Strack, and Strzalecki (2018). Although we note that the relationship between response time and the optimality of choice becomes weaker across trials. In our experiment, 54 of 56 triples are consistent with Strong Stochastic Transitivity and this is the median outcome in our simulated data. Finally, we find a relationship between choice and attention, although we find strong evidence that the relationship is endogenous.