Broadening the Tests of Learning Models

For many years psychological studies of the learning process have used a simulated medical diagnosis task in which symptom configurations are probabilistically related to diseases. Participants are given a set of symptoms and asked to indicate which disease is present, and feedback is given on each trial. We enrich this standard laboratory task in four different ways. First, the symptoms have four possible values (low, medium high, and high) rather than just two. Second, symptom configurations are generated from an expanded factorial design rather than a simple factorial design. Third, subject are asked to make a continuous judgment indicating their confidence in the diagnosis, rather than simply a binary judgment. Fourth, cumulated performance scores, payoffs, and the availability of a historical summary of the outcomes are varied in order to assess how these treatment modulate performance. These enrichments provide a broader data set and more challenging tests of models. Using 123 subjects each in 480 trials, we compare five existing learning models plus several variants, including the well-known Bayesian, Fuzzy Logic, connectionist, exemplar and ALCOVE models. We find that subjects do learn to distinguish the symptom configurations, that subjects are quite heterogeneous in their response to the task, and that only a small part of the variation across subjects arises from the differences in treatments. The most striking finding is that the model that best predicts subjects' behavior is a simple Bayesian model with a singe fitted parameter for prior precision to capture individual differences. We use recent rolling regression techniques to elucidate the behavior of this model over time and find some evidence of overresponse to current stimuli