Category effects on stimulus estimation: Shifting and skewed frequency distributions - A reexamination

Duffy, Huttenlocher, Hedges, and Crawford (2010) [Psychonomic Bulletin & Review, 17(2), 224-230] report on experiments where participants estimate the lengths of lines. These studies were designed to test the Category Adjustment Model (CAM), a Bayesian model of judgments. CAM predicts that there will exist a bias toward the running mean of the lines and that judgments will not be differentially affected by recent stimuli. The authors report that their analysis provides evidence consistent with CAM. We reexamine their data. First, we attempt to replicate their analysis and we obtain different results. Second, we conduct a different statistical analysis. We find significant recency effects and we identify several specifications where the running mean is not significantly related to judgment. Third, we conduct a test of an auxiliary prediction of CAM: that the bias towards the mean will increase with exposure to the distribution. We do not find such a relationship. Fourth, we produce a simulated dataset that is consistent with CAM and our methods correctly identify it as consistent with CAM. We conclude that the Duffy et al. (2010) dataset is not consistent with CAM. We also discuss how conventions in psychology do not sufficiently reduce the likelihood of these mistakes in future research.