Getting more Wisdom out of the Crowd : The Case of Competence-Weighted Aggregates

This paper shows that improving the calibration of individuals’ self-perceived competence increases the accuracy of competence-weighted, statistical aggregates to levels that match or even beat those of prediction markets. We conduct an experiment in which participants estimate quantities and state their self-perceived competence for various judgment problems. In addition, they engage in group discussions with other judges. We find that prior to participating in such group discussions, judges are poorly calibrated with respect to their own estimation accuracy, causing competence weighting to perform worse than prediction markets and simple averaging. However, the information exchange facilitated in group discussions improves judges’ calibration, yielding large performance gains of competence-weighted aggregates in subsequent and unrelated judgment problems. Our experimental results indicate that calibrating self-perceived competence provides a route to competence-weighted aggregates of and beyond prediction market quality