Publications
Publications
- Management Science
Trimmed Opinion Pools and the Crowd's Calibration Problem
By: Victor Richmond R. Jose, Yael Grushka-Cockayne and Kenneth C. Lichtendahl
Abstract
We introduce an alternative to the popular linear opinion pool for combining individual probability forecasts. One of the well-known problems with the linear opinion pool is that it can be poorly calibrated. It tends toward underconfidence as the crowd's diversity increases, i.e., as the variance in the individuals' means increases. To address this calibration problem, we propose the exterior-trimmed opinion pool. To form this pool, forecasts with low and high means, or cumulative distribution function (cdf) values, are trimmed away from a linear opinion pool. Exterior trimming decreases the pool's variance and improves its calibration. A linear opinion pool, however, will remain overconfident when individuals are overconfident and not very diverse. For these situations, we suggest trimming away forecasts with moderate means or cdf values. This interior trimming increases variance and reduces overconfidence. Using probability forecast data from U.S. and European Surveys of Professional Forecasters, we present empirical evidence that trimmed opinion pools can outperform the linear opinion pool.
Keywords
Citation
Jose, Victor Richmond R., Yael Grushka-Cockayne, and Kenneth C. Lichtendahl. "Trimmed Opinion Pools and the Crowd's Calibration Problem." Management Science 60, no. 2 (February 2014): 463–475.