ARCH models for multi-period forecast uncertainty-a reality check using a panel of density forecasts

We develop a theoretical framework to compare forecast uncertainty estimated from time series models to those available from survey density forecasts. The sum of the average variance of individual densities and the disagreement, which is the same as the variance of the aggregate density, is shown to approximate the predictive uncertainty from well specified time series models when the variance of the aggregate shocks is relatively small compared to that of the idiosyncratic shocks. We argue that due to grouping error problems, compositional effects of the panel, and other complications, the uncertainty measure has to be estimated from individual densities. Despite numerous reservations on the credibility of time series based measures of forecast uncertainty, we found that during normal times the uncertainty estimates based on ARCH models simulate the subjective survey measure remarkably well. However, during times of regime change and structural break, the two estimates do not overlap. We suggest ways to improve the time series measures during periods when forecast errors are apt to be large. The disagreement series is a good indicator of such periods.