On Not Evaluating Economic Models by Forecast Outcomes

Even in scientific disciplines, forecast failures occur.  Four possible states of nature (a model is good or bad, and it forecasts well or badly) are examined using a forecast-error taxonomy, which traces the many possible sources of forecast errors.  This analysis shows that a valid model can forecast badly, and a poor model can forecast successfully.  Delineating the main causes of forecast failure reveals transformations that can correct failure without altering the 'quality' of the model in use.  We conclude that judging a model by the accuracy of its forecasts is more like fools' gold than a gold standard.