The regression tournament: A novel approach to prediction model assessment

Standard methods to assess the statistical quality of econometric models implicitly assume there is only one person in the world, namely the forecaster with her model(s), and that there exists an objective and independent reality to which the model predictions may be compared. However, on many occasions, the reality with which we compare our predictions and in which we take our actions is co-determined and changed constantly by actions taken by other actors based on their own models. We propose a new method, called a regression tournament, to assess the utility of forecasting models and taking these interactions into account. We present an empirical case of betting on Australian Rules Football matches where the most accurate predictive model does not yield the highest betting return, or, in our terms, does not win a regression tournament.