Ranking of VaR and ES Models: Performance in developed and emerging markets

An inherent problem with comparing and ranking competing Value at Risk (VaR) and Expected shortfall (ES) models is that they measure only a single realization of the underlying data generation process. The question is whether there is any significant statistical difference in the performance of different models. Is it only a matter of chance that in a particular market and in particular time period a certain model performs better than some other? It all comes down to a question whether something that we subjectively perceive as different is actually statistically different. We introduce a new methodology for ranking and comparing the performance of VaR and ES models based on a nonparametric ANOVA test. The relative performance of VaR and ES models is analysed using daily returns for sixteen stock market indices (eight each from developed and emerging markets) prior to and during the global financial crisis. Results show that for a large number of different models there is no statistically significant difference in their performance. The top performers are conditional extreme value GARCH model, models based on volatility updating and nonparametric mirrored historical simulation. ES backtesting results are similar to VaR results with the models being even more closely matched. The same models that were the top performers in VaR comparison also perform significantly better in ES estimation.