On comparing the accuracy of default predictions in the rating industry

We consider 1927 borrowers from 54 countries who had a credit rating by both Moody's and S&P at the end of 1998, and their subsequent default history up to the end of 2002. Viewing bond ratings as predicted probabilities of default, we consider partial orderings among competing probability forecasters and show that Moody's and S&P cannot be ordered according to any of these. Therefore, the relative performance of the agencies depends crucially on the way in which probability predictions are compared.