Testing Probability Calibrations: Application to Credit Scoring Models

The validation of probability calibration is an inherently difficult task. We develop a testing procedure for credit-scoring models. The models comprise two components to check whether the ex-ante probabilities support the ex-post frequencies. The first component tests the level of the probability calibration under dependencies. In the long term, the number of events should equal the sum of assigned probabilities. The second component validates the shape, measuring the differentiation between high and low probability events. We construct a goodnessof- fit statistic for both level and shape together with a global statistics, which is asymptotically x2-distributed.