On a quest for robustness : about model risk, randomness and discretion in credit risk stress tests

In this paper we study the impact of model uncertainty, which occurs when linking a stress scenario to default probabilities, on reduced-form credit risk stress testing. This type of uncertainty is omnipresent in most macroeconomic stress testing applications due to short time series for banks' portfolio risk parameters and highly collinear macroeconomic covariates. We quantify the effect of model uncertainty on supervisory and bank stress tests in terms of predicted portfolio loss distributions and implied capital shortfalls by conducting a full-edged top-down credit risk stress test for over 1,500 German banks. Our results suggest that the impact of model uncertainty on predicted capital shortfalls can be huge, even among models with similar predictive power. This leaves both banks and supervisors with uncertainty when calculating stress impacts and implied capital requirements. To mitigate the impact of uncertainty, we suggest a modeling approach which filters the model space by combining the standard Bayesian model averaging (BMA) paradigm with a structural filter derived from the Merton/Vasicek credit risk model. Applying our stress testing framework, the dispersion decreases and the median stress effect is reduced from -5.0pp of CET1 ratio under the BMA model to -2.5pp under the structurally augmented BMA model, while the predicted capital shortfall is reduced by 70 %. The structural filter eliminates extreme outcomes on both sides of the stress forecast distribution, leading in our application to the German banking sector to a reduction in impact compared to the model without the "stress testing plausibility" filter.