Monotonicity and Heteroskedasticity in Machine Learning Models : An Application to Credit Risk

In recent years, machine learning (ML) techniques have become very popular as, in many cases, they outperform other classic methods such as regressions. However, ML techniques usually have two main drawbacks as the marginal effects of the variables of the model may a) be counterintuitive, and b) have different signs for different observations. In this paper we use a real-life credit risk database to quantify these drawbacks and implement a simple methodology to solve them. We build two XGBoost models and show that the monotonic XGBoost has similar accuracy as the non-monotonic one and, it fixes the two main drawbacks of the ML models. Finally, we show that ML models may show heteroskedastic behaviour, meaning that some observations can be predicted with bigger/smaller accuracy. Using the database and the monotonic XGBoost we identify the observations that can be predicted with higher/lower accuracy