Machine Learning Informed Decision-Making : A Field Intervention in Public Employment Service

As machine learning (ML) systems are getting used for making high-stakes decisions affecting people, there is a growing demand and focus on making these systems more interpretable and explainable. The assumption is that more explainable ML models can improve the decision-making process when humans are in the decision-making loop. While there is a lot of recent methodological work in this area, there is little rigorous empirical evidence if and how the added information (such as explanations) impacts human decision-making in real-world problems. Using quantitative and qualitative analysis of a field intervention in a public employment service setting with unemployment counselors, we find that the explanations have a small positive effect on confidence and a somewhat negative effect on the quality of the decisions. Also, we find that human decision-makers rely on the ML models more when there is no means to reduce their uncertainties about individual cases or when their personal heuristic does not hold due to a significant change in the environment, leading to a potential over-reliance on ML model recommendations when they are prone to be wrong