Human-Algorithm Ensembles

An “ensemble” approach to decision making involves aggregating the results from different decision makers solving the same problem (i.e., without specialization). We draw on the literature on ensemble decision making in machine learning-based Artificial Intelligence (AI) as well as among human decision makers to propose conditions under which human-AI ensembles can be useful. We argue that human and AI-based algorithmic decision making can be ensembled even when neither has a clear advantage over the other (in terms of predictive accuracy) at a decision task or its sub-components, and even if neither alone can attain satisfactory accuracy in absolute terms. Many managerial decisions have these attributes, and division of labor between humans and AI algorithms is usually ruled out in such contexts because the conditions for specialization are not met. However, we propose that human-AI ensembling is still a possibility under the conditions we identify