Service Interruptions in Large-Scale Service Systems

Large-scale service systems, where many servers respond to high demand, are appealing because they can provide great economy of scale, producing a high quality of service with high efficiency. Customer waiting times can be short, with a majority of customers served immediately upon arrival, while server utilizations remain close to 100%. However, we show that this confluence of quality and efficiency is not achieved without risk, because there can be severe congestion if the system does not operate as planned. In particular, we show that the large scale makes the system more vulnerable to service interruptions when (i) most customers remain waiting until they can be served, and (ii) when many servers are unable to function during the interruption, as may occur with a system-wide computer failure. Increasing scale leads to higher server utilizations, which in turn leads to longer recovery times from service interruptions and worse performance during such events. We quantify the impact of service interruptions with increasing scale by introducing and analyzing approximating deterministic fluid models. We also show that these fluid models can be obtained from many-server heavy-traffic limits.