Modeling performance of internet-based services using causal reasoning

The performance of Internet-based services depends on manyserver-side, client-side, and network related factors. Often, theinteraction among the factors or their effect on service performanceis not known or well-understood. The complexity of these servicesmakes it difficult to develop analytical models. Lack of modelsimpedes network management tasks, such as predicting performance whileplanning for changes to service infrastructure, or diagnosing causesof poor performance. We posit that we can use statistical causal methods to modelperformance for Internet-based services and facilitate performancerelated network management tasks. Internet-based services arewell-suited for statistical learning because the inherent variabilityin many factors that affect performance allows us to collectcomprehensive datasets that cover service performance under a widevariety of conditions. These conditional distributions represent thefunctions that govern service performance and dependencies that areinherent in the service infrastructure. These functions anddependencies are accurate and can be used in lieu of analytical modelsto reason about system performance, such as predicting performance ofa service when changing some factors, finding causes of poorperformance, or isolating contribution of individual factors inobserved performance. We present three systems, What-if Scenario Evaluator (WISE), How toImprove Performance (HIP), and Network Access Neutrality Observatory(NANO), that use statistical causal methods to facilitate networkmanagement tasks. WISE predicts performance for what-if configurationsand deployment questions for content distribution networks. For this,WISE learns the causal dependency structure among the latency-causingfactors, and when one or more factors is changed, WISE estimateseffect on other factors using the dependency structure. HIP extendsWISE and uses the causal dependency structure to invert theperformance function, find causes of poor performance, and helpanswers questions about how to improve performance or achieveperformance goals. NANO uses causal inference to quantify the impactof discrimination policies of ISPs on service performance. NANO is theonly tool to date for detecting destination-based discriminationtechniques that ISPs may use.We have evaluated these tools by application to large-scaleInternet-based services and by experiments on wide-area Internet.WISE is actively used at Google for predicting network-level andbrowser-level response time for Web search for new datacenterdeployments. We have used HIP to find causes of high-latency Websearch transactions in Google, and identified many cases wherehigh-latency transactions can be significantly mitigated with simpleinfrastructure changes. We have evaluated NANO using experiments onwide-area Internet and also made the tool publicly available torecruit users and deploy NANO at a global scale.