Causal Classification : Treatment Effect vs. Outcome Prediction

The goal of causal classification is to identify individuals whose outcome would be positively changed by a treatment. Examples include targeting advertisements and targeting retention incentives to reduce churn. Causal classification is challenging because we observe individuals under only one condition (treated or untreated), so we do not know who was influenced by the treatment, but we may estimate the potential outcomes under each condition to decide whom to treat by estimating treatment effects. Curiously, we often see practitioners using simple outcome prediction instead, e.g., will someone purchase if shown the ad? Rather than disregarding this as naive behavior, we present a theoretical analysis comparing treatment effect estimation and outcome prediction when addressing causal classification. We describe a causal bias-variance tradeoff: because treatment effect estimation depends on two outcome estimates, its larger variance may lead to more errors than the (biased) outcome prediction approach. Moreover, the often stronger signal-to-noise ratio in outcome prediction may help with intervention decisions. The theoretical results as well as large-scale simulations illustrate settings in which outcome prediction should be better, including cases where (1) the bias may be partially corrected by choosing a different threshold, (2) outcomes and treatment effects are positively correlated, and (3) data to estimate counterfactuals are limited