The Impact of Utility Balance and Endogeneity in Conjoint Analysis

Adaptive metric utility balance is at the heart of one of the most widely used and studied methods for conjoint analysis. We use formal models, simulations, and empirical data to suggest that adaptive metric utility balance leads to partworth estimates that are relatively biased—smaller partworths are upwardly biased relative to larger partworths. Such relative biases could lead to erroneous managerial decisions. Metric utility-balanced questions are also more likely to be inefficient and, in one empirical example, contrary to popular wisdom, lead to response errors that are at least as large as nonadaptive orthogonal questions. We demonstrate that this bias is because of endogeneity caused by a “winner’s curse.” Shrinkage estimates do not mitigate these biases. Combined with adaptive metric utility balance, shrinkage estimates of heterogeneous partworths are biased downward relative to homogeneous partworths. Although biases can affect managerial decisions, our data suggest that, empirically, biases and inefficiencies are of the order of response errors. We examine viable alternatives to metric utility balance that researchers can use without biases or inefficiencies to retain the desired properties of (1) individual-level adaptation and (2) challenging questions.