Reality Check : Combining Survey and Market Data to Estimate Choice Models

Discrete choice models estimated using hypothetical choices made in a survey setting (e.g., choice-based conjoint) are widely used to forecast the effects of product design and marketing mix decisions. Survey methods allow the researcher to estimate preferences for product features that do not yet exist in the market. However, parameters estimated from survey data often show marked inconsistencies with marginal effects inferred from the market, reducing their usefulness in forecasting and decision making. Several methods for adjusting survey-based choice models so that they more accurately predict market share have been suggested, but existing calibration methods are ad hoc and may change parameter values in ways that render them less consistent with other key empirical features of the data. We propose a new approach that produces more market-consistent parameter estimates by combining individual-level purchase data from the market with survey choice data in the formal estimation process.The proposed method relies on a new general framework for combining two or more sources of choice data to estimate a hierarchical discrete choice model. Past approaches to combining choice data assume that the population mean for the parameters is the same across both data sets and require that data sets are sampled from the same population. In contrast, we incorporate individual demographic and product-use variables into the model and assert only that the mapping between individuals' demographics and their preferences is the same across the data sets. This allows the model to accommodate differences in choice behavior across the data sets driven by differences in observed demographics. The framework also explicitly incorporates a model for the individual demographics which allows us to use Bayesian missing data techniques to handle the situation where each data set contains different demographic variables. This makes the method useful in practice for a wide range of existing market and survey data sets. We apply the method to a set of conjoint and market data for minivan choice and find that the proposed method predicts holdout market choices better than a model estimated from conjoint data alone or a model that does not include demographic variables