When to Sacrifice Prediction Accuracy : Machine Learning or MNL Choice Model for Assortment Planning

In revenue management, the customer discrete choice model is essential to describe customer purchase behavior. The multinomial logit (MNL) model is a classical random utility-based choice model that assumes that the consumer can purchase only one product from a set of substitute products. The revenue maximization problem of choosing the assortment under the MNL model must balance the cannibalization effect and nonpurchase probability. On the other hand, a large number of recommendation algorithms used in e-commerce, such as the DeepFM model with high prediction accuracy, tend to ignore the substitution effect among products. In this paper, we investigate whether and how better prediction accuracy transforms into better decisions for assortment planning. To answer this question, we compare MNL, DeepFM and a variant of DeepFM with the assortment information, called DeepFM-a. Instead utilizing the costly field experiment, we first use a real dataset of a flight browsing log and transaction records to train a machine learning model called Transformer, which has better prediction accuracy than MNL, DeepFM and DeepFM-a. Then, we utilize the trained Transformer model as a simulator to generate a synthetic dataset for consumer browsing and purchasing behavior. After training the MNL, DeepFM and DeepFM-a models, assortment decisions are given for simulated product pools with the three models. Then, the simulator is used to evaluate the revenue for each assortment from different choice models. Such a procedure utilizing the simulator can lessen the issue of validating decision models that change the observed data in the real world. Our findings are that a choice model with better prediction power may not yield higher revenue. When the outside option is less attractive, the MNL model provides comparable prediction power with much higher revenue. Fewer training data points reduce both prediction power and revenue for all three choice models. However, fewer features reduced the prediction power for all three choice models, but the assortment decision prescribed by DeepFM could increase the revenue