Improving short-term demand forecasting for short-lifecycle consumer products with data mining techniques

Today's economy is characterized by increased competition, faster product development and increased product differentiation. As a consequence product lifecycles become shorter and demand patterns become more volatile which especially affects the retail industry. This new situation imposes stronger requirements on demand forecasting methods. Due to shorter product lifecycles historical sales information, which is the most important source of information used for demand forecasts, becomes available only for short periods in time or is even unavailable when new or modified products are introduced. Furthermore the general trend of individualization leads to higher product differentiation and specialization, which in itself leads to increased unpredictability and variance in demand. At the same time companies want to increase accuracy and reliability of demand forecasting systems in order to utilize the full demand potential and avoid oversupply. This new situation calls for forecasting methods that can handle large variance and complex relationships of demand factors. This research investigates the potential of data mining techniques as well as alternative approaches to improve the short-term forecasting method for short-lifecycle products with high uncertainty in demand. We found that data mining techniques cannot unveil their full potential to improve short-term forecasting in this case due to the high demand uncertainty and the high variance of demand patterns. In fact we found that the higher the variance in demand patterns the less complex a demand forecasting method can be. Forecasting can often be improved by data preparation. The right preparation method can unveil important information hidden in the available data and decrease the perceived variance and uncertainty. In this case data preparation did not lead to a decrease in the perceived uncertainty to such an extent that a complex forecasting method could be used. Rather than using a data mining approach we found that using an alternative combined forecasting approach, incorporating judgmental adjustments of statistical forecasts, led to significantly improved short-term forecasting accuracy. The findings are validated on real world data in an extensive case study at a large retail company in Western Europe.