From Accuracy to Interpretability : Understanding Neural Networks in Forecast

Black-box algorithms with outstanding performance have been widely used in various fields; however, the lack of interpretability leads to great difficulties in troubleshooting and model improvement, hence severely confining the practical application. In response, we propose a framework that improves the interpretability of existing neural network algorithms without the loss of accuracy, and demonstrate how it works through a case of the sales forecast. Concretely, we first build an interpretable forecasting model by incorporating the key influencing factors into the original black-box model, which provides potential explanatory variables in addition to the original outputs. Afterward, we build a tree-based global surrogate in terms of the explanatory variables and implement TreeSHAP on the surrogate to explain the surrogate globally and locally. Further analysis comparing the explanations with background domain knowledge implies potential deficiencies for the surrogate and thereby the original forecasting algorithm, and reveals the prospect of model improvement. Overall, through extensive numerical experiments on the real data of JD.com, we validate the framework in effectiveness and feasibility and emphasize the critical position of interpretability when applying the black-box models in practice