Evaluating Neural Spatial Interaction Modelling by Bootstrapping

This paper exposes problems of the commonly used technique of splitting the available data in neural spatial interaction modelling into training, validation, and test sets that are held fixed and warns about drawing too strong conclusions from such static splits. Using a bootstrapping procedure, we compare the uncertainty in the solution stemming from the data splitting with model specific uncertainties such as parameter initialization. Utilizing the Austrian interregional telecommunication traffic data and the differential evolution method for solving the parameter estimation task for a fixed topology of the network model [ i.e. J = 9] this paper illustrates that the variation due to different resamplings is significantly larger than the variation due to different parameter initializations. This result implies that it is important to not over-interpret a model, estimated on one specific static split of the data.