A deterministic approach to small data set partitioning for neural networks

Several prominent data-mining studies have evaluated the performance of neural networks (NNs) against traditional statistical methods on the two-group classification problem in discriminant analysis. Although NNs often outperform traditional statistical methods, their performance can be hindered because of failings in the use of training data. This problem is particularly acute when using NNs on smaller data sets. A heuristic is presented that utilizes Mahalanobis distance measures (MDM) to deterministically partition training data so that the resulting NN models are less prone to overfitting. We show this heuristic produces classification results that are more accurate, on average, than traditional NNs and MDM.