Threshold Accepting Trained Hybrid Principal Component Neural Networks for Pattern Classification

This paper proposes a new architecture for neural networks where the hidden layer is completely replaced by what is referred to as a 'principal component layer'. This layer consists of only a few selected principal components that perform the function of hidden nodes. Then, an algorithm based on the threshold accepting (TA) meta-heuristic is suggested to train the hybrid principal component neural network (PC-NN). The new architecture reduces the number of weights by a great number as there are no formal connections between the input layer and the principal component layer. Further, one needs to pre-specify only two parameters that control the optimizer and the 'percentage variance explained' which determines the number of principal components to be selected. The algorithm is tested on wine classification and Wisconsin breast cancer problems. The results showed better generalization power for the new algorithm compared to the backpropagation. Also, its performance is comparable to that of a TA-based algorithm for the multilayer neural networks proposed elsewhere