Bundling Classifiers by Bagging Trees

The quest of the best classifier for a discriminant analysis problem is often rather hard and a combination of classifiers of different type promises to lead to improved predictive models compared to selecting one of the competitors. We propose to use the out-of-bag sample for training of arbritrary classifiers and classification trees to bundle their predictions for the bootstrap sample in order to construct a combined classifier. The misclassification error of the combined procedure converges to the Bayes error. Benchmark experiments show that the combined classifier is superior to any of the single classifiers in many applications