Majority Voting by Independent Classifiers Can Increase Error Rates

The technique of "majority voting" of classifiers is used in machine learning with the aim of constructing a new combined classification rule that has better characteristics than any of a given set of rules. The "Condorcet Jury Theorem" is often cited, incorrectly, as support for a claim that this practice leads to an improved classifier (i.e., one with smaller error probabilities) when the given classifiers are sufficiently good and are uncorrelated. We specifically address the case of two-category classification, and argue that a correct claim can be made for independent (not just uncorrelated) classification errors (not the classifiers themselves), and offer an example demonstrating that the common claim is false. Supplementary materials for this article are available online.