Implementing a new method for discriminant analysis when group covariance matrices are nearly singular
Winfried Theis, Christian Röver and Britta Pouwels
We consider a unified description of classification rules for nearly singular covariance matrices. When the covariance matrices of the groups or the pooled covariance matrix become nearly singular, bayesian classification rules become seriously unstable. Several procedures have been proposed to tackle this problem, e.g. SIMCA, and Regularized Discriminant Analysis. Naes and Indahl (1998) discovered common properties for all of these procedures and proposed a unified classifier that incorporates the functionality of them all. Since the unified approach needs many parameters, they also proposed an alternative classifier with fewer parameters. We implemented both classifiers and compared them in a simulation study to the procedures RDA, LDA, and QDA. To enhance the comparability of our results we based the simulation study on the study of Friedman (1989). In the implementation, we used a combination of the Nelder-Mead Simplexalgorithm and Simulated Annealing (Bohachevsky et al. (1986) to optimize the classification error directly.