The Tangent Classifier

Given a classifier, we describe a general method to construct a simple linear classification rule. This rule, called the <italic>tangent classifier</italic>, is obtained by computing the tangent hyperplane to the separation boundary of the groups (generated by the initial classifier) at a certain point. When applied to a quadratic region, the tangent classifier has a neat closed-form expression. We discuss various examples and the application of this new linear classifier in two situations under which standard rules may fail: when there is a fraction of outliers in the training sample and when the dimension of the data is large in comparison with the sample size.