When dealing with normally distributed classes, it is well known that the optimal discriminant function for two-classes is linear when the covariance matrices are equal. In this paper, we determine conditions for the optimal linear classifier when the covariance matrices are non-equal. In all the cases discussed here, the classifier is given by a pair of straight lines which is a particular case of the general equation of second degree. One of these cases is when we have two overlapping classes with equal means, which is a general case of the Minsky's Paradox for the Perceptron. Our results, which to our knowledge are the pioneering results for pairwise linear classifiers, yield a general linear classifier for this particular case, which can be obtained directly from the parameters of the distribution. Numerous other analytic results for two and d-dimensional normal vectors have been derived. Finally, we have also provided some empirical results in all the cases, and demonstrated that these linear classifiers achieve very good performance.

Additional Metadata
Series Lecture Notes in Computer Science
Citation
Rueda, L. (Luis), & Oommen, J. (2000). The foundational theory of optimal Bayesian pairwise linear classifiers. In Lecture Notes in Computer Science.