Restricted Bayes Optimal Classifiers

We introduce the notion of restricted Bayes optimal classifiers. These classifiers attempt to combine the flexibility of the generative approach to classification with the high accuracy associated with discriminative learning. They first create a model of the joint distribution over class labels and features. Instead of choosing the decision boundary induced directly from the model, they restrict the allowable types of decision boundaries and learn the one that minimizes the probability of misclassification relative to the estimated joint distribution. In this paper, we investigate two particular instantiations of this approach. The first uses a non-parametric density estimator -- Parzen Windows with Gaussian kernels -and hyperplane decision boundaries. We show that the resulting classifier is asymptotically equivalent to a maximal margin hyperplane classifier, a highly successful discriminative classifier. We therefore provide an alternative justification for maximal margin hyperplan...

• Type: Conference
• Year: 2000
• Where: AAAI
• Authors: Simon Tong, Daphne Koller
• Keywords: AAAI 2000, Decision Boundaries, Intelligent Agents, Margin Hyperplane Classifiers, Maximal Margin Hyperplane