Class-size Independent Generalization Analsysis of Some Discriminative Multi-Category Classification

We consider the problem of deriving class-size independent generalization bounds for some regularized discriminative multi-category classification methods. In particular, we obtain an expected generalization bound for a standard formulation of multi-category support vector machines. Based on the theoretical result, we argue that the formulation over-penalizes misclassification error, which in theory may lead to poor generalization performance. A remedy, based on a generalization of multi-category logistic regression (conditional maximum entropy), is then proposed, and its theoretical properties are examined.

• Type: Conference
• Year: 2004
• Where: NIPS
• Authors: Tong Zhang
• Keywords: Class-size Independent Generalization, Expected Generalization Bound, Generalization Bound, NIPS 2004, NIPS 2007