Maximum Relative Margin and Data-Dependent Regularization

Leading classification methods such as support vector machines (SVMs) and their counterparts achieve strong generalization performance by maximizing the margin of separation between data classes. While the maximum margin approach has achieved promising performance, this article identifies its sensitivity to affine transformations of the data and to directions with large data spread. Maximum margin solutions may be misled by the spread of data and preferentially separate classes along large spread directions. This article corrects these weaknesses by measuring margin not in the absolute sense but rather only relative to the spread of data in any projection direction. Maximum relative margin corresponds to a data-dependent regularization on the classification function while maximum absolute margin corresponds to an ℓ2 norm constraint on the classification function. Interestingly, the proposed improvements only require simple extensions to existing maximum margin formulations and p...

• Type: Journal
• Year: 2010
• Where: JMLR
• Authors: Pannagadatta K. Shivaswamy, Tony Jebara
• Keywords: JMLR 2010, Margin Corresponds, Maximum Margin, Relative Margin