Tight Sample Complexity of Large-Margin Learning

We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the -adapted-dimension, which is a simple function of the spectrum of a distribution's covariance matrix, and show distribution-specific upper and lower bounds on the sample complexity, both governed by the -adapted-dimension of the source distribution. We conclude that this new quantity tightly characterizes the true sample complexity of large-margin classification. The bounds hold for a rich family of sub-Gaussian distributions.

• Type: Journal
• Year: 2010
• Where: CORR
• Authors: Sivan Sabato, Nathan Srebro, Naftali Tishby
• Keywords: CORR 2010, Education, Large-margin Classification, Sample Complexity, Tight Distribution-specific Characterization