On Kernels, Margins, and Low-Dimensional Mappings

Kernel functions are typically viewed as providing an implicit mapping of points into a high-dimensional space, with the ability to gain much of the power of that space without incurring a high cost if the result is linearly-separable by a large margin γ. However, the JohnsonLindenstrauss lemma suggests that in the presence of a large margin, a kernel function can also be viewed as a mapping to a low-dimensional space, one of dimension only ˜O(1/γ2 ). In this paper, we explore the question of whether one can efficiently produce such low-dimensional mappings, using only black-box access to a kernel function. That is, given just a program that computes K(x, y) on inputs x, y of our choosing, can we efficiently construct an explicit (small) set of features that effectively capture the power of the implicit high-dimensional space? We answer this question in the affirmative if our method is also allowed black-box access to the underlying data distribution (i.e., unlabeled examples). We ...

• Type: Conference
• Year: 2004
• Where: ALT
• Authors: Maria-Florina Balcan, Avrim Blum, Santosh Vempala
• Keywords: ALT 2004, Black-box Kernel Function, Machine Learning, Polynomial Kernel, Standard Kernel Functions