Limitations of Learning via Embeddings in Euclidean Half-Spaces

The notion of embedding a class of dichotomies in a class of linear half spaces is central to the support vector machines paradigm. We examine the question of determining the minimal Euclidean dimension and the maximal margin that can be obtained when the embedded class has a finite VC dimension. We show that an overwhelming majority of the family of finite concept classes of any constant VC dimension cannot be embedded in low-dimensional half spaces. (In fact, we show that the Euclidean dimension must be almost as high as the size of the instance space.) We strengthen this result even further by showing that an overwhelming majority of the family of finite concept classes of any constant VC dimension cannot be embedded in half spaces (of arbitrarily high Euclidean dimension) with a large margin. (In fact, the margin cannot be substantially larger than the margin achieved by the trivial embedding.) Furthermore, these bounds are robust in the sense that allowing each image half spac...

• Type: Conference
• Year: 2001
• Where: COLT
• Authors: Shai Ben-David, Nadav Eiron, Hans-Ulrich Simon
• Keywords: COLT 2001, Euclidean Dimension, Half Space, Machine Learning, VC Dimension