Unconditional lower bounds for learning intersections of halfspaces

We prove new lower bounds for learning intersections of halfspaces, one of the most important concept classes in computational learning theory. Our main result is that any statistical-query algorithm for learning the intersection of √ n halfspaces in n dimensions must make 2Ω( √ n) queries. This is the first non-trivial lower bound on the statistical query dimension for this concept class (the previous best lower bound was nΩ(logn)). Our lower bound holds even for intersections of low-weight halfspaces. In the latter case, it is nearly tight. We also show that the intersection of two majorities (low-weight halfspaces) cannot be computed by a polynomial threshold function (PTF) with fewer than nΩ(logn/loglogn) monomials. This is the first super-polynomial lower bound on the PTF length of this concept class, and is nearly optimal. For intersections of k = ω(logn) low-weight halfspaces, we improve our lower bound to min{2Ω( √ n),nΩ(k/logk)}, which too is nearly optimal...

• Type: Journal
• Year: 2007
• Where: ML
• Authors: Adam R. Klivans, Alexander A. Sherstov
• Keywords: Concept Class, Low-weight Halfspaces, Lower Bound, Machine Learning, ML 2007