Efficient Learning Algorithms Yield Circuit Lower Bounds

We describe a new approach for understanding the difficulty of designing efficient learning algorithms. We prove that the existence of an efficient learning algorithm for a circuit class C in Angluin's model of exact learning from membership and equivalence queries or in Valiant's PAC model yields a lower bound against C. More specifically, we prove that any subexponential time, determinstic exact learning algorithm for C (from membership and equivalence queries) implies the existence of a function f in EXPNP such that f C. If C is PAC learnable with membership queries under the uniform distribution or Exact learnable in randomized polynomial time, we prove that there exists a function f BPEXP (the exponential time analog of BPP) such that f C. For C equal to polynomial-size, depth-two threshold circuits (i.e., neural networks with a polynomial number of hidden nodes), our result shows that efficient learning algorithms for this class would solve one of the most challengi...

• Type: Conference
• Year: 2006
• Where: COLT
• Authors: Lance Fortnow, Adam R. Klivans
• Keywords: COLT 2006, Efficient Learning Algorithms, Equivalence Queries, Exact Learning, Machine Learning