Lower Bounds on the VC-Dimension of Smoothly Parametrized Function Classes

We examine the relationship between the VCdimension and the number of parameters of a smoothly parametrized function class. We show that the VC-dimension of such a function class is at least k if there exists a k-dimensional differentiable manifold in the parameter space such that each member of the manifold corresponds to a different decision boundary. Using this result, we are able to obtain lower bounds on the VC-dimension proportional to the number of parameters for several function classes including two-layer neural networks with certain smooth activation functions and radial basis functions with a gaussian basis. These lower bounds hold even if the magnitudes of the parameters are restricted to be arbitrarily small. In Valiant's probably approximately correct learning framework, this implies that the number of examples necessary for learning these function classes is at least linear in the number of parameters.

• Type: Conference
• Year: 1994
• Where: COLT
• Authors: Wee Sun Lee, Peter L. Bartlett, Robert C. Williamson
• Keywords: COLT 1994, Function Class, Machine Learning, Parameters, Smoothly Parametrized Function