The Role of Critical Sets in Vapnik-Chervonenkis Theory

In the present paper, we present the theoretical basis, as well as an empirical validation, of a protocol designed to obtain effective VC dimension estimations in the case of a simple pattern recognition issue. We first formulate particular (distributiondependent) VC bounds in which a special attention has been given to the exact exponential rate of convergence. We show indeed that the most significant contribution in such bounds is due to the "worst" elements of the model class (designated as the critical sets). We then explain how these results can lead to a rigorous framework for computer simulations involving speed-up techniques for rare event simulation (importance sampling) as well as parameter estimation (linear regression). We thus obtain accurate empirical estimates of the complexity measure and of the multiplicative constant in VC bounds. In particular, we develop the idea of a local complexity characterization associated to every critical set.

• Type: Conference
• Year: 2000
• Where: COLT
• Authors: Nicolas Vayatis
• Keywords: COLT 2000, Critical Set, Machine Learning, VC Bounds, VC Dimension Estimations