On the Interaction between Norm and Dimensionality: Multiple Regimes in Learning

A learning problem might have several measures of complexity (e.g., norm and dimensionality) that affect the generalization error. What is the interaction between these complexities? Dimension-free learning theory bounds and parametric asymptotic analyses each provide a partial picture of the full learning curve. In this paper, we use high-dimensional asymptotics on two classical problems--mean estimation and linear regression--to explore the learning curve more completely. We show that these curves exhibit multiple regimes, where in each regime, the excess risk is controlled by a subset of the problem complexities.

• Type: Conference
• Year: 2010
• Where: ICML
• Authors: Percy Liang, Nati Srebro
• Keywords: ICML 2010, Learning Curve, Learning Theory Bounds, Machine Learning, Parametric Asymptotic Analyses