The Subspace Information Criterion for Infinite Dimensional Hypothesis Spaces

A central problem in learning is selection of an appropriate model. This is typically done by estimating the unknown generalization errors of a set of models to be selected from and then choosing the model with minimal generalization error estimate. In this article, we discuss the problem of model selection and generalization error estimation in the context of kernel regression models, e.g., kernel ridge regression, kernel subset regression or Gaussian process regression. Previously, a non-asymptotic generalization error estimator called the subspace information criterion (SIC) was proposed, that could be successfully applied to finite dimensional subspace models. SIC is an unbiased estimator of the generalization error for the finite sample case under the conditions that the learning target function belongs to a specified reproducing kernel Hilbert space (RKHS) H and the reproducing kernels centered on training sample points span the whole space H. These conditions hold only if dim H...

• Type: Journal
• Year: 2002
• Where: JMLR
• Authors: Masashi Sugiyama, Klaus-Robert Müller
• Keywords: Generalization Error, Generalization Error Estimator, JMLR 2002, Reproducing Kernels