Optimal design of regularization term and regularization parameter by subspace information criterion

The problem of designing the regularization term and regularization parameter for linear regression models is discussed. Previously, we derived an approximation to the generalization error called the subspace information criterion (SIC), which is an unbiased estimator of the generalization error with finite samples under certain conditions. In this paper, we apply SIC to regularization learning and use it for (a) choosing the optimal regularization term and regularization parameter from given candidates, and (b) obtaining the closed form of the optimal regularization parameter for a fixed regularization term. The effectiveness of SIC is demonstrated through computer simulations with artificial and real data. Keywords supervised learning, generalization error, linear regression, regularization learning, ridge regression, model selection, regularization parameter, subspace information criterion 

• Type: Journal
• Year: 2002
• Where: NN
• Authors: Masashi Sugiyama, Hidemitsu Ogawa
• Keywords: Generalization Error, Neural Networks, NN 2002, Regularization Parameter, Regularization Term