Regularization by Adding Redundant Features

The Pseudo Fisher Linear Discriminant (PFLD) based on a pseudo-inverse technique shows a peaking behaviour of the generalization error for training sample sizes that are about the feature size: with an increase in the training sample size the generalization error at first decreases reaching the minimum, then increases reaching the maximum at the point where the training sample size is equal to the data dimensionality and afterwards begins again to decrease. A number of ways exist to solve this problem. In this paper it is shown that noise injection by adding redundant features to the data also helps to improve the generalization error of this classifier for critical training sample sizes.

• Type: Conference
• Year: 1998
• Where: SSPR
• Authors: Marina Skurichina, Robert P. W. Duin
• Keywords: Feature Size, Generalization Error, Pattern Recognition, SSPR 1998, Training Sample Sizes