From Transformation-Based Dimensionality Reduction to Feature Selection

Many learning applications are characterized by high dimensions. Usually not all of these dimensions are relevant and some are redundant. There are two main approaches to reduce dimensionality: feature selection and feature transformation. When one wishes to keep the original meaning of the features, feature selection is desired. Feature selection and transformation are typically presented separately. In this paper, we introduce a general approach for converting transformationbased methods to feature selection methods through 1/ regularization. Instead of solving feature selection as a discrete optimization, we relax and formulate the problem as a continuous optimization problem. An additional advantage of our formulation is that our optimization criterion optimizes for feature relevance and redundancy removal automatically. Here, we illustrate how our approach can be utilized to convert linear discriminant analysis (LDA) and the dimensionality reduction version of the Hilbert-Schmid...

• Type: Conference
• Year: 2010
• Where: ICML
• Authors: Mahdokht Masaeli, Glenn Fung, Jennifer G. Dy
• Keywords: Feature Selection, Feature Selection Approaches, Feature Selection Methods, ICML 2010, Machine Learning