A Least-Squares Framework for Component Analysis

— Over the last century, Component Analysis (CA) methods such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Canonical Correlation Analysis (CCA), Laplacian Eigenmaps (LE), and Spectral Clustering (SC) have been extensively used as a feature extraction step for modeling, clustering, classification, and visualization. CA techniques are appealing because many can be formulated as eigen-problems, offering great potential for learning linear and non-linear representations of data in closed-form. However, the eigen-formulation often conceals important analytic and computational drawbacks of CA techniques, such as solving generalized eigen-problems with rank deficient matrices (e.g., small sample size problem), lacking intuitive interpretation of normalization factors, and understanding commonalities and differences between CA methods. This paper proposes a unified least-squares framework to formulate many CA methods. We show how PCA, LDA, CCA, LE, SC, and th...

• Type: Journal
• Year: 2012
• Where: PAMI
• Authors: Fernando De la Torre
• Keywords: Linear Discriminant Analysis, Linear Representations, PAMI 2012, Principal Component Analysis, Software Engineering