Fast Algorithms for the Generalized Foley-Sammon Discriminant Analysis

Linear Discriminant Analysis (LDA) is one of the most popular approaches for feature extraction and dimension reduction to overcome the curse of the dimensionality of the high-dimensional data in many applications of data mining, machine learning, and bioinformatics. In this paper, we made two main contributions to an important LDA scheme, the generalized Foley-Sammon transform (GFST [7, 13], or a trace ratio model [28]) and its regularization (RGFST) which handles the undersampled problem that involves small samples size n but with high number of features N (N > n) and arises frequently in many modern applications. Our first main result is to establish an equivalent reduced model for the RGFST which effectively improves the computational overhead. The iteration method proposed in [28] is applied to solve the GFST or the reduced RGFST. It has been proven [28] that this iteration converges globally and fast convergence was observed numerically, but there is no theoretical analysis on...

• Type: Journal
• Year: 2010
• Where: SIAMMAX
• Authors: Lei-Hong Zhang, Li-Zhi Liao, Michael K. Ng
• Keywords: Convergence, Linear Discriminant Analysis, Quadratic Convergence, SIAMMAX 2010