Fast Pixel/Part Selection with Sparse Eigenvectors

We extend the "Sparse LDA" algorithm of [7] with new sparsity bounds on 2-class separability and efficient partitioned matrix inverse techniques leading to 1000-fold speed-ups. This mitigates the O(n4 ) scaling that has limited this algorithm's applicability to vision problems and also prioritizes the less-myopic backward elimination stage by making it faster than forward selection. Experiments include "sparse eigenfaces" and gender classification on FERET data as well as pixel/part selection for OCR on MNIST data using Bayesian (GP) classification. SparseLDA is an attractive alternative to the more demanding Automatic Relevance Determination. State-of-the-art recognition is obtained while discarding the majority of pixels in all experiments. Our sparse models also show a better fit to data in terms of the "evidence" or marginal likelihood.

• Type: Conference
• Year: 2007
• Where: ICCV
• Authors: Bernard Moghaddam, Yair Weiss, Shai Avidan
• Keywords: Backward Elimination Stage, Computer Vision, Demanding Automatic Relevance, Efficient Partitioned Matrix, ICCV 2007, Sparse LDA, Sparse Models