A simple iterative algorithm for parsimonious binary kernel Fisher discrimination

By applying recent results in optimization theory variously known as optimization transfer or majorize/minimize algorithms, an algorithm for binary, kernel, Fisher discriminant analysis is introduced that makes use of a non-smooth penalty on the coefficients to provide a parsimonious solution. The problem is converted into a smooth optimization that can be solved iteratively with no greater overhead than iteratively re-weighted least-squares. The result is simple, easily programmed and is shown to perform, in terms of both accuracy and parsimony, as well as or better than a number of leading machine learning algorithms on two well-studied and substantial benchmarks. Key words Kernel machines – Fisher discriminant analysis – majorizeminimize algorithms – sparsity – parsimony

• Type: Journal
• Year: 2010
• Where: PAA
• Authors: Robert F. Harrison, Kitsuchart Pasupa
• Keywords: Algorithms, Distributed And Parallel Computing, Non-smooth Penalty, PAA 2010, Parsimonious Solution