Optimally Regularised Kernel Fisher Discriminant Analysis

Mika et al. [3] introduce a non-linear formulation of Fisher's linear discriminant, based the now familiar "kernel trick", demonstrating state-of-the-art performance on a wide range of real-world benchmark datasets. In this paper, we show that the usual regularisation parameter can be adjusted so as to minimise the leave-one-out cross-validation error with a computational complexity of only O( 2 ) operations, where is the number of training patterns, rather than the O( 4 ) operations required for a na??eve implementation of the leave-one-out procedure. This procedure is then used to form a component of an ef?cient heirarchical model selection strategy where the regularisation parameter is optimised within the inner loop while the kernel parameters are optimised in the outer loop.

• Type: Conference
• Year: 2004
• Where: ICPR
• Authors: Gavin C. Cawley, Kamel Saadi, Nicola L. C. Talbot
• Keywords: Computer Vision, ICPR 2004, Leave-one-out Cross-validation Error, Leave-one-out Procedure, Usual Regularisation Parameter