Robust Discriminant Analysis Based on Nonparametric Maximum Entropy

In this paper, we propose a Robust Discriminant Analysis based on maximum entropy (MaxEnt) criterion (MaxEnt-RDA), which is derived from a nonparametric estimate of Renyi’s quadratic entropy. MaxEnt-RDA uses entropy as both objective and constraints; thus the structural information of classes is preserved while information loss is minimized. It is a natural extension of LDA from Gaussian assumption to any distribution assumption. Like LDA, the optimal solution of MaxEnt-RDA can also be solved by an eigen-decomposition method, where feature extraction is achieved by designing two Parzen probability matrices that characterize the within-class variation and the between-class variation respectively. Furthermore, MaxEnt-RDA makes use of high order statistics (entropy) to estimate the probability matrix so that it is robust to outliers. Experiments on toy problem , UCI datasets and face datasets demonstrate the effectiveness of the proposed method with comparison to other state-of-the-art ...

• Type: Conference
• Year: 2009
• Where: ACML
• Authors: Ran He, Bao-Gang Hu, Xiaotong Yuan
• Keywords: ACML 2009, Machine Learning, Maximum Entropy, Renyi’s Quadratic Entropy, Robust Discriminant Analysis