Linear Discriminant Analysis and Discriminative Log-linear Modeling

We discuss the relationship between the discriminative training of Gaussian models and the maximum entropy framework for log-linear models. Observing that linear transforms leave the distributions resulting from the loglinear model unchanged, we derive a discriminative linear feature reduction technique from the maximum entropy approach and compare it to the well-known linear discriminant analysis. From experiments on different corpora we observe that the new technique performs better than linear discriminant analysis if the dimensionality of the feature space is large with respect to the number of classes.

• Type: Conference
• Year: 2004
• Where: ICPR
• Authors: Daniel Keysers, Hermann Ney
• Keywords: Computer Vision, Discriminative Linear Feature, ICPR 2004, Linear Discriminant Analysis, Maximum Entropy Framework