DiscLDA: Discriminative Learning for Dimensionality Reduction and Classification

Probabilistic topic models have become popular as methods for dimensionality reduction in collections of text documents or images. These models are usually treated as generative models and trained using maximum likelihood or Bayesian methods. In this paper, we discuss an alternative: a discriminative framework in which we assume that supervised side information is present, and in which we wish to take that side information into account in finding a reduced dimensionality representation. Specifically, we present DiscLDA, a discriminative variation on Latent Dirichlet Allocation (LDA) in which a class-dependent linear transformation is introduced on the topic mixture proportions. This parameter is estimated by maximizing the conditional likelihood. By using the transformed topic mixture proportions as a new representation of documents, we obtain a supervised dimensionality reduction algorithm that uncovers the latent structure in a document collection while preserving predictive power f...

• Type: Conference
• Year: 2008
• Where: NIPS
• Authors: Simon Lacoste-Julien, Fei Sha, Michael I. Jordan
• Keywords: Dimensionality Reduction, Information Technology, NIPS 2008, Probabilistic Topic Models, Topic Mixture Proportions