Logistic Normal Priors for Unsupervised Probabilistic Grammar Induction

We explore a new Bayesian model for probabilistic grammars, a family of distributions over discrete structures that includes hidden Markov models and probabilistic context-free grammars. Our model extends the correlated topic model framework to probabilistic grammars, exploiting the logistic normal distribution as a prior over the grammar parameters. We derive a variational EM algorithm for that model, and then experiment with the task of unsupervised grammar induction for natural language dependency parsing. We show that our model achieves superior results over previous models that use different priors.

• Type: Conference
• Year: 2008
• Where: NIPS
• Authors: Shay B. Cohen, Kevin Gimpel, Noah A. Smith
• Keywords: Correlated Topic Model, Information Technology, NIPS 2008, Probabilistic Context-Free Grammars, Probabilistic Grammars