Variational Inference for Adaptor Grammars

Adaptor grammars extend probabilistic context-free grammars to define prior distributions over trees with "rich get richer" dynamics. Inference for adaptor grammars seeks to find parse trees for raw text. This paper describes a variational inference algorithm for adaptor grammars, providing an alternative to Markov chain Monte Carlo methods. To derive this method, we develop a stick-breaking representation of adaptor grammars, a representation that enables us to define adaptor grammars with recursion. We report experimental results on a word segmentation task, showing that variational inference performs comparably to MCMC. Further, we show a significant speed-up when parallelizing the algorithm. Finally, we report promising results for a new application for adaptor grammars, dependency grammar induction.

• Type: Journal
• Year: 2010
• Where: NAACL
• Authors: Shay B. Cohen, David M. Blei, Noah A. Smith
• Keywords: Adaptor Grammars, Computational Linguistics, NAACL 2010, Probabilistic Context-Free Grammars, Variational Inference