Bayesian Inference for PCFGs via Markov Chain Monte Carlo

This paper presents two Markov chain Monte Carlo (MCMC) algorithms for Bayesian inference of probabilistic context free grammars (PCFGs) from terminal strings, providing an alternative to maximum-likelihood estimation using the Inside-Outside algorithm. We illustrate these methods by estimating a sparse grammar describing the morphology of the Bantu language Sesotho, demonstrating that with suitable priors Bayesian techniques can infer linguistic structure in situations where maximum likelihood methods such as the Inside-Outside algorithm only produce a trivial grammar.

• Type: Conference
• Year: 2007
• Where: NAACL
• Authors: Mark Johnson, Thomas L. Griffiths, Sharon Goldwater
• Keywords: Computational Linguistics, Inside-Outside Algorithm, Markov Chain Monte Carlo, NAACL 2007, Priors Bayesian Techniques