Discriminative n-gram language modeling

This paper describes discriminative language modeling for a large vocabulary speech recognition task. We contrast two parameter estimation methods: the perceptron algorithm, and a method based on maximizing the regularized conditional log-likelihood. The models are encoded as deterministic weighted finite state automata, and are applied by intersecting the automata with word-lattices that are the output from a baseline recognizer. The perceptron algorithm has the benefit of automatically selecting a relatively small feature set in just a couple of passes over the training data. We describe a method based on regularized likelihood that makes use of the feature set given by the perceptron algorithm, and initialization with the perceptron’s weights; this method gives an addition 0.5% reduction in word error

• Type: Journal
• Year: 2007
• Where: CSL
• Authors: Brian Roark, Murat Saraclar, Michael Collins
• Keywords: Automated Reasoning, CSL 2007, Parameter Estimation Methods, Perceptron Algorithm, Regularized Conditional Log-likelihood