Matrix updates for perceptron training of continuous density hidden Markov models

In this paper, we investigate a simple, mistakedriven learning algorithm for discriminative training of continuous density hidden Markov models (CD-HMMs). Most CD-HMMs for automatic speech recognition use multivariate Gaussian emission densities (or mixtures thereof) parameterized in terms of their means and covariance matrices. For discriminative training of CDHMMs, we reparameterize these Gaussian distributions in terms of positive semidefinite matrices that jointly encode their mean and covariance statistics. We show how to explore the resulting parameter space in CD-HMMs with perceptron-style updates that minimize the distance between Viterbi decodings and target transcriptions. We experiment with several forms of updates, systematically comparing the effects of different matrix factorizations, initializations, and averaging schemes on phone accuracies and convergence rates. We present experimental results for context-independent CD-HMMs trained in this way on the TIMIT speech cor...

• Type: Conference
• Year: 2009
• Where: ICML
• Authors: Chih-Chieh Cheng, Fei Sha, Lawrence K. Saul
• Keywords: Discriminative Training, Gaussian Emission Densities, ICML 2009, Machine Learning, Positive Semidefinite Matrices