Controlling the Complexity of HMM Systems by Regularization

This paper introduces a method for regularization of HMM systems that avoids parameter overfitting caused by insufficient training data. Regularization is done by augmenting the EM training method by a penalty term that favors simple and smooth HMM systems. The penalty term is constructed as a mixture model of negative exponential distributions that is assumed to generate the state dependent emission probabilities of the HMMs. This new method is the successful transfer of a well known regularization approach in neural networks to the HMM domain and can be interpreted as a generalization of traditional state-tying for HMM systems. The effect of regularization is demonstrated for continuous speech recognition tasks by improvingoverfitted triphone models and by speaker adaptation with limited training data.

• Type: Conference
• Year: 1998
• Where: NIPS
• Authors: Christoph Neukirchen, Gerhard Rigoll
• Keywords: HMM Systems, NIPS 1998, NIPS 2007, Penalty Term, Smooth Hmm Systems