Softmax-Margin CRFs: Training Log-Linear Models with Cost Functions

We describe a method of incorporating taskspecific cost functions into standard conditional log-likelihood (CLL) training of linear structured prediction models. Recently introduced in the speech recognition community, we describe the method generally for structured models, highlight connections to CLL and max-margin learning for structured prediction (Taskar et al., 2003), and show that the method optimizes a bound on risk. The approach is simple, efficient, and easy to implement, requiring very little change to an existing CLL implementation. We present experimental results comparing with several commonly-used methods for training structured predictors for named-entity recognition.

• Type: Journal
• Year: 2010
• Where: NAACL
• Authors: Kevin Gimpel, Noah A. Smith
• Keywords: Computational Linguistics, Linear Structured Prediction, NAACL 2010, Standard Conditional Log-likelihood, Structured