Training Conditional Random Fields with Multivariate Evaluation Measures

This paper proposes a framework for training Conditional Random Fields (CRFs) to optimize multivariate evaluation measures, including non-linear measures such as F-score. Our proposed framework is derived from an error minimization approach that provides a simple solution for directly optimizing any evaluation measure. Specifically focusing on sequential segmentation tasks, i.e. text chunking and named entity recognition, we introduce a loss function that closely reflects the target evaluation measure for these tasks, namely, segmentation F-score. Our experiments show that our method performs better than standard CRF training.

• Type: Conference
• Year: 2006
• Where: ACL
• Authors: Jun Suzuki, Erik McDermott, Hideki Isozaki
• Keywords: ACL 2006, ACL 2007, Evaluation Measure, Multivariate Evaluation Measures, Sequential Segmentation Tasks