Sparse higher order conditional random fields for improved sequence labeling

In real sequence labeling tasks, statistics of many higher order features are not sufficient due to the training data sparseness, very few of them are useful. We describe Sparse Higher Order Conditional Random Fields (SHO-CRFs), which are able to handle local features and sparse higher order features together using a novel tractable exact inference algorithm. Our main insight is that states and transitions with same potential functions can be grouped together, and inference is performed on the grouped states and transitions. Though the complexity is not polynomial, SHO-CRFs are still efficient in practice because of the feature sparseness. Experimental results on optical character recognition and Chinese organization name recognition show that with the same higher order feature set, SHO-CRFs significantly outperform previous approaches.

• Type: Conference
• Year: 2009
• Where: ICML
• Authors: Xian Qian, Xiaoqian Jiang, Qi Zhang, Xuanjing Huang, Lide Wu
• Keywords: ICML 2009, Machine Learning, Sequence Labeling Tasks, Tractable Exact Inference, Training Data Sparseness