Adaptive Chinese Word Segmentation

This paper presents a Chinese word segmentation system which can adapt to different domains and standards. We first present a statistical framework where domain-specific words are identified in a unified approach to word segmentation based on linear models. We explore several features and describe how to create training data by sampling. We then describe a transformation-based learning method used to adapt our system to different word segmentation standards. Evaluation of the proposed system on five test sets with different standards shows that the system achieves state- of-the-art performance on all of them.

• Type: Conference
• Year: 2004
• Where: ACL
• Authors: Jianfeng Gao, Andi Wu, Cheng-Ning Huang, Hongqiao Li, Xinsong Xia, Hauwei Qin
• Keywords: ACL 2004, ACL 2007, Chinese Word Segmentation, Word Segmentation, Word Segmentation Standards