Unknown Word Guessing and Part-of-Speech Tagging Using Support Vector Machines

The accuracy of part-of-speech (POS) tagging for unknown words is substantially lower than that for known words. Considering the high accuracy rate of up-to-date statistical POS taggers, unknown words account for a non-negligible portion of the errors. This paper describes POS prediction for unknown words using Support Vector Machines. We achieve high accuracy in POS tag prediction using substrings and surrounding context as the features. Furthermore, we integrate this method with a practical English POS tagger, and achieve accuracy of 97.1%, higher than conventional approaches.

• Type: Conference
• Year: 2001
• Where: NLPRS
• Authors: Tetsuji Nakagawa, Taku Kudo, Yuji Matsumoto
• Keywords: Natural Language Processing, NLPRS 2001, POS Tagger, Statistical Pos Taggers, Unknown Words