A Maximum Entropy Tagger with Unsupervised Hidden Markov Models

We describe a new tagging model where the states of a hidden Markov model (HMM) estimated by unsupervised learning are incorporated as the features in a maximum entropy model. Our method for exploiting unsupervised learning of a probabilistic model can reduce the cost of building taggers with no dictionary and a small annotated corpus. Experimental results on English POS tagging and Japanese word segmentation show that in both tasks our method greatly improves the tagging accuracy when the model is trained with a small annotated corpus. Furthermore, our English POS tagger achieved betterthan-state-of-the-art POS tagging accuracy (96.84%) when a large annotated corpus is available.

• Type: Conference
• Year: 2001
• Where: NLPRS
• Authors: Jun'ichi Kazama, Yusuke Miyao, Jun-ichi Tsujii
• Keywords: Natural Language Processing, NLPRS 2001, POS Tagging Accuracy, Small Annotated Corpus, Unsupervised Learning