Morphological Richness Offsets Resource Demand - Experiences in Constructing a POS Tagger for Hindi

In this paper we report our work on building a POS tagger for a morphologically rich language- Hindi. The theme of the research is to vindicate the stand that- if morphology is strong and harnessable, then lack of training corpora is not debilitating. We establish a methodology of POS tagging which the resource disadvantaged (lacking annotated corpora) languages can make use of. The methodology makes use of locally annotated modestly-sized corpora (15,562 words), exhaustive morpohological analysis backed by high-coverage lexicon and a decision tree based learning algorithm (CN2). The evaluation of the system was done with 4-fold cross validation of the corpora in the news domain (www.bbc.co.uk/hindi). The current accuracy of POS tagging is 93.45% and can be further improved. 1 Motivation and Problem Definition Part-Of-Speech (POS) tagging is a complex task fraught with challenges like ambiguity of parts of speech and handling of "lexical absence" (proper nouns, foreign words...

• Type: Conference
• Year: 2006
• Where: ACL
• Authors: Smriti Singh, Kuhoo Gupta, Manish Shrivastava, Pushpak Bhattacharyya
• Keywords: ACL 2006, ACL 2007, Decision Trees, Pos Taggers, Rich Languages