An Empirical Study of Feature Set Selection for Text Chunking

This paper presents an empirical study for improving the performance of text chunking. We focus on two issues: the problem of selecting feature spaces, and the problem of alleviating the data sparseness. To select a proper feature space, we use a heuristic and exhaustive method of evaluating the performance of models under various feature spaces. Besides, for smoothing the data sparseness, we suggest a method of combining words and word classes based on WordNet. Experimental results showed that words within a given context window are the most important features, and some peculiar features contribute to the improvement of the performance for the particular chunk types. Furthermore, the partial combination of word classes and words gives not only a smoothing effect but also the reduction of the feature space.

• Type: Conference
• Year: 2001
• Where: NLPRS
• Authors: Young-Sook Hwang, Yong-Jae Kwak, Hoo-Jung Chung, So-Young Park, Hae-Chang Rim
• Keywords: Data Sparseness, Feature Space, Natural Language Processing, NLPRS 2001, Word Classes