Word Folding: Taking the Snapshot of Words Instead of the Whole

The snapshot of a word means the most informative fragment of the word. By taking the snapshot instead of the whole, the value space of the lexical feature can be significantly reduced. From the perspective of machine learning, a small space of feature values implies a loss of information but less dataspareness and less unseen data. The snapshot of words can be taken by using the word folding technique, the goal of which is to reduce the value space of lexical features while minimizing the loss of information. Experimental results show the technique can reduce the size of the tagging models and improve the performance at the same time.

• Type: Conference
• Year: 2004
• Where: IJCNLP
• Authors: Jin-Dong Kim, Jun-ichi Tsujii
• Keywords: IJCNLP 2004, Lexical Features, Value Space, Word Folding Technique