Unsupervised and Knowledge-Free Morpheme Segmentation and Analysis

This paper presents a revised version of an unsupervised and knowledge-free morpheme boundary detection algorithm based on letter successor variety (LSV) and a trie classifier [5]. Additionally a morphemic analysis based on contextual similarity provides knowledge about relatedness of the found morphs. For the boundary detection the challenge of increasing recall of found morphs while retaining a high precision is tackled by adding a compound splitter, iterating the LSV analysis and dividing the trie classifier into two distinctly applied clasifiers. The result is a significantly improved overall performance and a decreased reliance on corpus size. Further possible improvements and analyses are discussed. Keywords letter successor variety, morpheme boundary detection, morpheme analysis, distributed similarity

• Type: Conference
• Year: 2007
• Where: CLEF
• Authors: Stefan Bordag
• Keywords: CLEF 2007, Information Retrieval, Letter Successor Variety, Morpheme Boundary Detection, Trie Classifier