Automatic Sanskrit Segmentizer Using Finite State Transducers

In this paper, we propose a novel method for automatic segmentation of a Sanskrit string into different words. The input for our segmentizer is a Sanskrit string either encoded as a Unicode string or as a Roman transliterated string and the output is a set of possible splits with weights associated with each of them. We followed two different approaches to segment a Sanskrit text using sandhi1 rules extracted from a parallel corpus of manually sandhi split text. While the first approach augments the finite state transducer used to analyze Sanskrit morphology and traverse it to segment a word, the second approach generates all possible segmentations and validates each constituent using a morph analyzer.

• Type: Journal
• Year: 2010
• Where: ACL
• Authors: Vipul Mittal
• Keywords: ACL 2010, Computational Linguistics, Finite State Transducer, Sanskrit String, Unicode String