A Beam-Search Extraction Algorithm for Comparable Data

This paper extends previous work on extracting parallel sentence pairs from comparable data (Munteanu and Marcu, 2005). For a given source sentence S, a maximum entropy (ME) classifier is applied to a large set of candidate target translations . A beam-search algorithm is used to abandon target sentences as non-parallel early on during classification if they fall outside the beam. This way, our novel algorithm avoids any document-level prefiltering step. The algorithm increases the number of extracted parallel sentence pairs significantly, which leads to a BLEU improvement of about 1 % on our SpanishEnglish data.

• Type: Journal
• Year: 2009
• Where: ACL
• Authors: Christoph Tillmann
• Keywords: ACL 2009, Algorithm, Computational Linguistics, Parallel Sentence, Parallel Sentence Pairs