Contrastive Lexical Evaluation of Machine Translation

This paper advocates a complementary measure of translation performance that focuses on the constrastive ability of two or more systems or system versions to adequately translate source words. This is motivated by three main reasons : 1) existing automatic metrics sometimes do not show significant differences that can be revealed by fine-grained focussed human evaluation, 2) these metrics are based on direct comparisons between system hypotheses with the corresponding reference translations, thus ignoring the input words that were actually translated, and 3) as these metrics do not take input hypotheses from several systems at once, fine-grained contrastive evaluation can only be done indirectly. This proposal is illustrated on a multi-source Machine Translation scenario where multiple translations of a

• Type: Conference
• Year: 2010
• Where: LREC
• Authors: Aurélien Max, Josep Maria Crego, François Yvon
• Keywords: Automatic Metrics, Corresponding Reference Translations, Education, Fine-grained Contrastive Evaluation, LREC 2010