The Contribution of Linguistic Features to Automatic Machine Translation Evaluation

A number of approaches to Automatic MT Evaluation based on deep linguistic knowledge have been suggested. However, n-gram based metrics are still today the dominant approach. The main reason is that the advantages of employing deeper linguistic information have not been clarified yet. In this work, we propose a novel approach for meta-evaluation of MT evaluation metrics, since correlation cofficient against human judges do not reveal details about the advantages and disadvantages of particular metrics. We then use this approach to investigate the benefits of introducing linguistic features into evaluation metrics. Overall, our experiments show that (i) both lexical and linguistic metrics present complementary advantages and (ii) combining both kinds of metrics yields the most robust metaevaluation performance.

• Type: Journal
• Year: 2009
• Where: ACL
• Authors: Enrique Amigó, Jesús Giménez, Julio Gonzalo, Felisa Verdejo
• Keywords: ACL 2009, Computational Linguistics, Evaluation Metrics, MT Evaluation, MT Evaluation Metrics