The Impact of Reference Quality on Automatic MT Evaluation

Language resource quality is crucial in NLP. Many of the resources used are derived from data created by human beings out of an NLP context, especially regarding MT and reference translations. Indeed, automatic evaluations need highquality data that allow the comparison of both automatic and human translations. The validation of these resources is widely recommended before being used. This paper describes the impact of using different-quality references on evaluation. Surprisingly enough, similar scores are obtained in many cases regardless of the quality. Thus, the limitations of the automatic metrics used within MT are also discussed in this regard.

• Type: Conference
• Year: 2008
• Where: COLING
• Authors: Olivier Hamon, Djamel Mostefa
• Keywords: COLING 2008, Computational Linguistics, Language Resource Quality, NLP Context, Resources