Coreference Resolution across Corpora: Languages, Coding Schemes, and Preprocessing Information

This paper explores the effect that different corpus configurations have on the performance of a coreference resolution system, as measured by MUC, B3, and CEAF. By varying separately three parameters (language, annotation scheme, and preprocessing information) and applying the same coreference resolution system, the strong bonds between system and corpus are demonstrated. The experiments reveal problems in coreference resolution evaluation relating to task definition, coding schemes, and features. They also expose systematic biases in the coreference evaluation metrics. We show that system comparison is only possible when corpus parameters are in exact agreement.

• Type: Journal
• Year: 2010
• Where: ACL
• Authors: Marta Recasens, Eduard H. Hovy
• Keywords: ACL 2010, Computational Linguistics, Coreference Evaluation Metrics, Coreference Resolution, Coreference Resolution Evaluation