The GREC Challenges 2010: Overview and Evaluation Results

There were three GREC Tasks at Generation Challenges 2010: GREC-NER required participating systems to identify all people references in texts; for GRECNEG, systems selected coreference chains for all people entities in texts; and GRECFull combined the NER and NEG tasks, i.e. systems identified and, if appropriate, replaced references to people in texts. Five teams submitted 10 systems in total, and we additionally created baseline systems for each task. Systems were evaluated automatically using a range of intrinsic metrics. In addition, systems were assessed by human judges using preference strength judgements. This report presents the evaluation results, along with descriptions of the three GREC tasks, the evaluation methods, and the participating systems.

• Type: Journal
• Year: 2010
• Where: INLG
• Authors: Anja Belz, Eric Kow
• Keywords: Grec Tasks, INLG 2010, Natural Language Processing, Preference Strength Judgements, Systems