Utility Evaluation of Cross-document Information Extraction

We describe a utility evaluation to determine whether cross-document information extraction (IE) techniques measurably improve user performance in news summary writing. Two groups of subjects were asked to perform the same time-restricted summary writing tasks, reading news under different conditions: with no IE results at all, with traditional single-document IE results, and with cross-document IE results. Our results show that, in comparison to using source documents only, the quality of summary reports assembled using IE results, especially from cross-document IE, was significantly better and user satisfaction was higher. We also compare the impact of different user groups on the results.

• Type: Journal
• Year: 2010
• Where: NAACL
• Authors: Heng Ji, Zheng Chen, Jonathan Feldman, Antonio Gonzalez, Ralph Grishman, Vivek Upadhyay
• Keywords: Computational Linguistics, Cross-document Information Extraction, NAACL 2010, Summary, User