Combining Coherence Models and Machine Translation Evaluation Metrics for Summarization Evaluation

An ideal summarization system should produce summaries that have high content coverage and linguistic quality. Many state-ofthe-art summarization systems focus on content coverage by extracting content-dense sentences from source articles. A current research focus is to process these sentences so that they read fluently as a whole. The current AESOP task encourages research on evaluating summaries on content, readability, and overall responsiveness. In this work, we adapt a machine translation metric to measure content coverage, apply an enhanced discourse coherence model to evaluate summary readability, and combine both in a trained regression model to evaluate overall responsiveness. The results show significantly improved performance over AESOP 2011 submitted metrics.

• Type: Journal
• Year: 2012
• Where: ACL
• Authors: Ziheng Lin, Chang Liu, Hwee Tou Ng, Min-Yen Kan
• Keywords: ACL 2012, Computational Linguistics, Discourse Coherence, Regression Model, Summarization Systems