PEM: A Paraphrase Evaluation Metric Exploiting Parallel Texts

We present PEM, the first fully automatic metric to evaluate the quality of paraphrases, and consequently, that of paraphrase generation systems. Our metric is based on three criteria: adequacy, fluency, and lexical dissimilarity. The key component in our metric is a robust and shallow semantic similarity measure based on pivot language N-grams that allows us to approximate adequacy independently of lexical similarity. Human evaluation shows that PEM achieves high correlation with human judgments.

• Type: Journal
• Year: 2010
• Where: EMNLP
• Authors: Chang Liu, Daniel Dahlmeier, Hwee Tou Ng
• Keywords: EMNLP 2010, Metric, Natural Language Processing, Paraphrase Generation Systems, Shallow Semantic Similarity