The Entropy Rate Principle as a Predictor of Processing Effort: An Evaluation against Eye-tracking Data

This paper provides evidence for Genzel and Charniak's (2002) entropy rate principle, which predicts that the entropy of a sentence increases with its position in the text. We show that this principle holds for individual sentences (not just for averages), but we also find that the entropy rate effect is partly an artifact of sentence length, which also correlates with sentence position. Secondly, we evaluate a set of predictions that the entropy rate principle makes for human language processing; using a corpus of eye-tracking data, we show that entropy and processing effort are correlated, and that processing effort is constant throughout a text.

• Type: Conference
• Year: 2004
• Where: EMNLP
• Authors: Frank Keller
• Keywords: EMNLP 2004, EMNLP 2007, Entropy Rate Effect, Entropy Rate Principle, Paper Provides Evidence