Using "Annotator Rationales" to Improve Machine Learning for Text Categorization

We propose a new framework for supervised machine learning. Our goal is to learn from smaller amounts of supervised training data, by collecting a richer kind of training data: annotations with “rationales.” When annotating an example, the human teacher will also highlight evidence supporting this annotation—thereby teaching the machine learner why the example belongs to the category. We provide some rationale-annotated data and present a learning method that exploits the rationales during training to boost performance significantly on a sample task, namely sentiment classification of movie reviews. We hypothesize that in some situations, providing rationales is a more fruitful use of an annotator’s time than annotating more examples.

• Type: Conference
• Year: 2007
• Where: NAACL
• Authors: Omar Zaidan, Jason Eisner, Christine D. Piatko
• Keywords: Computational Linguistics, NAACL 2007, Supervised Machine, Supervised Training Data, Training Data