All, and only, the Errors: more Complete and Consistent Spelling and OCR-Error Correction Evaluation

Some time in the future, some spelling error correction system will correct all the errors, and only the errors. We need evaluation metrics that will tell us when this has been achieved and that can help guide us there. We survey the current practice in the form of the evaluation scheme of the latest major publication on spelling correction in a leading journal. We are forced to conclude that while the metric used there can tell us exactly when the ultimate goal of spelling correction research has been achieved, it offers little in the way of directions to be followed to eventually get there. We propose to consistently use the well-known metrics Recall and Precision, as combined in the F score, on 5 possible levels of measurement that should guide us more informedly along that path. We describe briefly what is then measured or measurable at these levels and propose a framework that should allow for concisely stating what it is one performs in one's evaluations. We finally contras...

• Type: Conference
• Year: 2008
• Where: LREC
• Authors: Martin Reynaert
• Keywords: Education, Evaluation Metrics, Latest Major Publication, LREC 2008, Spelling Error Correction