Estimating Upper and Lower Bounds on the Performance of Word-Sense Disambiguation Programs

We have recently reported on two new word-sense disambiguation systems, one trained on bilingual material (the Canadian Hansards) and the other trained on monolingual material (Roget's Thesaurus and Grolier's Encyclopedia). After using both the monolingual and bilingual classifiers for a few months, we have convinced ourselves that the performance is remarkably good. Nevertheless, we would really like to be able to make a stronger statement, and therefore, we decided to try to develop some more objective evaluation measures. Although there has been a fair amount of literature on sense-disambiguation, the literature does not offer much guidance in how we might establish the success or failure of a proposed solution such as the two systems mentioned in the previous paragraph. Many papers avoid quantitative evaluations altogether, because it is so difficult to come up with credible estimates of performance. This paper will attempt to establish upper and lower bounds on the leve...

• Type: Conference
• Year: 1992
• Where: ACL
• Authors: William A. Gale, Kenneth Ward Church, David Yarowsky
• Keywords: ACL 1992, ACL 2007, Lower Bound, Word-sense Disambiguation, Word-sense Disambiguation Systems