Exploiting dictionaries in named entity extraction: combining semi-Markov extraction processes and data integration methods

We consider the problem of improving named entity recognition (NER) systems by using external dictionaries--more specifically, the problem of extending state-of-the-art NER systems by incorporating information about the similarity of extracted entities to entities in an external dictionary. This is difficult because most high-performance named entity recognition systems operate by sequentially classifying words as to whether or not they participate in an entity name; however, the most useful similarity measures score entire candidate names. To correct this mismatch we formalize a semi-Markov extraction process, which is based on sequentially classifying segments of several adjacent words, rather than single words. In addition to allowing a natural way of coupling high-performance NER methods and highperformance similarity functions, this formalism also allows the direct use of other useful entity-level features, and provides a more natural formulation of the NER problem than sequentia...

• Type: Conference
• Year: 2004
• Where: KDD
• Authors: William W. Cohen, Sunita Sarawagi
• Keywords: Data Mining, KDD 2004, NER Problem, State-of-the-art Ner Systems, Useful Similarity Measures