Generating Search Term Variants for Text Collections with Historic Spellings

In this paper, we describe a new approach for retrieval in texts with non-standard spelling, which is important for historic texts in English or German. For this purpose, we present a new algorithm for generating search term variants in ancient orthography. By applying a spell checker on a corpus of historic texts, we generate a list of candidate terms for which the contemporary spellings have to be assigned manually. Then our algorithm produces a set of probabilistic rules. These probabilities can be considered for ranking in the retrieval stage. An experimental comparison shows that our approach outperforms competing methods.

• Type: Conference
• Year: 2006
• Where: ECIR
• Authors: Andrea Ernst-Gerlach, Norbert Fuhr
• Keywords: ECIR 2006, Historic Texts, Information Technology, Non-standard Spelling, Search Term Variants