Collection Selection and Results Merging with Topically Organized U.S. Patents and TREC Data

We investigate three issues in distributed information retrieval, considering both TREC data and U.S. Patents: (1) topical organization of large text collections, (2) collection ranking and selection with topically organized collections (3) results merging, particularly document score normalization, with topically organized collections. We find that it is better to organize collections topically, and that topical collections can be well ranked using either INQUERY’s CORI algorithm, or the Kullback-Leibler divergence (KL), but KL is far worse than CORI for non-topically organized collections. For results merging, collections organized by topic require global idfs for the best performance. Contrary to results found elsewhere, normalized scores are not as good as global idfs for merging when the collections are topically organized. Keywords Information retrieval, Collection selection, Topical organization.

• Type: Conference
• Year: 2000
• Where: CIKM
• Authors: Leah S. Larkey, Margaret E. Connell, James P. Callan
• Keywords: CIKM 2000, Information Management, Large Text Collections, Topical Organization, Topically