Query-driven indexing for peer-to-peer text retrieval

We describe a query-driven indexing framework for scalable text retrieval over structured P2P networks. To cope with the bandwidth consumption problem that has been identified as the major obstacle for full-text retrieval in P2P networks, we truncate posting lists associated with indexing features to a constant size storing only top-k ranked document references. To compensate for the loss of information caused by the truncation, we extend the set of indexing features with carefully chosen term sets. Indexing term sets are selected based on the query statistics extracted from query logs, thus we index only such combinations that are a) frequently present in user queries and b) non-redundant w.r.t the rest of the index. The distributed index is compact and efficient as it constantly evolves adapting to the current query popularity distribution. Moreover, it is possible to control the tradeoff between the storage/bandwidth requirements and the quality of query answering by tuning the ind...

• Type: Conference
• Year: 2007
• Where: WWW
• Authors: Gleb Skobeltsyn, Toan Luu, Karl Aberer, Martin Rajman, Ivana Podnar Zarko
• Keywords: Internet Technology, P2P Text Retrieval, Retrieval Query-Driven Indexing, Scalable Text Retrieval, WWW 2007