Scalable Peer-to-Peer Web Retrieval with Highly Discriminative Keys

The suitability of Peer-to-Peer (P2P) approaches for fulltext web retrieval has recently been questioned because of the claimed unacceptable bandwidth consumption induced by retrieval from very large document collections. In this contribution we formalize a novel indexing/retrieval model that achieves high performance, costefficient retrieval by indexing with highly discriminative keys (HDKs) stored in a distributed global index maintained in a structured P2P network. HDKs correspond to carefully selected terms and term sets appearing in a small number of collection documents. We provide a theoretical analysis of the scalability of our retrieval model and report experimental results obtained with our HDK-based P2P retrieval engine. These results show that, despite increased indexing costs, the total traffic generated with the HDK approach is significantly smaller than the one obtained with distributed single-term indexing strategies. Furthermore, our experiments show that the retrieva...

• Type: Conference
• Year: 2007
• Where: ICDE
• Authors: Ivana Podnar, Martin Rajman, Toan Luu, Fabius Klemm, Karl Aberer
• Keywords: Database, Fulltext Web Retrieval, ICDE 2007, P2P Retrieval Engine, Truly-decentralized Web Retrieval