Aggregation of Document Frequencies in Unstructured P2P Networks

Peer-to-peer (P2P) systems have been recently proposed for providing search and information retrieval facilities over distributed data sources, including web data. Terms and their document frequencies are the main building blocks of retrieval and as such need to be computed, aggregated, and distributed throughout the system. This is a tedious task, as the local view of each peer may not reflect the global document collection, due to skewed document distributions. Moreover, central assembly of the total information is not feasible, due to the prohibitive cost of storage and maintenance, and also because of issues related to digital rights management. In this paper, we propose an efficient approach for aggregating the document frequencies of carefully selected terms based on a hierarchical overlay network. To this end, we examine unsupervised feature selection techniques at the individual peer level, in order to identify only a limited set of the most important terms for aggregation. We...

• Type: Conference
• Year: 2009
• Where: WISE
• Authors: Robert Neumayer, Christos Doulkeridis, Kjetil Nørvåg
• Keywords: Computer Science, Document Frequencies, Global Document Collection, Skewed Document Distributions, WISE 2009