Abstract. Crawling the deep web often requires the selection of an appropriate set of queries so that they can cover most of the documents in the data source with low cost. This ca...
The study of the web as a graph is not only fascinating in its own right, but also yields valuable insight into web algorithms for crawling, searching and community discovery, and...
Andrei Z. Broder, Ravi Kumar, Farzin Maghoul, Prab...
We propose an approach to content-based Distributed Information Retrieval based on the periodic and incremental centralisation of full-content indices of widely dispersed and auto...
Fabio Simeoni, Murat Yakici, Steve Neely, Fabio Cr...
The Deep Web, i.e., content hidden behind HTML forms, has long been acknowledged as a significant gap in search engine coverage. Since it represents a large portion of the structu...
Jayant Madhavan, David Ko, Lucja Kot, Vignesh Gana...
This paper describes a decentralized peer-to-peer model for building a Web crawler. Most of the current systems use a centralized client-server model, in which the crawl is done by...