Estimating Size of Search Engines in an Uncooperative Environment

The number of documents that are indexed by a search engine is referred to as the size of the search engine. The information about the size of each underlying search engine is essential for any metasearch engine to conduct search engine selection, result merging and a few other processes. Thus, effectively estimating the size of search engines is important for a metasearch engine that incorporates multiple autonomous search engines. In this paper, we propose an algorithm that achieves better accuracy compared to the other existing methods for estimating the size of search engines, without losing efficiency. Compared to the Sample-Resample approach, which is the best-known approach in literature, our technique also shows much better tolerance to unfavorable environments.

• Type: Conference
• Year: 2004
• Where: WEBI
• Authors: Surendra Karnatapu, Karthik Ramachandran, Zonghuan Wu, Biren Shah, Vijay V. Raghavan, Ryan Benton
• Keywords: Computer Science, Metasearch Engine, Search Engine Selection, Search Engines, WEBI 2004