Cooperation Schemes between a Web Server and a Web Search Engine

Search engines provide search results based on a large repository of pages downloaded by a web crawler from several servers. To provide best results, this repository must be kept as fresh as possible, but this can be difficult due to the large volume of pages involved and to the fact that polling is the only method for detecting changes. In this paper, we explore and compare several alternatives for keeping fresh repositories that involve some degree of cooperation from servers.

• Type: Conference
• Year: 2003
• Where: LAWEB
• Authors: Carlos Castillo
• Keywords: Human Computer Interaction, Internet Technology, Large Repository, LAWEB 2003, Search Engines, Web Crawler