178 search results - page 4 / 36 » Scheduling Algorithms for Web Crawling |
WIDM
2004
ACM
14 years 28 days ago
2004
ACM
A Focused crawler must use information gleaned from previously crawled page sequences to estimate the relevance of a newly seen URL. Therefore, good performance depends on powerfu...
WWW
2008
ACM
14 years 8 months ago
2008
ACM
This paper proposes a method of crawling Web servers connected to the Internet without imposing a high processing load. We are using the crawler for a field survey of the digital ...
PVLDB
2008
13 years 7 months ago
2008
The Deep Web, i.e., content hidden behind HTML forms, has long been acknowledged as a significant gap in search engine coverage. Since it represents a large portion of the structu...
WWW
2006
ACM
14 years 1 months ago
2006
ACM
We consider the problem of dust: Different URLs with Similar Text. Such duplicate URLs are prevalent in web sites, as web server software often uses aliases and redirections, and...
WWW
2006
ACM
14 years 8 months ago
2006
ACM
Identifying and tracking new information on the Web is important in sociology, marketing, and survey research, since new trends might be apparent in the new information. Such chan...