1611 search results - page 135 / 323 » Crawler-Friendly Web Servers |
123
click to vote
CN
1999
15 years 4 months ago
1999
Interactive web services are increasingly replacing traditional static web pages. Producing web services seems to require a tremendous amount of laborious lowlevel coding due to t...
119
click to vote
WWW
2004
ACM
16 years 5 months ago
2004
ACM
This paper presents an algorithm to bound the bandwidth of a Web crawler. The crawler collects statistics on the transfer rate of each server to predict the expected bandwidth use...
127
click to vote
LCN
2002
IEEE
15 years 9 months ago
2002
IEEE
We investigate methods of using CRC32 for compressing Web URL strings and sharing of URL lists between servers, caches, and URL switches. Using trace-based evaluation, we compare ...
149
click to vote
WWW
2009
ACM
16 years 5 months ago
2009
ACM
Web archives preserve the history of Web sites and have high long-term value for media and business analysts. Such archives are maintained by periodically re-crawling entire Web s...
165
click to vote
JCDL
2004
ACM
15 years 10 months ago
2004
ACM
Today’s web is so huge and diverse that it arguably reflects the real world. For this reason, searching the web is a promising approach to find things in the real world. This ...