Sciweavers

41 search results - page 9 / 9
» Taming the inaccessible web
Sort
View
WEBI
2007
Springer
14 years 1 months ago
Determining Bias to Search Engines from Robots.txt
Search engines largely rely on robots (i.e., crawlers or spiders) to collect information from the Web. Such crawling activities can be regulated from the server side by deploying ...
Yang Sun, Ziming Zhuang, Isaac G. Councill, C. Lee...