19 search results - page 1 / 4 » Effective web-scale crawling through website analysis |
WWW
2006
ACM
14 years 9 months ago
2006
ACM
The web crawler space is often delimited into two general areas: full-web crawling and focused crawling. We present netSifter, a crawler system which integrates features from thes...
WWW
2007
ACM
14 years 9 months ago
2007
ACM
Search engines largely rely on Web robots to collect information from the Web. Due to the unregulated open-access nature of the Web, robot activities are extremely diverse. Such c...
EMNLP
2009
13 years 6 months ago
2009
Computing the pairwise semantic similarity between all words on the Web is a computationally challenging task. Parallelization and optimizations are necessary. We propose a highly...
SIGIR
2006
ACM
14 years 2 months ago
2006
ACM
Since the website is one of the most important organizational structures of the Web, how to effectively rank websites has been essential to many Web applications, such as Web sear...
ASSETS
2004
ACM
14 years 1 months ago
2004
ACM
Using Internet Archive’s Wayback Machine, a random sample of websites from 1997-2002 were retrospectively analyzed for effects that technology has on accessibility for persons w...