Sciweavers

299 search results - page 27 / 60
» User-centric Web crawling
Sort
View
ICDE
2002
IEEE
161views Database» more  ICDE 2002»
14 years 10 months ago
Design and Implementation of a High-Performance Distributed Web Crawler
Broad web search engines as well as many more specialized search tools rely on web crawlers to acquire large collections of pages for indexing and analysis. Such a web crawler may...
Vladislav Shkapenyuk, Torsten Suel
WWW
2007
ACM
14 years 9 months ago
A large-scale study of robots.txt
Search engines largely rely on Web robots to collect information from the Web. Due to the unregulated open-access nature of the Web, robot activities are extremely diverse. Such c...
Yang Sun, Ziming Zhuang, C. Lee Giles
ICCS
2007
Springer
14 years 14 days ago
Estimating the Change of Web Pages
This paper presents the estimation methods computing the probabilities of how many times web pages are downloaded and modified, respectively, in the future crawls. The methods can ...
Sung Jin Kim, Sang Ho Lee
MAICS
2004
13 years 10 months ago
Creation of a Style Independent Intelligent Autonomous Citation Indexer to Support Academic Research
This paper describes the current state of RUgle, a system for classifying and indexing papers made available on the World Wide Web, in a domain-independent and universal manner. B...
Eric G. Berkowitz, Mohamed Reda Elkhadiri
NIPS
2000
13 years 10 months ago
The Missing Link - A Probabilistic Model of Document Content and Hypertext Connectivity
We describe a joint probabilistic model for modeling the contents and inter-connectivity of document collections such as sets of web pages or research paper archives. The model is...
David A. Cohn, Thomas Hofmann