Collaborative crawling: mining user experiences for topical resource discovery

The rapid growth of the world wide web had made the problem of topic speci c resource discovery an important one in recent years. In this problem, it is desired to nd web pages which satisfy a predicate speci ed by the user. Such a predicate could be a keyword query, a topical query, or some arbitrary contraint. Several techniques such as focussed crawling and intelligent crawling have recently been proposed for topic speci c resource discovery. All these crawlers are linkage based, since they use the hyperlink behavior in order to perform resource discovery. Recent studies have shown that the topical correlations in hyperlinks are quite noisy and may not always show the consistency necessary for a reliable resource discovery process. In this paper, we will approach the problem of resource discovery from an entirely di erent perspective; we will mine the signi cant browsing patterns of world wide web users in order to model the likelihood of web pages belonging to a speci ed predicate...

• Type: Conference
• Year: 2002
• Where: KDD
• Authors: Charu C. Aggarwal
• Keywords: Data Mining, KDD 2002, Reliable Resource Discovery, Resource Discovery, World Wide Web