AutoFeed: an unsupervised learning system for generating webfeeds

The AutoFeed system automatically extracts data from semistructured web sites. Previously, researchers have developed two types of supervised learning approaches for extracting web data: methods that create precise, site-specific extraction rules and methods that learn less-precise site-independent extraction rules. In either case, significant training is required. AutoFeed follows a third, more ambitious approach, in which unsupervised learning is used to analyze sites and discover their structure. Our method relies on a set of heterogeneous “experts”, each of which is capable of identifying certain types of generic structure. Each expert represents its discoveries as “hints”. Based on these hints, our system clusters the pages and identifies semi-structured data that can be extracted. To identify a good clustering, we use a probabilistic model of the hint-generation process. This paper summarizes our formulation of the fully-automatic web-extraction problem, our clusterin...

• Type: Conference
• Year: 2005
• Where: KCAP
• Authors: Bora Gazen, Steven Minton
• Keywords: Extraction Rules, KCAP 2005, Site-specific Extraction Rules, Supervised Learning Approaches