Probe, Cluster, and Discover: Focused Extraction of QA-Pagelets from the Deep Web

In this paper, we introduce the concept of a QA-Pagelet to refer to the content region in a dynamic page that contains query matches. We present THOR, a scalable and efficient mining system for discovering and extracting QAPagelets from the Deep Web. A unique feature of THOR is its two-phase extraction framework. In the first phase, pages from a deep web site are grouped into distinct clusters of structurally-similar pages. In the second phase, pages from each page cluster are examined through a subtree filtering algorithm that exploits the structural and content similarity at subtree level to identify the QA-Pagelets.

• Type: Conference
• Year: 2004
• Where: ICDE
• Authors: James Caverlee, Ling Liu, David Buttler
• Keywords: Database, Deep Web, Deep Web Site, ICDE 2004, Two-phase Extraction Framework