Closing the loop in webpage understanding

The two most important tasks in information extraction from the Web are webpage structure understanding and natural language sentences processing. However, little work has been done towards an integrated statistical model for understanding webpage structures and processing natural language sentences within the HTML elements. Our recent work on webpage understanding introduces a joint model of Hierarchical Conditional Random Fields (i.e. HCRF) and extended SemiMarkov Conditional Random Fields (i.e. Semi-CRF) to leverage the page structure understanding results in free text segmentation and labeling. In this top-down integration model, the decision of the HCRF model could guide the decision-making of the Semi-CRF model. However, the drawback of the top-down integration strategy is also apparent, i.e., the decision of the Semi-CRF model could not be used by the HCRF model to guide its decision-making. This paper proposed a novel framework called WebNLP, which enables bidirectional integra...

• Type: Conference
• Year: 2008
• Where: CIKM
• Authors: Chunyu Yang, Yong Cao, Zaiqing Nie, Jie Zhou, Ji-Rong Wen
• Keywords: CIKM 2008, Conditional Random Fields, Information Management, Page Structure Understanding, Structure Understanding