Multi-task text segmentation and alignment based on weighted mutual information

Text segmentation is important for text analysis, while text alignment is to determine shared sub-topics among similar documents. Multi-task text segmentation and alignment is the extension of single-task segmentation to utilize information of multi-source documents. In this paper we introduce a novel domain-independent unsupervised method for multitask segmentation and alignment based on the idea that the optimal segmentation and alignment maximizes weighted mutual information, mutual information with term weights. The experiment results show that our approach works well. Categories and Subject Descriptors: H.3.3 [Information Storage and Retrieval]: Information Search and RetrievalClustering; I.2.7 [Artificial Intelligence]: Natural Language Processing-Text analysis; General Terms: Algorithms, Design, Experimentation.

• Type: Conference
• Year: 2006
• Where: CIKM
• Authors: Bingjun Sun, Ding Zhou, Hongyuan Zha, John Yen
• Keywords: CIKM 2006, Information Management, Multi-task Text Segmentation, Text Alignment, Text Segmentation