A Statistical Model for Topic Segmentation and Clustering

This paper presents a statistical model for discovering topical clusters of words in unstructured text. The model uses a hierarchical Bayesian structure and it is also able to identify segments of text which are topically coherent. The model is able to assign each segment to a particular topic and thus categorizes the corresponding document to potentially multiple topics. We present some initial results indicating that the word topics discovered by the proposed model are more consistent compared to other models. Our early experiments show that our model clustering performance compares well with other clustering models on a real text corpus, although they do not provide topic segmentation. Segmentation performance of our model is also comparable to a recently proposed segmentation model which does not provide document clustering.

• Type: Conference
• Year: 2008
• Where: AI
• Authors: M. Mahdi Shafiei, Evangelos E. Milios
• Keywords: AI 2008, Artificial Intelligence, Model, Model Clustering Performance, Statistical Model