Thread detection in dynamic text message streams

Text message stream is a newly emerging type of Web data which is produced in enormous quantities with the popularity of Instant Messaging and Internet Relay Chat. It is beneficial for detecting the threads contained in the text stream for various applications, including information retrieval, expert recognition and even crime prevention. Despite its importance, not much research has been conducted so far on this problem due to the characteristics of the data in which the messages are usually very short and incomplete. In this paper, we present a stringent definition of the thread detection task and our preliminary solution to it. We propose three variations of a single-pass clustering algorithm for exploiting the temporal information in the streams. An algorithm based on linguistic features is also put forward to exploit the discourse structure information. We conducted several experiments to compare our approaches with some existing algorithms on a real dataset. The results show t...

• Type: Conference
• Year: 2006
• Where: SIGIR
• Authors: Dou Shen, Qiang Yang, Jian-Tao Sun, Zheng Chen
• Keywords: Algorithm, Basic Single-pass Algorithm, SIGIR 2006, Single-pass Algorithm