Exploiting Thread Structures to Improve Smoothing of Language Models for Forum Post Retrieval

Due to many unique characteristics of forum data, forum post retrieval is different from traditional document retrieval and web search, raising interesting research questions about how to optimize the accuracy of forum post retrieval. In this paper, we study how to exploit the naturally available raw thread structures of forums to improve retrieval accuracy in the language modeling framework. Specifically, we propose and study two different schemes for smoothing the language model of a forum post based on the thread containing the post. We explore several different variants of the two schemes to exploit thread structures in different ways. We also create a human annotated test data set for forum post retrieval and evaluate the proposed smoothing methods using this data set. The experiment results show that the proposed methods for leveraging forum threads to improve estimation of document language models are effective, and they outperform the existing smoothing methods for the fo...

• Type: Journal
• Year: 2011
• Where: ECIR
• Authors: Huizhong Duan, Chengxiang Zhai
• Keywords: ECIR 2011, Information Technology, Language Models, Retrieval Accuracy, Traditional Document