An Effective Approach to Verbose Queries Using a Limited Dependencies Language Model

Intuitively, any `bag of words' approach in IR should benefit from taking term dependencies into account. Unfortunately, for years the results of exploiting such dependencies have been mixed or inconclusive. To improve the situation, this paper shows how the natural language properties of the target documents can be used to transform and enrich the term dependencies to more useful statistics. This is done in three steps. The term co-occurrence statistics of queries and documents are each represented by a Markov chain. The paper proves that such a chain is ergodic, and therefore its asymptotic behavior is unique, stationary, and independent of the initial state. Next, the stationary distribution is taken to model queries and documents, rather than their initial distributions. Finally, ranking is achieved following the customary language modeling paradigm. The main contribution of this paper is to argue why the asymptotic behavior of the document model is a better representation the...

• Type: Journal
• Year: 2009
• Where: ICTIR
• Authors: Eduard Hoenkamp, Peter Bruza, Dawei Song, Qiang Huang
• Keywords: Asymptotic Behavior, Document, ICTIR 2009, Information Technology, Initial Distribution