Parsimonious translation models for information retrieval

In the KL divergence framework, the extended language modeling approach has a critical problem estimating a query model, which is the probabilistic model that encodes user’s information need. For query expansion in initial retrieval, the translation model had been proposed to involve term cooccurrence statistics. However, the translation model was a difficult to apply it, because term co-occurrence statistics must be constructed in the offline time. Especially in large collection, constructing such a large matrix of term cooccurrences statistics prohibitively increases time and space complexity. More seriously, reliable retrieval performance cannot be guaranteed because the translation model may comprise noisy non-topical terms in documents. To resolve these problems, this paper investigates an effective method to construct co-occurrence statistics and eliminate noisy terms by employing a parsimonious translation model. The parsimonious translation model is a compact version of a t...

• Type: Journal
• Year: 2007
• Where: IPM
• Authors: Seung-Hoon Na, In-Su Kang, Jong-Hyeok Lee
• Keywords: IPM 2007, Parsimonious Translation Model, Query Model, Translation Model