Discriminative probabilistic models for passage based retrieval

The approach of using passage-level evidence for document retrieval has shown mixed results when it is applied to a variety of test beds with different characteristics. One main reason of the inconsistent performance is that there exists no unified framework to model the evidence of individual passages within a document. This paper proposes two probabilistic models to formally model the evidence of a set of top ranked passages in a document. The first probabilistic model follows the retrieval criterion that a document is relevant if any passage in the document is relevant, and models each passage independently. The second probabilistic model goes a step further and incorporates the similarity correlations among the passages. Both models are trained in a discriminative manner. Furthermore, we present a combination approach to combine the ranked lists of document retrieval and passage-based retrieval. An extensive set of experiments have been conducted on four different TREC test beds t...

• Type: Journal
• Year: 2008
• Where: SIGIR
• Authors: Mengqiu Wang, Luo Si
• Keywords: Document Retrieval, Information Technology, Passage-based Retrieval, Probabilistic Models, SIGIR 2008