Query expansion is a well-known technique that has been shown to improve average retrieval performance. This technique has not been used in many operational systems because of the fact that it can greatly degrade the performance of some individual queries. We show how comparison between language models of the unexpanded and expanded retrieval results can be used to predict when the expanded retrieval has strayed from the original sense of the query. In these cases, the unexpanded results are used while the expanded results are used in the remaining cases (where such straying is not detected). We evaluate this method on a wide variety of TREC collections. Categories and Subject Descriptors: H.3.3 Information Storage and Retrieval: Query Formulation General Terms: Experimentation
Stephen Cronen-Townsend, Yun Zhou, W. Bruce Croft