Clickthrough-based translation models for web search: from word models to phrase models

Web search is challenging partly due to the fact that search queries and Web documents use different language styles and vocabularies. This paper provides a quantitative analysis of the language discrepancy issue, and explores the use of clickthrough data to bridge documents and queries. We assume that a query is parallel to the titles of documents clicked on for that query. Two translation models are trained and integrated into retrieval models: A word-based translation model that learns the translation probability between single words, and a phrase-based translation model that learns the translation probability between multi-term phrases. Experiments are carried out on a real world data set. The results show that the retrieval systems that use the translation models outperform significantly the systems that do not. The paper also demonstrates that standard statistical machine translation techniques such as word alignment, bilingual phrase extraction, and phrase-based decoding, can b...

• Type: Journal
• Year: 2010
• Where: CIKM
• Authors: Jianfeng Gao, Xiaodong He, Jian-Yun Nie
• Keywords: CIKM 2010, Clickthrough Data, Information Technology, Translation Model, Translation Probability