Fusion vs. Two-Stage for Multimodal Retrieval

We compare two methods for retrieval from multimodal collections. The first is a score-based fusion of results, retrieved visually and textually. The second is a two-stage method that visually re-ranks the top-K results textually retrieved. We discuss their underlying hypotheses and practical limitations, and contact a comparative evaluation on a standardized snapshot of Wikipedia. Both methods are found to be significantly more effective than single-modality baselines, with no clear winner but with different robustness features. Nevertheless, two-stage retrieval provides efficiency benefits over fusion.

• Type: Journal
• Year: 2011
• Where: ECIR
• Authors: Avi Arampatzis, Konstantinos Zagoris, Savvas A. Chatzichristofis
• Keywords: Baselines, Comparative Evaluation, ECIR 2011, Information Technology, Modality