A Novel Multi-modal Integration and Propagation Model for Cross-Media Information Retrieval

In this paper, we present a novel PLSA-based aspect model and turn cross-media retrieval into two parts of multi-modal integration and correlation propagation. We first use multivariate Gaussian distributions to model continuous quantity in PLSA, avoiding information loss between featureinstance versus real-world matching. Multi-modal correlations are learned in an asymmetrical manner, giving a better control of the respective influence of each modality in the latent space. Then we propose a new propagation pattern to refine multi-modal correlations by efficiently taking the complementary from multi-modalities. Experimental results demonstrate that our method is accurate and robust for cross-media information retrieval.

• Type: Journal
• Year: 2012
• Where: MMM
• Authors: Wanxia Lin, Tong Lu, Feng Su
• Keywords: Gaussian Distributions, Information Loss, Media Retrieval, MMM 2012, Multimedia