Learning Socially Embedded Visual Representation from Scratch

Learning image representation by deep model has recently made remarkable achievements for semantic-oriented applications, such as image classification. However, for usercentric tasks, such as image search and recommendation, simply employing the representation learnt from semanticoriented tasks may fail to capture user intentions. In this paper, we propose a novel Socially Embedded VIsual Representation Learning (SEVIR) approach, where an Asymmetric Multi-task CNN (amtCNN ) model is proposed to embed user intention learning task into semantic learning task. Specifically, to address the sparsity and unreliability problems in social behavioral data, we propose to use user clustering, reliability evaluation, random dropout in output layer in our amtCNN. With its the partially shared network architecture, the learnt representation can capture both semantics and user intentions. Comprehensive experiments are conducted to investigate the effectiveness of our approach in applications of u...

• Type: Journal
• Year: 2015
• Where: MM
• Authors: Shaowei Liu, Peng Cui, Wenwu Zhu 0001, Shiqiang Yang
• Keywords: MM 2015, Multimedia