Multi-modal Clustering for Multimedia Collections

Most of the online multimedia collections, such as picture galleries or video archives, are categorized in a fully manual process, which is very expensive and may soon be infeasible with the rapid growth of multimedia repositories. In this paper, we present an effective method for automating this process within the unsupervised learning framework. We exploit the truly multi-modal nature of multimedia collections--they have multiple views, or modalities, each of which contributes its own perspective to the collection's organization. For example, in picture galleries, image captions are often provided that form a separate view on the collection. Color histograms (or any other set of global features) form another view. Additional views are blobs, interest points and other sets of local features. Our model, called Comraf* (pronounced Comraf-Star), efficiently incorporates various views in multi-modal clustering, by which it allows great modeling flexibility. Comraf* is a light-weight...

• Type: Conference
• Year: 2007
• Where: CVPR
• Authors: Ron Bekkerman, Jiwoon Jeon
• Keywords: Computer Vision, CVPR 2007, Markov Random Field, Multi-modal Clustering, Picture Galleries, Real-world Image Galleries, Unsupervised Learning Framework