Supervised Learning of Semantic Classes for Image Annotation and Retrieval

—A probabilistic formulation for semantic image annotation and retrieval is proposed. Annotation and retrieval are posed as classification problems where each class is defined as the group of database images labeled with a common semantic label. It is shown that, by establishing this one-to-one correspondence between semantic labels and semantic classes, a minimum probability of error annotation and retrieval are feasible with algorithms that are 1) conceptually simple, 2) computationally efficient, and 3) do not require prior semantic segmentation of training images. In particular, images are represented as bags of localized feature vectors, a mixture density estimated for each image, and the mixtures associated with all images annotated with a common semantic label pooled into a density estimate for the corresponding semantic class. This pooling is justified by a multiple instance learning argument and performed efficiently with a hierarchical extension of expectation-maximization....

• Type: Journal
• Year: 2007
• Where: PAMI
• Authors: Gustavo Carneiro, Antoni B. Chan, Pedro J. Moreno, Nuno Vasconcelos
• Keywords: Images, PAMI 2007, Semantic, Semantic Label