Representations of Keypoint-Based Semantic Concept Detection: A Comprehensive Study

Based on the local keypoints extracted as salient image patches, an image can be described as a "bag-of-visualwords (BoW)" and this representation has appeared promising for object and scene classification. The performance of BoW features in semantic concept detection for large-scale multimedia databases is subject to various representation choices. In this paper, we conduct a comprehensive study on the representation choices of BoW, including vocabulary size, weighting scheme, stop word removal, feature selection, spatial information, and visual bi-gram. We offer practical insights in how to optimize the performance of BoW by choosing appropriate representation choices. For the weighting scheme, we elaborate a soft-weighting method to assess the significance of a visual word to an image. We experimentally show that the soft-weighting outperforms other popular weighting schemes such as TF-IDF with a large margin. Our extensive experiments on TRECVID data sets also indicate th...

• Type: Journal
• Year: 2010
• Where: TMM
• Authors: Yu-Gang Jiang, Jun Yang 0003, Chong-Wah Ngo, Alexander G. Hauptmann
• Keywords: Appropriate Representation Choices, Management, Representation Choices, Semantic Concepts, TMM 2010