A case for query by image and text content: searching computer help using screenshots and keywords

The multimedia information retrieval community has dedicated extensive research effort to the problem of content-based image retrieval (CBIR). However, these systems find their main limitation in the difficulty of creating pictorial queries. As a result, few systems offer the option of querying by visual examples, and rely on automatic concept detection and tagging techniques to provide support for searching visual content using textual queries. This paper proposes and studies a practical multimodal web search scenario, where CBIR fits intuitively to improve the retrieval of rich information queries. Many online articles contain useful know-how knowledge about computer applications. These articles tend to be richly illustrated by screenshots. We present a system to search for such software know-how articles that leverages the visual correspondences between screenshots. Users can naturally create pictorial queries simply by taking a screenshot of the application to retrieve a list of a...

• Type: Journal
• Year: 2011
• Where: WWW
• Authors: Tom Yeh, Brandyn White, Jose San Pedro, Boris Katz, Larry S. Davis
• Keywords: Content-based Image Retrieval, Internet Technology, Pictorial Queries, Screenshots, WWW 2011