NLP for Indexing and Retrieval of Captioned Photographs

We present a text-based approach for the automatic indexing and retrieval of digital photographs taken at crime scenes. Our research prototype, SOCIS, goes beyond keyword-based approaches and methods that extract syntactic relations from captions; it relies on advanced Natural Language Processing techniques in order to extract relational facts. These relational facts consist of a “pragmatic relation” and the entities this relation connects (triples of the form: ARG1REL- ARG2). In SOCIS, the triples are used as complex image indexing terms; however, the extraction mechanism is used not only for indexing purposes but also for image retrieval using free text queries. The retrieval mechanism computes similarity scores between querytriples and indexing-triples making use of a domain-specific ontology. 1 Indexing and Retrieval of Photographs The normal practice in human indexing or cataloguing of photographs is to use a text-based representation of the pictorial record having recourse ...

• Type: Conference
• Year: 2003
• Where: EACL
• Authors: Horacio Saggion, Katerina Pastra, Yorick Wilks
• Keywords: Automatic Indexing, Complex Image Indexing, EACL 2003, Natural Language Processing, Relational Facts