Smart Sight: A Tourist Assistant System

In this paper, we present our e orts towards developing an intelligent tourist system. The system is equipped with a unique combination of sensors and software. The hardware includes two computers, a GPS receiver, a lapel microphone plus an earphone, a video camera and a head-mounted display. This combination enables a multimodal interface to take advantage of speech and gesture input to provide assistance for a tourist. The software supports natural language processing, speech recognition, machine translation, handwriting recognition and multimodal fusion. A vision module is trained to locate and read written language, is able to adapt to to new environments, and is able to interpret intentions o ered by the user, such as a spoken clari cation or pointing gesture. We illustrate the applications of the system using two examples.

• Type: Conference
• Year: 1999
• Where: ISWC
• Authors: Jie Yang, Weiyi Yang, Matthias Denecke, Alex Waibel
• Keywords: Human Computer Interaction, Intelligent Tourist System, ISWC 1999, Natural Language Processing, Spoken Clari Cation