Temporal Symbolic Integration Applied to a Multimodal System Using Gestures and Speech

Abstract. This paper presents a technical approach for temporal symbol integration aimed to be generally applicable in unimodal and multimodal user interfaces. It draws its strength from symbolic data representation and an underlying rulebased system, and is embedded in a multi-agent system. The core method for temporal integration is motivated by findings from cognitive science research. We discuss its application for a gesture recognition task and speech-gesture integration in a Virtual Construction scenario. Finally an outlook of an empirical evaluation is given.

• Type: Conference
• Year: 1999
• Where: GW
• Authors: Timo Sowa, Martin Fröhlich, Marc Erich Latoschik
• Keywords: Biometrics, GW 1999, Multimodal User Interfaces, Temporal Integration, Temporal Symbol Integration