The Role of Context in Head Gesture Recognition

Head pose and gesture offer several key conversational grounding cues and are used extensively in face-to-face interaction among people. We investigate how dialog context from an embodied conversational agent (ECA) can improve visual recognition of user gestures. We present a recognition framework which (1) extracts contextual features from an ECA's dialog manager, (2) computes a prediction of head nod and head shakes, and (3) integrates the contextual predictions with the visual observation of a vision-based head gesture recognizer. We found a subset of lexical, punctuation and timing features that are easily available in most ECA architectures and can be used to learn how to predict user feedback. Using a discriminative approach to contextual prediction and multi-modal integration, we were able to improve the performance of head gesture detection even when the topic of the test set was significantly different than the training set.

• Type: Conference
• Year: 2006
• Where: AAAI
• Authors: Louis-Philippe Morency, Candace L. Sidner, Christopher Lee, Trevor Darrell
• Keywords: AAAI 2006, Contextual Prediction, Head Gesture, Head Gesture Recognizer, Intelligent Agents