Interpreting Hand-Over-Face Gestures

People often hold their hands near their faces as a gesture in natural conversation, which can interfere with affective inference from facial expressions. However, these gestures are valuable as an additional channel for multi-modal inference. We analyse hand-over-face gestures in a corpus of naturalistic labelled expressions and propose the use of those gestures as a novel affect cue for automatic inference of cognitive mental states. We define three hand cues for encoding hand-over-face gestures, namely hand shape, hand action and facial region occluded, serving as a first step in automating the interpretation process.

• Type: Journal
• Year: 2011
• Where: ACII
• Authors: Marwa Mahmoud, Peter Robinson
• Keywords: ACII 2011, Applied Computing, Automatic Inference, Facial Expressions, Hand Shape