Visual Prosody: Facial Movements Accompanying Speech

As we articulate speech, we usually move the head and exhibit various facial expressions. This visual aspect of speech aids understanding and helps communicating additional information, such as the speaker’s mood. In this paper we analyze quantitatively head and facial movements that accompany speech and investigate how they relate to the text’s prosodic structure. We recorded several hours of speech and measured the locations of the speakers’ main facial features as well as their head poses. The text was evaluated with a prosody prediction tool, identifying phrase boundaries and pitch accents. Characteristic for most speakers are simple motion patterns that are repeatedly applied in synchrony with the main prosodic events. Direction and strength of head movements vary widely from one speaker to another, yet their timing is typically well synchronized with the spoken text. Understanding quantitatively the correlations between head movements and spoken text is important for synth...

• Type: Conference
• Year: 2002
• Where: FGR
• Authors: Hans Peter Graf, Eric Cosatto, Volker Strom, Fu Jie Huang
• Keywords: Biometrics, FGR 2002, Head Movements, Speech Aids Understanding, Text’s Prosodic Structure