Analysis of Head Gesture and Prosody Patterns for Prosody-Driven Head-Gesture Animation

We propose a new two-stage framework for joint analysis of head gesture and speech prosody patterns of a speaker toward automatic realistic synthesis of head gestures from speech prosody. In the first stage analysis, we perform Hidden Markov Model (HMM)-based unsupervised temporal segmentation of head gesture and speech prosody features separately to determine elementary head gesture and speech prosody patterns, respectively, for a particular speaker. In the second stage, joint analysis of correlations between these elementary head gesture and prosody patterns is performed using Multistream HMMs to determine an audio-visual mapping model. The resulting audio-visual mapping model is then employed to synthesize natural head gestures from arbitrary input test speech given a head model for the speaker. In the synthesis stage, the audio-visual mapping model is used to predict a sequence of gesture patterns from the prosody pattern sequence computed for the input test speech. The Euler angle...

• Type: Journal
• Year: 2008
• Where: PAMI
• Authors: Mehmet Emre Sargin, Yücel Yemez, Engin Erzin, A. Murat Tekalp
• Keywords: Head Gestures, PAMI 2008, Prosody Pattern, Speech Prosody