Towards perceptually realistic talking heads: models, methods and McGurk

Motivated by the need for an informative, unbiased and quantitative perceptual method for the development and evaluation of a talking head we are developing, we propose a new test based on the “McGurk Effect”. Our approach helps to identify strengths and weaknesses in underlying talking head algorithms, and uses this insight to guide further development. The test also evaluates the realism of talking head behavior in comparison to real speaker footage, painting an overall picture of a talking head’s performance. By distracting a participant’s attention away from the true nature of the test, we also obtain an unbiased view on talking head performance - since the participant’s prior concerning what is synthetic animation and what is real footage is not encouraged to develop. Our current talking head is a hierarchical 2D image based model, trained from real speaker video footage and continuous speech signals. After training, the talking head may be animated using new continuous...

• Type: Conference
• Year: 2004
• Where: APGV
• Authors: Darren Cosker, Susan Paddock, A. David Marshall, Paul L. Rosin, Simon K. Rushton
• Keywords: APGV 2004, Artificial Intelligence, Continuous Speech Signals, Current Talking Head