Multimodal expression in virtual humans

This work proposes a real-time virtual human multimodal expression model. Five modalities explore the affordances of the body: deterministic, non-deterministic, gesticulation, facial and vocal expression. Deterministic expression is keyframe body animation. Non-deterministic expression is robotics-based procedural body animation. Vocal expression is voice synthesis, through Festival, and parameterization, through SABLE. Facial expression is lip-synch and emotion expression through a parametric muscle-based face model. Inspired by psycholinguistics, gesticulation expression is unconventional, idiosyncratic and unconscious hand gestures animation described as sequences of Portuguese Sign Language hand shapes, position and orientation. Inspired by the arts, one modality goes beyond the body to explore the affordances of the environment and express emotions through camera, lights and music. To control multimodal expression, this work proposes a high-level integrated synchronized markup lan...

• Type: Journal
• Year: 2006
• Where: JVCA
• Authors: Celso de Melo, Ana Paiva
• Keywords: JVCA 2006, Multimodal Expression, Multimodal Expression Model, Vocal Expression