Fusion of audio and visual cues for laughter detection

Past research on automatic laughter detection has focused mainly on audio-based detection. Here we present an audiovisual approach to distinguishing laughter from speech and we show that integrating the information from audio and video channels leads to improved performance over singlemodal approaches. Each channel consists of 2 streams (cues), facial expressions and head movements for video and spectral and prosodic features for audio. We used decision level fusion to integrate the information from the two channels and experimented using the SUM rule and a neural network as the integration functions. The results indicate that even a simple linear function such as the SUM rule achieves very good performance in audiovisual fusion. We also experimented with different combinations of cues with the most informative being the facial expressions and the spectral features. The best combination of cues is the integration of facial expressions, spectral and prosodic features when a neural netw...

• Type: Conference
• Year: 2008
• Where: CIVR
• Authors: Stavros Petridis, Maja Pantic
• Keywords: Audiovisual, Audiovisual Approach, CIVR 2008, Facial Expressions, Image Analysis