Continuous Audio-Visual Speech Recognition

The Multi-Stream automatic speech recognition approach was investigated in this work as a framework for Audio-Visual data fusion and speech recognition. This method presents many potential advantages for such a task. It particularly allows for synchronous decoding of continuous speech while still allowing for some asynchrony of the visual and acoustic information streams. First, the Multi-Stream formalism is brie y recalled. Then, on top of the Multi-Stream motivations, experiments on the M2VTS multimodal database are presented and discussed. To our knowledge, these are the rst experiments addressing multi-speaker continuous Audio-Visual Speech Recognition AVSR.It is shown that the Multi-Stream approach can yield improved Audio-Visual speech recognition performance when the acoustic signal is corrupted by noise as well as for clean speech.

• Type: Conference
• Year: 1998
• Where: ECCV
• Authors: Juergen Luettin, Stéphane Dupont
• Keywords: Audio-Visual Speech Recognition, Computer Vision, ECCV 1998, Speech Recognition, Speech Recognition Approach