Robust Automatic Human Identification Using Face, Mouth, and Acoustic Information

Discriminatory information about person identity is multimodal. Yet, most person recognition systems are unimodal, e.g. the use of facial appearance. With a view to exploiting the complementary nature of different modes of information and increasing pattern recognition robustness to test signal degradation, we developed a multiple expert biometric person identification system that combines information from three experts: face, visual speech, and audio. The system uses multimodal fusion in an automatic unsupervised manner, adapting to the local performance and output reliability of each of the experts. The expert weightings are chosen automatically such that the reliability measure of the combined scores is maximized. To test system robustness to train/test mismatch, we used a broad range of Gaussian noise and JPEG compression to degrade the audio and visual signals, respectively. Experiments were carried out on the XM2VTS database. The multimodal expert system out performed each of the...

• Type: Conference
• Year: 2005
• Where: AMFG
• Authors: Niall A. Fox, Ralph Gross, Jeffrey F. Cohn, Richard B. Reilly
• Keywords: AMFG 2005, Biometrics, Expert Biometric Person, Pattern Recognition Robustness, Person Recognition Systems