Real-Time Auditory and Visual Multiple-Object Tracking for Humanoids

This paper presents a real-time auditory and visual tracking of multiple objects for humanoid under real-world environments. Real-time processing is crucial for sensorimotor tasks in tracking, and multiple-object tracking is crucial for real-world applications. Multiple sound source tracking needs perception of a mixture of sounds and cancellation of motor noises caused by body movements. However its real-time processing has not been reported yet. Real-time tracking is attained by fusing information obtained by sound source localization, multiple face recognition, speaker tracking, focus of attention control, and motor control. Auditory streams with sound source direction are extracted by active audition system with motor noise cancellation capability from 48 KHz sampling sounds. Visual streams with face ID and 3D-position are extracted by combining skincolor extraction, correlation-based matching, and multiple-scale image generation from a single camera. These auditory and visual str...

• Type: Conference
• Year: 2001
• Where: IJCAI
• Authors: Kazuhiro Nakadai, Ken-ichi Hidai, Hiroshi Mizoguchi, Hiroshi G. Okuno, Hiroaki Kitano
• Keywords: IJCAI 2001, IJCAI 2007, Real-time Processing, Sound Source, Visual Tracking