Microphone Arrays as Generalized Cameras for Integrated Audio Visual Processing

Combinations of microphones and cameras allow the joint audio visual sensing of a scene. Such arrangements of sensors are common in biological organisms and in applications such as meeting recording and surveillance where both modalities are necessary to provide scene understanding. Microphone arrays provide geometrical information on the source location, and allow the sound sources in the scene to be separated and the noise suppressed, while cameras allow the scene geometry and the location and motion of people and other objects to be estimated. In most previous work the fusion of the audio-visual information occurs at a relatively late stage. In contrast, we take the viewpoint that both cameras and microphone arrays are geometry sensors, and treat the microphone arrays as generalized cameras. We employ computer-vision inspired algorithms to treat the combined system of arrays and cameras. In particular, we consider the geometry introduced by a general microphone array and spherical ...

• Type: Conference
• Year: 2007
• Where: CVPR
• Authors: Adam O'Donovan, Ramani Duraiswami, Jan Neumann
• Keywords: Central Projection Cameras, Computer Vision, CVPR 2007, General Microphone Array, Generalized Cameras, Microphone Arrays, Spherical Microphone Arrays