Visual Attention Driven by Auditory Cues - Selecting Visual Features in Synchronization with Attracting Auditory Events

Human visual attention can be modulated not only by visual stimuli but also by ones from other modalities such as audition. Hence, incorporating auditory information into a human visual attention model would be a key issue for building more sophisticated models. However, the way of integrating multiple pieces of information arising from audio-visual domains still remains a challenging problem. This paper proposes a novel computational model of human visual attention driven by auditory cues. Founded on the Bayesian surprise model that is considered to be promising in the literature, our model uses surprising auditory events to serve as a clue for selecting synchronized visual features and then emphasizes the selected features to form the final surprise map. Our approach to audio-visual integration focuses on using effective visual features alone but not all available features for simulating visual attention with the help of auditory information. Experiments using several video clips sh...

• Type: Journal
• Year: 2015
• Where: MMM
• Authors: Jiro Nakajima, Akisato Kimura, Akihiro Sugimoto, Kunio Kashino
• Keywords: MMM 2015, Multimedia