Effects of audio and visual surrogates for making sense of digital video

Video surrogates are meant to help people quickly make sense of the content of a video before downloading or seeking more detailed information. In this paper we present the results of a study comparing the effectiveness of three different surrogates for objects in digital video libraries. Thirty-six people participated in a within subjects user study in which they did five tasks for each of three surrogate alternatives: visual alone (a storyboard), audio alone (spoken description), and combined visual and audio (a storyboard augmented with spoken description). The results show that combined surrogates are more effective, strongly preferred, and do not penalize efficiency. The results also demonstrate that spoken descriptions alone lead to better understanding of the video segments than do visual storyboards alone, although people like to have visual surrogates and use them to confirm interpretations and add context. Participants were able to easily use the combined surrogates even tho...

• Type: Conference
• Year: 2007
• Where: CHI
• Authors: Yaxiao Song, Gary Marchionini
• Keywords: CHI 2007, Human Computer Interaction, Multimodal Surrogates, Video Surrogates, Visual Surrogates