Automatic music video generation based on temporal pattern analysis

Music video (MV) is a short film meant to present a visual representation of a popular music song. In this paper, we present a system that automatically generates MV-like videos from personal home videos based on observations that generally there are obvious repetitive visual and aural patterns in MVs. Based on a set of video and music analysis algorithms, the automatic music video (AMV) generation system automatically extracts temporal structures of the video and music, as well as repetitive patterns in the music. And then, according to the structure and patterns, a set of highlight segments from the raw home video footage are selected, aiming at matching the visual content with the aural structure and pattern. And last, the output music video is rendered by connecting the selected highlight video segments with appropriate transition effects, accompanied with the music. Experiments show that the results are compelling and promising. Categories and Subject Descriptors H.5.1 [Informati...

• Type: Conference
• Year: 2004
• Where: MM
• Authors: Xian-Sheng Hua, Lie Lu, HongJiang Zhang
• Keywords: Automatic Music Video, MM 2004, Music Analysis, Music Video