Incremental polyphonic audio to score alignment using beat tracking for singer robots

— We aim at developing a singer robot capable of listening to music with its own “ears” and interacting with a human’s musical performance. Such a singer robot requires at least three functions: listening to the music, understanding what position in the music is being performed, and generating a singing voice. In this paper, we focus on the second function, that is, the capability to align an audio signal to its musical score represented symbolically. Issues underlying the score alignment problem are: (1) diversity in the sounds of various musical instruments, (2) difference between the audio signal and the musical score, (3) fluctuation in tempo of the musical performance. Our solutions to these issues are as follows: (1) the design of features based on a chroma vector in the 12-tone model and onset of the sound, (2) defining the rareness for each tone based on the idea that scarcely used tone is salient in the audio signal, and (3) the use of a switching Kalman filter for ...

• Type: Conference
• Year: 2009
• Where: IROS
• Authors: Takuma Otsuka, Toru Takahashi, Hiroshi G. Okuno, Kazunori Komatani, Tetsuya Ogata, Kazumasa Murata, Kazuhiro Nakadai
• Keywords: Audio Signal, IROS 2009, Musical Score, Robotics, Score Alignment