This paper reports a research project intended to build a real-time music-understanding system producing intuitively meaningful descriptions of real-world musical audio signals, such as the melody lines and chorus sections. This paper also introduces our efforts to add correct descriptions (metadata) to the pieces in a music database.