This paper proposes an interface for improvisational ensemble plays which synthesizes musical sounds and graphical images on the floor from people’s act of “walking.” The aim of this paper is to develop such a system that enables nonprofessional people in our public spaces to play good contrapuntal music without any knowledge of music theory. The people are just walking. This system is based on the i-trace system [1] which can capture the people’s behavior and give some visual feedback. Keywords Improvisational Ensemble Play, Contrapuntal Music, Human Tracking, Traces, Spatially Augmented Reality