Synthetic synesthesia: mixing sound with color

An interface is described that uses color and spatial relations to provide an intuitive interface for sound manipulation. A simple geometric shape, called the Geometric Sound Mixer (GSM), is used to mix sounds. Timbre is represented as color within the GSM; the relative loudness of these sound sources is represented visually by the color mixture. A dynamic representation of any sound mix can be viewed on the Mix Time Line, where relative moment-to-moment audio levels control the color mix and brightness as the sounds play in real time. Perceptually linear audio and color mixes are achieved using psychophysical functions. The result is an environment that allows for complex manipulations of sound in a highly simplified, structured environment.

• Type: Conference
• Year: 1993
• Where: CHI
• Authors: Kristinn R. Thórisson, Karen Donoghue
• Keywords: CHI 1993, Color, Geometric Sound Mixer, Human Computer Interaction, Simple Geometric Shape