A system is described for simulating environmental sound in interactive virtual worlds, using the physical state of objects as control parameters. It contains a unified framework for integration with physics simulation engines, and synthesis algorithms that are tailored to work within the framework. A range of behaviours can be simulated, including diffuse and non-linear resonators, and loose surfaces. The overall aim has been to produce a flexible and practical system with intuitive controls that will appeal to sound design professionals. This could be valuable for computer game design, and in other areas where realistic environmental audio is required. A review of previous work is included, and discussion of the issues which influence the overall design of the system.