This paper presents the current state in an ongoing development of the Emonic Environment (EE): a real-time improvisational system employing evolutionary principles for the mutation of media space. We position the associated problems in the context of user interaction, provide eight principles essential for creating an improvisational media environment, follow with a description of how the EE implements these principles, and conclude with a description of the evolutionary algorithms’ functionality.