A Visual Medium for Programmatic Control of Interactive Applications

The VisMap system provides for “visual manipulation” of arbitrary off-the-shelf applications, through an application’s graphical user interface. VisMap’s API-independent control has advantages for tasks that can benefit from direct access to the functions of the user interface. We describe the design goals and architecture of the system, and we discuss two applications, a user-controlled visual scripting program and an autonomous solitaire-playing program, which together demonstrate some of the capabilities and limitations of the approach. Keywords Interaction techniques, agents, demonstrational interfaces, development tools

• Type: Conference
• Year: 1999
• Where: CHI
• Authors: Luke S. Zettlemoyer, Robert St. Amant
• Keywords: Arbitrary Off-the-shelf Applications, CHI 1999, Graphical User Interface, Human Computer Interaction, User Interface