Haptic Gaze-Tracking Based Perception of Graphical User Interfaces

This paper presents a novel human-computer interface that enables the computer display to be perceived without any use of the eyes. Our system works by tracking the user’s head position and orientation to obtain their ‘gaze’ point on a virtual screen, and by indicating to the user what object is present at the gaze location via haptic feedback to the fingers and synthetic speech or Braille text. This is achieved by using the haptic vibration frequency delivered to the fingers to indicate the type of screen object at the gaze position, and the vibration amplitude to indicate the screen object’s window-layer, when the object is contained in overlapping windows. Also, objects that are gazed at momentarily have their name output to the user via a Braille display or synthetic speech. Our experiments have shown that by browsing over the screen and receiving haptic and voice (or Braille) feedback in this manner, the user is able to acquire a mental two-dimensional representation of...

• Type: Conference
• Year: 2007
• Where: IV
• Authors: Simon Meers, Koren Ward
• Keywords: Blind Screen Perception, Information Technology, IV 2007, Screen Objects, Virtual Screen