Toward natural interaction in the real world: real-time gesture recognition

Using a new hand tracking technology capable of tracking 3D hand postures in real-time, we developed a recognition system for continuous natural gestures. By natural gestures, we mean those encountered in spontaneous interaction, rather than a set of artificial gestures chosen to simplify recognition. To date we have achieved 95.6% accuracy on isolated gesture recognition, and 73% recognition rate on continuous gesture recognition, with data from 3 users and twelve gesture classes. We connected our gesture recognition system to Google Earth, enabling real time gestural control of a 3D map. We describe the challenges of signal accuracy and signal interpretation presented by working in a real-world environment, and detail how we overcame them. Categories and Subject Descriptors H.5.2 [Information Interfaces and Presentation]: User Interfaces--input devices and strategies, interaction styles General Terms Human Factors Keywords Multimodal interaction, natural human computer interaction, ...

• Type: Journal
• Year: 2010
• Where: ICMI
• Authors: Ying Yin, Randall Davis
• Keywords: Biometrics, Gesture, Gesture Recognition, ICMI 2010, Natural Gestures