Recognition of hands-free speech and hand pointing action for conversational TV

In this paper, we propose a structure and components of a conversational television set(TV) to which we can ask anything on the broadcasted contents and receive the interesting information from the TV. The conversational TV is composed of two types of processing; back end processing and front end processing. In the back end processing, broadcasted contents are analyzed using speech and video recognition techniques and both of the meta data and the structure are extracted. In the front end processing, human speech and hand action are recognized to understand the user intention. We show some applications, being developed in this conversational TV with multi-modal interactions, such as word explanation, human information retrieval, event retrieval in soccer and baseball video games with contextual awareness. Categories and Subject Descriptors H.5.2 [Information Interfaces and Presentation]: User Interfaces—user-centered design, voice I/O General Terms Design, Experimentation Keywords H...

• Type: Conference
• Year: 2005
• Where: MM
• Authors: Yasuo Ariki, Tetsuya Takiguchi, Atsushi Sako
• Keywords: Conversational Television, Conversational Tv, End Processing, MM 2005