Hand Gestures in Disambiguating Types of You Expressions in Multiparty Meetings

The second person pronoun you serves different functions in English. Each of these different types often corresponds to a different term when translated into another language. Correctly identifying different types of you can be beneficial to machine translation systems. To address this issue, we investigate disambiguation of different types of you occurrences in multiparty meetings with a new focus on the role of hand gesture. Our empirical results have shown that incorporation of gesture improves performance on differentiating between the generic use of you (e.g., refer to people in general) and the referential use of you (e.g., refer to a specific person or a group of people). Incorporation of gesture can also compensate for limitations in automated language processing (e.g., reliable recognition of dialogue acts) and achieve comparable results.

• Type: Journal
• Year: 2010
• Where: SIGDIAL
• Authors: Tyler Baldwin, Joyce Yue Chai, Katrin Kirchhoff
• Keywords: Gesture, Hand Gesture, Multiparty Meetings, Natural Language Processing, SIGDIAL 2010