Communicative gestures in coreference identification in multiparty meetings

During multiparty meetings, participants can use non-verbal modalities such as hand gestures to make reference to the shared environment. Therefore, one hypothesis is that incorporating hand gestures can improve coreference identification, a task that automatically identifies what participants refer to with their linguistic expressions. To evaluate this hypothesis, this paper examines the role of hand gestures in coreference identification, in particular, focusing on two questions: (1) what signals can distinguish communicative gestures that can potentially help coreference identification from non-communicative gestures; and (2) in what ways can communicative gestures help coreference identification. Based on the AMI data, our empirical results have shown that the length of gesture production is highly indicative of whether a gesture is communicative and potentially helpful in language understanding. Our experiments on the automated identification of coreferring expressions indicate t...

• Type: Journal
• Year: 2009
• Where: ICMI
• Authors: Tyler Baldwin, Joyce Y. Chai, Katrin Kirchhoff
• Keywords: Biometrics, Coreference Identification, Gesture, Hand Gesture, ICMI 2009