What's There to Talk About? A Multi-Modal Model of Referring Behavior in the Presence of Shared Visual Information

This paper describes the development of a rule-based computational model that describes how a feature-based representation of shared visual information combines with linguistic cues to enable effective reference resolution. This work explores a language-only model, a visualonly model, and an integrated model of reference resolution and applies them to a corpus of transcribed task-oriented spoken dialogues. Preliminary results from a corpus-based analysis suggest that integrating information from a shared visual environment can improve the performance and quality of existing discoursebased models of reference resolution.

• Type: Conference
• Year: 2006
• Where: EACL
• Authors: Darren Gergle
• Keywords: EACL 2006, Effective Reference Resolution, Natural Language Processing, Reference Resolution, Rule-based Computational Model