Embodied Active Vision in Language Learning and Grounding

Abstract. Most cognitive studies of language acquisition in both natural systems and artificial systems have focused on the role of purely linguistic information as the central constraint. However, we argue that non-linguistic information, such as vision and talkers’ attention, also plays a major role in language acquisition. To support this argument, this chapter reports two studies of embodied language learning – one on natural intelligence and one on artificial intelligence. First, we developed a novel method that seeks to describe the visual learning environment from a young child’s point of view. A multi-camera sensing environment is built which consists of two head-mounted mini cameras that are placed on both the child’s and the parent’s foreheads respectively. The major result is that the child uses their body to constrain the visual information s/he perceives and by doing so adapts to an embodied solution to deal with the reference uncertainty problem in language le...

• Type: Conference
• Year: 2007
• Where: WAPCV
• Authors: Chen Yu
• Keywords: Computer Vision, Language Acquisition, Language Learning, Visual Learning Environment, WAPCV 2007