Modeling Mutual Context of Object and Human Pose in Human-Object Interaction Activities

Detecting objects in cluttered scenes and estimating articulated human body parts are two challenging problems in computer vision. The difficulty is particularly pronounced in activities involving human-object interactions (e.g. playing tennis), where the relevant object tends to be small or only partially visible, and the human body parts are often self-occluded. We observe, however, that objects and human poses can serve as mutual context to each other – recognizing one facilitates the recognition of the other. In this paper we propose a new random field model to encode the mutual context of objects and human poses in human-object interaction activities. We then cast the model learning task as a structure learning problem, of which the structural connectivity between the object, the overall human pose, and different body parts are estimated through a structure search approach, and the parameters of the model are estimated by a new max-margin algorithm. On a sports data set of si...

• Type: Conference
• Year: 2010
• Where: CVPR
• Authors: Bangpeng Yao, Li Fei-Fei
• Keywords: Computer Vision, CVPR 2010, Human Body Parts, Human Poses, Mutual Context