Evaluating the ICRA 2008 HRI challenge

This paper reports on the evaluation of the ICRA 2008 Human-Robot Interaction (HRI) Challenge. Five research groups demonstrated state-of-the-art work on HRI with a special focus on social and learning abilities. The demonstrations were rated by expert evaluators, in charge of awarding the prize, and 269 participants, i.e. 20 percent of the conference attendees through a standardized questionnaire (semantic differential). The data was analyzed with respect to six independent variables: expert evaluators vs. attendees, nationality of participants, origin region of the demo, age, gender and knowledge level of the attendees. Conference attendees tended to give higher scores for Social Skills, General Impression, and Overall Score than the expert evaluators. Irrespectively of the level of knowledge, age, and gender, conference attendees rated all demos relatively homogeneously. However, a comparative analysis of the conference attendees's ratings nationality-wise showed that demonstr...

• Type: Journal
• Year: 2009
• Where: HRI
• Authors: Astrid Weiss, Thomas Scherndl, Manfred Tscheligi, Aude Billard
• Keywords: Evaluation, Expert Evaluators, HRI 2009, Human Computer Interaction, Human-robot Interaction