Facilitating the Evaluation of Automated Negotiators using Peer Designed Agents

Computer agents are increasingly deployed in settings in which they make decisions with people, such as electronic commerce, collaborative interfaces, and cognitive assistants. However, the scientific evaluation of computational strategies for human-computer decision-making is a costly process, involving time, effort and personnel. This paper investigates the use of Peer Designed Agents (PDA)--computer agents developed by human subjects--as a tool for facilitating the evaluation process of automatic negotiators that were developed by researchers. It compares the performance between automatic negotiators that interacted with PDAs to automatic negotiators that interacted with actual people in different domains. The experiments included more than 300 human subjects and 50 PDAs developed by students. Results showed that the automatic negotiators outperformed PDAs in the same situations in which they outperformed people, and that on average, they exhibited the same measure of generosity to...

• Type: Conference
• Year: 2010
• Where: AAAI
• Authors: Raz Lin, Sarit Kraus, Yinon Oshrat, Ya'akov (Kobi) Gal
• Keywords: AAAI 2010, Automatic Negotiators, Intelligent Agents, Negotiators Outperformed Pdas, PDAs