Facilitating the Evaluation of Automated Negotiators Using Peer Designed Agents

Raz Lin, Sarit Kraus, Yinon Oshrat, Yakov Gal

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Computer agents are increasingly deployed in settings in which they make decisions with people, such as electronic commerce, collaborative interfaces, and cognitive assistants. However, the scientific evaluation of computational strategies for human-computer decision-making is a costly process, involving time, effort and personnel. This paper investigates the use of Peer Designed Agents (PDA)—computer agents developed by human subjects—as a tool for facilitating the evaluation process of automatic negotiators that were developed by researchers. It compares the performance between automatic negotiators that interacted with PDAs to automatic negotiators that interacted with actual people in different domains. The experiments included more than 300 human subjects and 50 PDAs developed by students. Results showed that the automatic negotiators outperformed PDAs in the same situations in which they outperformed people, and that on average, they exhibited the same measure of generosity towards their negotiation partners. These patterns occurred for all types of domains, and for all types of automated negotiators, despite the fact that there were individual differences between the behavior of PDAs and people. The study thus provides an empirical proof that PDAs can alleviate the evaluation process of automatic negotiators, and facilitate their design.
Original languageEnglish
Title of host publicationProceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI-10)
PublisherAAAI Press
Pages817-822
Number of pages6
Publication statusPublished - 2010
EventTwenty-Fourth AAAI Conference on Artificial Intelligence - Atlanta, United States
Duration: 11 Jul 201015 Jul 2010

Conference

ConferenceTwenty-Fourth AAAI Conference on Artificial Intelligence
Abbreviated titleAAAI-10
Country/TerritoryUnited States
CityAtlanta
Period11/07/1015/07/10

Fingerprint

Dive into the research topics of 'Facilitating the Evaluation of Automated Negotiators Using Peer Designed Agents'. Together they form a unique fingerprint.

Cite this