In many settings, agents need to identify competent partners to assist them in accomplishing tasks. Direct experience may not provide sufficient data to learn the competence of other agents. Reputation--a community-based assessment of agent competence--can augment direct experience, but is prone to error. This paper addresses the question of when reputation information is useful, examining a variety of multi-agent settings. It provides a systematic study of the way the utility of reputation varies by group size, group competency, level of error, and whether reputation information is available. Results demonstrate that the utility received from reputation increases as group size increases. However, the experiments also show that reputation is useful in small groups, during early rounds of a game series. These results also revealed a "pigeonholing phenomenon" in which highly capable agents are miscategorized by the reputation system as having low competence based on early sequ...
Philip Hendrix, Barbara J. Grosz