QUICR-Learning for Multi-Agent Coordination

Coordinating multiple agents that need to perform a sequence of actions to maximize a system level reward requires solving two distinct credit assignment problems. First, credit must be assigned for an action taken at time step t that results in a reward at time step t > t. Second, credit must be assigned for the contribution of agent i to the overall system performance. The first credit assignment problem is typically addressed with temporal difference methods such as Q-learning. The second credit assignment problem is typically addressed by creating custom reward functions. To address both credit assignment problems simultaneously, we propose the "Q Updates with Immediate Counterfactual Rewards-learning" (QUICR-learning) designed to improve both the convergence properties and performance of Q-learning in large multi-agent problems. QUICR-learning is based on previous work on single-time-step counterfactual rewards described by the collectives framework. Results on a tra...

• Type: Conference
• Year: 2006
• Where: AAAI
• Authors: Adrian K. Agogino, Kagan Tumer
• Keywords: AAAI 2006, Credit Assignment Problems, Distinct Credit Assignment, Intelligent Agents, Time Step