Analyzing Multi-agent Reinforcement Learning Using Evolutionary Dynamics

In this paper, we show how the dynamics of Q-learning can be visualized and analyzed from a perspective of Evolutionary Dynamics (ED). More specifically, we show how ED can be used as a model for Qlearning in stochastic games. Analysis of the evolutionary stable strategies and attractors of the derived ED from the Reinforcement Learning (RL) application then predict the desired parameters for RL in MultiAgent Systems (MASs) to achieve Nash equilibriums with high utility. Secondly, we show how the derived fine tuning of parameter settings from the ED can support application of the COllective INtelligence (COIN) framework. COIN is a proved engineering approach for learning of cooperative tasks in MASs. We show that the derived link between ED and RL predicts performance of the COIN framework and visualizes the incentives provided in COIN toward cooperative behavior.

• Type: Conference
• Year: 2004
• Where: ECML
• Authors: Pieter Jan't Hoen, Karl Tuyls
• Keywords: ECML 2004, Evolutionary Dynamics, Evolutionary Stable Strategies, RL Predicts Performance