Theoretical advantages of lenient Q-learners: an evolutionary game theoretic perspective

This paper presents the dynamics of multiple reinforcement learning agents from an Evolutionary Game Theoretic (EGT) perspective. We provide a Replicator Dynamics model for traditional multiagent Q-learning, and we extend these differential equations to account for lenient learners: agents that forgive possible mistakes of their teammates that resulted in lower rewards. We use this extended formal model to visualize the basins of attraction of both traditional and lenient multiagent Q-learners in two benchmark coordination problems. The results indicate that lenience provides learners with more accurate estimates for the utility of their actions, resulting in higher likelihood of convergence to the globally optimal solution. In addition, our research supports the strength of EGT as a backbone for multiagent reinforcement learning.

• Type: Conference
• Year: 2007
• Where: ATAL
• Authors: Liviu Panait, Karl Tuyls
• Keywords: ATAL 2007, Lenient Multiagent Q-learners, Multiagent, Reinforcement Learning