Decentralized Learning in Markov Games

Learning Automata (LA) were recently shown to be valuable tools for designing Multi-Agent Reinforcement Learning algorithms. One of the principal contributions of LA theory is that a set of decentralized, independent learning automata is able to control a finite Markov Chain with unknown transition probabilities and rewards. In this paper we extend this result to the framework of Markov Games, a straightforward extension of single-agent Markov Decision Problems to distributed multi-agent decision problems. We put a simple learning automaton for every agent in each state and show that the problem can be viewed from 3 different perspectives; the single superagent view, in which a single agent is represented by the whole set of automata, the multi-agent view, in which each agent is represented by the automata it was associated with in each state and finally the LA-view, i.e. the view in which each automaton itself represents an agent. We show that under the same ergodic assumptions of th...

• Type: Journal
• Year: 2008
• Where: TSMC
• Authors: Peter Vrancx, Katja Verbeeck, Ann Nowé
• Keywords: Finite Markov Chain, Multi-agent, Pure Equilibrium Points, TSMC 2008