Reinforcement learning for DEC-MDPs with changing action sets and partially ordered dependencies

Decentralized Markov decision processes are frequently used to model cooperative multi-agent systems. In this paper, we identify a subclass of general DEC-MDPs that features regularities in the way agents interact with one another. This class is of high relevance for many real-world applications and features provably reduced complexity (NP-complete) compared to the general problem (NEXP-complete). Since optimally solving larger-sized NP-hard problems is intractable, we keep the learning as much decentralized as possible and use multi-agent reinforcement learning to improve the agents' behavior online. Further, we suggest a restricted message passing scheme that notifies other agents about forthcoming effects on their state transitions and that allows the agents to acquire approximate joint policies of high quality. Categories and Subject Descriptors I.2.11 [Artificial Intelligence]: Distributed AI General Terms Algorithms, Design, Theory Keywords Decentralized MDPs, Interaction, ...

• Type: Conference
• Year: 2008
• Where: ATAL
• Authors: Thomas Gabel, Martin A. Riedmiller
• Keywords: ATAL 2008, Cooperative Multi-agent Systems, Intelligent Agents, Keywords Decentralized MDPs, Markov Decision Processes