An Iterative Algorithm for Solving Constrained Decentralized Markov Decision Processes

Despite the significant progress to extend Markov Decision Processes (MDP) to cooperative multi-agent systems, developing approaches that can deal with realistic problems remains a serious challenge. Existing approaches that solve Decentralized Markov Decision Processes (DEC-MDPs) suffer from the fact that they can only solve relatively small problems without complex constraints on task execution. OCDEC-MDP has been introduced to deal with large DECMDPs under resource and temporal constraints. However, the proposed algorithm to solve this class of DEC-MDPs has some limits: it suffers from overestimation of opportunity cost and restricts policy improvement to one sweep (or iteration). In this paper, we propose to overcome these limits by first introducing the notion of Expected Opportunity Cost to better assess the influence of a local decision of an agent on the others. We then describe an iterative version of the algorithm to incrementally improve the policies of agents leading to hi...

• Type: Conference
• Year: 2006
• Where: AAAI
• Authors: Aurélie Beynier, Abdel-Illah Mouaddib
• Keywords: AAAI 2006, Cooperative Multi-agent Systems, Intelligent Agents, Markov Decision, Opportunity Cost