Solving Decentralized Continuous Markov Decision Problems with Structured Reward

We present an approximation method that solves a class of Decentralized hybrid Markov Decision Processes (DEC-HMDPs). These DEC-HMDPs have both discrete and continuous state variables and represent individual agents with continuous measurable state-space, such as resources. Adding to the natural complexity of decentralized problems, continuous state variables lead to a blowup in potential decision points. Representing value functions as Rectangular Piecewise Constant (RPWC) functions, we formalize and detail an extension to the Coverage Set Algorithm (CSA) [1] that solves transition independent DECHMDPs with controlled error. We apply our algorithm to a range of multi-robot exploration problems with continuous resource constraints.

• Type: Journal
• Year: 2007
• Where: KI
• Authors: Emmanuel Benazera
• Keywords: Artificial Intelligence, Continuous Measurable State-space, Continuous State Variables, KI 2007, State Variables