Structured Reachability Analysis for Markov Decision Processes

Recent research in decision theoretic planning has focussedon making the solution of Markov decision processes (MDPs) more feasible. We develop a family of algorithms for structured reachability analysis of MDPs that are suitable when an initial state (or set of states) is known. Using compact, structured representations of MDPs (e.g., Bayesian networks), our methods, which vary in the tradeoff between complexity and accuracy, produce structured descriptions of (estimated) reachable states that can be used to eliminate variables or variable values from the problem description, reducing the size of the MDP and making it easier to solve. One contribution of our work is the extension of ideas from GRAPHPLAN to deal with the distributed nature of action representations typically embodied within Bayesnets andthe problem of correlated action effects. We also demonstrate that our algorithm can be made more complete by using k-ary constraints instead of binary constraints. Another contributio...

• Type: Conference
• Year: 1998
• Where: UAI
• Authors: Craig Boutilier, Ronen I. Brafman, Christopher W. Geib
• Keywords: Decision Theoretic Planning, Structured Reachability Analysis, Structured Representations, UAI 1998, UAI 2008