Solving Generalized Semi-Markov Decision Processes Using Continuous Phase-Type Distributions

We introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, we show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing MDP techniques. The techniques we present can also be seen as an alternative approach for solving SMDPs, and we

• Type: Conference
• Year: 2004
• Where: AAAI
• Authors: Håkan L. S. Younes, Reid G. Simmons
• Keywords: AAAI 2004, Decision Processes, Intelligent Agents, Semi-Markov Decision, Stochastic Decision Processes