Recursive Markov Decision Processes and Recursive Stochastic Games

We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), which are classes of (finitely presented) countable-state MDPs and zero-sum turn-based (perfect information) stochastic games. They extend standard finite-state MDPs and stochastic games with a recursion feature. We study the decidability and computational complexity of these games under termination objectives for the two players: one player’s goal is to maximize the probability of termination at a given exit, while the other player’s goal is to minimize this probability. In the quantitative termination problems, given an RMDP (or RSSG) and probability p, we wish to decide whether the value of such a termination game is at least p (or at most p); in the qualitative termination problem we wish to decide whether the value is

• Type: Conference
• Year: 2005
• Where: ICALP
• Authors: Kousha Etessami, Mihalis Yannakakis
• Keywords: ICALP 2005, Simple Stochastic Games, Stochastic Games, Termination Problem, Theoretical Computer Science