Qualitative Analysis of Partially-Observable Markov Decision Processes

We study observation-based strategies for partially-observable Markov decision processes (POMDPs) with parity objectives. An observationbased strategy relies on partial information about the history of a play, namely, on the past sequence of observations. We consider qualitative analysis problems: given a POMDP with a parity objective, decide whether there exists an observation-based strategy to achieve the objective with probability 1 (almostsure winning), or with positive probability (positive winning). Our main results are twofold. First, we present a complete picture of the computational complexity of the qualitative analysis problem for POMDPs with parity objectives and its subclasses: safety, reachability, B¨uchi, and coB¨uchi objectives. We establish several upper and lower bounds that were not known in the literature. Second, we give optimal bounds (matching upper and lower bounds) for the memory required by pure and randomized observation-based strategies for each class of o...

• Type: Journal
• Year: 2010
• Where: MFCS
• Authors: Krishnendu Chatterjee, Laurent Doyen, Thomas A. Henzinger
• Keywords: MFCS 2010, Observation-based Strategies, Parity Objectives, Qualitative Analysis Problem, Theoretical Computer Science