Reachability analysis of uncertain systems using bounded-parameter Markov decision processes

Verification of reachability properties for probabilistic systems is usually based on variants of Markov processes. Current methods assume an exact model of the dynamic behavior and are not suitable for realistic systems that operate in the presence of uncertainty and variability. This research note extends existing methods for Bounded-parameter Markov Decision Processes (BMDPs) to solve the reachability problem. BMDPs are a generalization of MDPs that allows modeling uncertainty. Our results show that interval value iteration converges in the case of an undiscounted reward criterion that is required to formulate the problems of maximizing the probability of reaching a set of desirable states or minimizing the probability of reaching an unsafe set. Analysis of the computational complexity is also presented.

• Type: Journal
• Year: 2008
• Where: AI
• Authors: Di Wu, Xenofon D. Koutsoukos
• Keywords: AI 2008, Artificial Intelligence, Bounded-parameter Markov Decision, Interval Value Iteration, Reachability Properties