A Characterization of Meaningful Schedulers for Continuous-Time Markov Decision Processes

Abstract. Continuous-time Markov decision process are an important variant of labelled transition systems having nondeterminism through labels and stochasticity through exponential fire-time distributions. Nondeterministic choices are resolved using the notion of a scheduler. In this paper we characterize the class of measurable schedulers, which is the most general one, and show how a measurable scheduler induces a unique probability measure on the sigma-algebra of infinite paths. We then give evidence that for particular reachability properties it is sufficient to consider a subset of measurable schedulers. Having analyzed schedulers and their induced probability measures we finally show that each probability measure on the sigma-algebra of infinite paths is indeed induced by a measurable scheduler which proves that this class is complete.

• Type: Conference
• Year: 2006
• Where: FORMATS
• Authors: Nicolás Wolovick, Sven Johr
• Keywords: Formal Methods, FORMATS 2006, Infinite Paths, Measurable Scheduler, Probability Measure