Statistic Analysis for Probabilistic Processes

—We associate a statistical vector to a trace and a geometrical embedding to a Markov Decision Process, based on a distance on words, and study basic Membership and Equivalence problems. The Membership problem for a trace w and a Markov Decision Process S decides if there exists a strategy on S which generates with high probability traces close to w. We prove that Membership of a trace is testable and Equivalence of MDPs is polynomial time approximable. For Probabilistic Automata, Membership is not testable, and approximate Equivalence is undecidable. We give a class of properties, based on results concerning the structure of the tail sigma-field of a finite Markov chain, which characterizes equivalent Markov Decision Processes in this context.

• Type: Conference
• Year: 2009
• Where: LICS
• Authors: Michel de Rougemont, Mathieu Tracol
• Keywords: Computer Science, LICS 2009, Markov Decision, Markov Decision Process, Markov Decision Processes