Recursive Markov Chains, Stochastic Grammars, and Monotone Systems of Nonlinear Equations

We introduce and study Recursive Markov Chains (RMCs), which extend ordinary finite state Markov chains with the ability to invoke other Markov chains in a potentially recursive manner. They offer l abstract model for probabilistic programs with procedures, and are a probabilistic version of Recursive State Machines. RMCs generalize Stochastic Context-Free Grammars (SCFG) and multi-type Branching Processes, and are intimately related to Probabilistic Pushdown Systems. We focus here on termination and reachability analysis for RMCs. We present both positive and negative results for the general class of RMCs, as well as for important subclasses including SCFGs.

• Type: Conference
• Year: 2005
• Where: STACS
• Authors: Kousha Etessami, Mihalis Yannakakis
• Keywords: Markov Chains, Recursive Markov Chains, STACS 2005, State Markov Chains, Theoretical Computer Science