Series Expansions for Continuous-Time Markov Processes

We present exchange formulas that allow to express the stationary distribution of a continuous Markov chain with denumerable state-space having generator matrix Q∗ through a continuous time Markov chain with generator matrix Q. Under suitable stability conditions, numerical approximations can be derived from the exchange formulas, and we show that the algorithms converge at a geometric rate. Applications to sensitivity analysis and bounds on perturbations are discussed as well. Numerical examples are presented to illustrate the numerical efficiency of the proposed algorithm.

• Type: Journal
• Year: 2010
• Where: IOR
• Authors: Bernd Heidergott, Arie Hordijk, Nicole Leder
• Keywords: Continuous Time Markov Chain, Exchange Formulas, IOR 2010, Markov Chain