Convergence in Markovian models with implications for efficiency of inference