The unichain classification problem detects whether a finite state and action MDP is unichain under all deterministic policies. This problem is NP-hard [11]. This paper provides polynomial algorithms for the unichain classification for an MDP with either a state that is recurrent under all deterministic policies or with a state that is absorbing under some action.
Eugene A. Feinberg, Fenghsu Yang