Decomposing Large-Scale POMDP Via Belief State Analysis

Partially observable Markov decision process (POMDP) is commonly used to model a stochastic environment with unobservable states for supporting optimal decision making. Computing the optimal policy for a large-scale POMDP is known to be intractable. Belief compression, being an approximate solution, reduces the belief state to be of low dimension and has recently been shown to be both efficient and effective in improving the problem tractability. In this paper, with the conjecture that temporally close belief states could be characterized by a low intrinsic dimension, a novel belief state clustering criterion function is proposed, which considers the belief states’ spatial (in the belief space) and temporal similarities, resulting in belief state clusters as sub-POMDPs of much lower intrinsic dimension and to be distributed to a set of agents for collaborative problem solving. The proposed method has been tested using a synthesized navigation problem (Hallway2) and empirically show...

• Type: Conference
• Year: 2005
• Where: IAT
• Authors: Xin Li, William K. Cheung, Jiming Liu
• Keywords: Belief Compression, Belief State, IAT 2005, Intelligent Agent, Partially Observable Markov Decision Process