Approximate Planning in POMDPs with Macro-Actions

Recent research has demonstrated that useful POMDP solutions do not require consideration of the entire belief space. We extend this idea with the notion of temporal abstraction. We present and explore a new reinforcement learning algorithm over grid-points in belief space, which uses macro-actions and Monte Carlo updates of the Q-values. We apply the algorithm to a large scale robot navigation task and demonstrate that with abstraction we can consider an even smaller part of the belief space, we can learn POMDP policies faster, and we can do information gathering more efficiently.

• Type: Conference
• Year: 2003
• Where: NIPS
• Authors: Georgios Theocharous, Leslie Pack Kaelbling
• Keywords: Belief Space, Entire Belief Space, NIPS 2003, NIPS 2007, Reinforcement Learning Algorithm