Sample-Based Planning for Continuous Action Markov Decision Processes

In this paper, we present a new algorithm that integrates recent advances in solving continuous bandit problems with sample-based rollout methods for planning in Markov Decision Processes (MDPs). Our algorithm, Hierarchical Optimistic Optimization applied to Trees (HOOT) addresses planning in continuous-action MDPs. Empirical results are given that show that the performance of our algorithm meets or exceeds that of a similar discrete action planner by eliminating the problem of manual discretization of the action space.

• Type: Journal
• Year: 2011
• Where: AIPS
• Authors: Christopher R. Mansley, Ari Weinstein, Michael L. Littman
• Keywords: Action Planner, AIPS 2011, Artificial Intelligence, Bandit Problems, Markov Decision Processes