Incremental plan aggregation for generating policies in MDPs

Despite the recent advances in planning with MDPs, the problem of generating good policies is still hard. This paper describes a way to generate policies in MDPs by (1) determinizing the given MDP model into a classical planning problem; (2) building partial policies off-line by producing solution plans to the classical planning problem and incrementally aggregating them into a policy, and (3) using sequential Monte-Carlo (MC) simulations of the partial policies before execution, in order to assess the probability of replanning for a policy during execution. The objective of this approach is to quickly generate policies whose probability of replanning is low and below a given threshold. We describe our planner RFF, which incorporates the above ideas. We present theorems showing the termination, soundness and completeness properties of RFF. RFF was the winner of the fully-observable probabilistic track in the 2008 International Planning Competition (IPC-08). In addition to our analyses...

• Type: Conference
• Year: 2010
• Where: ATAL
• Authors: Florent Teichteil-Königsbuch, Ugur Kuter, Guillaume Infantes
• Keywords: ATAL 2010, Classical Planning Problem, Intelligent Agents, Partial Policies, Policies