Planning in Factored Action Spaces with Symbolic Dynamic Programming

We consider symbolic dynamic programming (SDP) for solving Markov Decision Processes (MDP) with factored state and action spaces, where both states and actions are described by sets of discrete variables. Prior work on SDP has considered only the case of factored states and ignored structure in the action space, causing them to scale poorly in terms of the number of action variables. Our main contribution is to present the first SDP-based planning algorithm for leveraging both state and action space structure in order to compute compactly represented value functions and policies. Since our new algorithm can potentially require more space than when action structure is ignored, our second contribution is to describe an approach for smoothly tradingoff space versus time via recursive conditioning. Finally, our third contribution is to introduce a novel SDP approximation that often significantly reduces planning time with little loss in quality by exploiting action structure in weakly c...

• Type: Journal
• Year: 2012
• Where: AAAI
• Authors: Aswin Raghavan, Saket Joshi, Alan Fern, Prasad Tadepalli, Roni Khardon
• Keywords: AAAI 2012, Action Space, Action Spaces, Discrete Variables, Intelligent Agents