Automatic Induction of Bellman-Error Features for Probabilistic Planning

Domain-specific features are important in representing problem structure throughout machine learning and decision-theoretic planning. In planning, once state features are provided, domainindependent algorithms such as approximate value iteration can learn weighted combinations of those features that often perform well as heuristic estimates of state value (e.g., distance to the goal). Successful applications in real-world domains often require features crafted by human experts. Here, we propose automatic processes for learning useful domain-specific feature sets with little or no human intervention. Our methods select and add features that describe state-space regions of high inconsistency in the Bellman equation (statewise Bellman error) during approximate value iteration. Our method can be applied using any real-valued-feature hypothesis space and corresponding learning method for selecting features from training sets of state-value pairs. We evaluate the method with hypothesis sp...

• Type: Journal
• Year: 2010
• Where: JAIR
• Authors: Jia-Hong Wu, Robert Givan
• Keywords: Approximate Value Iteration, JAIR 2010, Planning, Stochastic Relational Planning