Feature-Discovering Approximate Value Iteration Methods

Sets of features in Markov decision processes can play a critical role ximately representing value and in abstracting the state space. Selection of features is crucial to the success of a system and is most often conducted by a human. We study the problem of automatically selecting problem features, and propose and evaluate a simple approach reducing the problem of selecting a new feature to standard classification learning. We learn a classifier that predicts the sign of the Bellman error over a training set of states. By iteratively adding new classifiers as features with this method, training between iterations with approximate value iteration, we find a Tetris feature set that outperforms randomly constructed features significantly, and obtains a score of about three-tenths of the highest score obtained by using a carefully hand-constructed feature set. We also show that features learned with this method outperform those learned with the previous method of Patrascu et al. [4] ...

• Type: Conference
• Year: 2005
• Where: SARA
• Authors: Jia-Hong Wu, Robert Givan
• Keywords: Approximate Value Iteration, Artificial Intelligence, Markov Decision Processes, Problem Features, SARA 2005