An analysis of linear models, linear value-function approximation, and feature selection for reinforcement learning

We show that linear value-function approximation is equivalent to a form of linear model approximation. We then derive a relationship between the model-approximation error and the Bellman error, and show how this relationship can guide feature selection for model improvement and/or value-function improvement. We also show how these results give insight into the behavior of existing feature-selection algorithms.

• Type: Conference
• Year: 2008
• Where: ICML
• Authors: Ronald Parr, Lihong Li, Gavin Taylor, Christopher Painter-Wakefield, Michael L. Littman
• Keywords: Bellman Error, ICML 2008, Linear Model Approximation, Linear Value-function Approximation, Machine Learning