Experiments in Value Function Approximation with Sparse Support Vector Regression

Abstract. We present first experiments using Support Vector Regression as function approximator for an on-line, sarsa-like reinforcement learner. To overcome the batch nature of SVR two ideas are employed. The first is sparse greedy approximation: the data is projected onto the subspace spanned by only a small subset of the original data (in feature space). This subset can be built up in an on-line fashion. Second, we use the sparsified data to solve a reduced quadratic problem, where the number of variables is independent of the total number of training samples seen. The feasability of this approach is demonstrated on two common toy-problems.

• Type: Conference
• Year: 2004
• Where: ECML
• Authors: Tobias Jung, Thomas Uthmann
• Keywords: ECML 2004, Sarsa-like Reinforcement Learner, Sparse Greedy Approximation, Support Vector Regression