Gradient-Based Learning Updates Improve XCS Performance in Multistep Problems

This paper introduces a gradient-based reward prediction update mechanism to the XCS classifier system as applied in neuralnetwork type learning and function approximation mechanisms. A strong relation of XCS to tabular reinforcement learning and more importantly to neural-based reinforcement learning techniques is drawn. The resulting gradient-based XCS system learns more stable and reliable in previously investigated hard multistep problems. While the investigations are limited to the binary XCS classifier system, the applied gradient-based update mechanism appears also suitable for the real-valued XCS and other learning classifier systems.

• Type: Conference
• Year: 2004
• Where: GECCO
• Authors: Martin V. Butz, David E. Goldberg, Pier Luca Lanzi
• Keywords: GECCO 2004, Reinforcement Learning, Tabular Reinforcement Learning, Update Mechanism