Anticipatory Learning Classifier Systems and Factored Reinforcement Learning

Factored Reinforcement Learning (frl) is a new technique to solve Factored Markov Decision Problems (fmdps) when the structure of the problem is not known in advance. Like Anticipatory Learning Classifier Systems (alcss), it is a model-based Reinforcement Learning approach that includes generalization mechanisms in the presence of a structured domain. In general, frl and alcss are explicit, stateanticipatory approaches that learn generalized state transition models to improve system behavior based on model-based reinforcement learning techniques. In this contribution, we highlight the conceptual similarities and differences between frl and alcss, focusing on the one hand on spiti, an instance of frl method, and on alcss, macs and xacs, on the other hand. Though frl systems seem to benefit from a clearer theoretical grounding, an empirical comparison between spiti and xacs on two benchmark problems reveals that the latter scales much better than the former when some combination of state...

• Type: Conference
• Year: 2008
• Where: ABIALS
• Authors: Olivier Sigaud, Martin V. Butz, Olga Kozlova, Christophe Meyer
• Keywords: ABIALS 2008, Artificial Intelligence, Frl Systems, Model-based Reinforcement, Reinforcement Learning