Reinforcement Learning in Complex Environments Through Multiple Adaptive Partitions

The application of Reinforcement Learning (RL) algorithms to learn tasks for robots is often limited by the large dimension of the state space, which may make prohibitive its application on a tabular model. In this paper, we introduce LEAP (Learning Entities Adaptive Partitioning), a model-free learning algorithm that uses overlapping partitions which are dynamically modified to learn near-optimal policies with a small number of parameters. Starting from a coarse aggregation of the state space, LEAP generates refined partitions whenever it detects an incoherence between the current action values and the actual rewards from the environment. Since in highly stochastic problems the adaptive process can lead to over-refinement, we introduce a mechanism that prunes the macrostates without affecting the learned policy. Through refinement and pruning, LEAP builds a multi-resolution state representation specialized only where it is actually needed. In the last section, we present some exp...

• Type: Conference
• Year: 2007
• Where: AIIA
• Authors: Andrea Bonarini, Alessandro Lazaric, Marcello Restelli
• Keywords: AIIA 2007, Artificial Intelligence, Model-free Learning Algorithm, Multi-resolution State Representation, State Space