Multiscale Anticipatory Behavior by Hierarchical Reinforcement Learning

Abstract. In order to establish autonomous behavior for technical systems, the well known trade-off between reactive control and deliberative planning has to be considered. Within this paper, we combine both principles by proposing a two-level hierarchical reinforcement learning scheme to enable the system to autonomously determine suitable solutions to new tasks. The approach is based on a behavior representation specified by hybrid automata, which combines continuous and discrete behavior, to predict (anticipate) the outcome of a sequence of actions. igher layer of the hierarchical scheme, the behavior is abstracted in the form of finite state automata, on which value function iteration is performed to obtain a goal leading sequence of subtasks. This sequence is realized on the lower layer by applying policy gradient-based reinforcement learning to the hybrid automaton model. The iteration between both layers leads to a consistent and goal-attaining behavior, as shown for a simple ro...

• Type: Conference
• Year: 2008
• Where: ABIALS
• Authors: Matthias Rungger, Hao Ding, Olaf Stursberg
• Keywords: ABIALS 2008, Artificial Intelligence, Behavior, Reinforcement Learning, Two-level Hierarchical Reinforcement