Discovering Hierarchy in Reinforcement Learning with HEXQ

An open problem in reinforcement learning is discovering hierarchical structure. HEXQ, an algorithm which automatically attempts to decompose and solve a model-free factored MDP hierarchically is described. By searching for aliased Markov sub-space regions based on the state variables the algoes temporal and state abstraction to construct a hierarchy of interlinked smaller MDPs.

• Type: Conference
• Year: 2002
• Where: ICML
• Authors: Bernhard Hengst
• Keywords: ICML 2002, Interlinked Smaller Mdps, Machine Learning, Markov Sub-space Regions, Model-free Factored Mdp