Automatic discovery and transfer of MAXQ hierarchies

We present an algorithm, HI-MAT (Hierarchy Induction via Models And Trajectories), that discovers MAXQ task hierarchies by applying dynamic Bayesian network models to a successful trajectory from a source reinforcement learning task. HI-MAT discovers subtasks by analyzing the causal and temporal relationships among the actions in the trajectory. Under appropriate assumptions, HI-MAT induces hierarchies that are consistent with the observed trajectory and have compact value-function tables employing safe state abstractions. We demonstrate empirically that HI-MAT constructs compact hierarchies that are comparable to manuallyengineered hierarchies and facilitate significant speedup in learning when transferred to a target task.

• Type: Conference
• Year: 2008
• Where: ICML
• Authors: Neville Mehta, Soumya Ray, Prasad Tadepalli, Thomas G. Dietterich
• Keywords: Compact Hierarchies, HI-MAT Induces Hierarchies, ICML 2008, Machine Learning, MAXQ Task Hierarchies