A causal approach to hierarchical decomposition of factored MDPs

We present Variable Influence Structure Analysis, an algorithm that dynamically performs hierarchical decomposition of factored Markov decision processes. Our algorithm determines causal relationships between state variables and introduces temporallyextended actions that cause the values of state variables to change. Each temporallyextended action corresponds to a subtask that is significantly easier to solve than the overall task. Results from experiments show great promise in scaling to larger tasks.

• Type: Conference
• Year: 2005
• Where: ICML
• Authors: Anders Jonsson, Andrew G. Barto
• Keywords: Determines Causal Relationships, Factored Markov Decision, ICML 2005, Machine Learning, State Variables