Linear options

Learning, planning, and representing knowledge in large state t multiple levels of temporal abstraction are key, long-standing challenges for building flexible autonomous agents. The options framework provides a formal mechanism for specifying and learning temporally-extended skills. Although past work has demonstrated the benefit of acting according to options in continuous state spaces, one of the advantages of temporal abstraction--the ability to plan using a temporally abstract model--remains a challenging problem when the number of environment states is large or infinite. In this work, we develop a knowledge construct, the linear option, which is capable of modeling temporally dynamics in continuous state spaces. We show that planning with a linear expectation model of an option's dynamics converges to a fixed point with low Temporal Difference (TD) error. Next, building on recent work on linear feature selection, we show conditions under which a linear feature set is suffic...

• Type: Conference
• Year: 2010
• Where: ATAL
• Authors: Jonathan Sorg, Satinder P. Singh
• Keywords: ATAL 2010, Continuous State Spaces, Intelligent Agents, Linear, Linear Option