Online Linear Regression and Its Application to Model-Based Reinforcement Learning

We provide a provably efficient algorithm for learning Markov Decision Processes (MDPs) with continuous state and action spaces in the online setting. Specifically, we take a model-based approach and show that a special type of online linear regression allows us to learn MDPs with (possibly kernalized) linearly parameterized dynamics. This result builds on Kearns and Singh’s work that provides a provably efficient algorithm for finite state MDPs. Our approach is not restricted to the linear setting, and is applicable to other classes of continuous MDPs.

• Type: Conference
• Year: 2007
• Where: NIPS
• Authors: Alexander L. Strehl, Michael L. Littman
• Keywords: Continuous State, Efficient Algorithm, Information Technology, NIPS 2007, Online Linear Regression