Learning low dimensional predictive representations

Predictive state representations (PSRs) have recently been proposed as an alternative to partially observable Markov decision processes (POMDPs) for representing the state of a dynamical system (Littman et al., 2001). We present a learning algorithm that learns a PSR from observational data. Our algorithm produces a variant of PSRs called transformed predictive state representations (TPSRs). We provide an efficient principal-components-based algorithm for learning a TPSR, and show that TPSRs can perform well in comparison to Hidden Markov Models learned with Baum-Welch in a real world robot tracking task for low dimensional representations and long prediction horizons.

• Type: Conference
• Year: 2004
• Where: ICML
• Authors: Matthew Rosencrantz, Geoffrey J. Gordon, Sebastian Thrun
• Keywords: ICML 2004, Low Dimensional Representations, Machine Learning, Observable Markov Decision, Predictive State Representations