A Planning Algorithm for Predictive State Representations

We address the problem of optimally controlling stochastic environments that are partially observable. The standard method for tackling such problems is to define and solve a Partially Observable Markov Decision Process (POMDP). However, it is well known that exactly solving POMDPs is very costly computationally. Recently, Littman, Sutton and Singh (2002) have proposed an alternative representation of partially observable environments, called predictive state representations (PSRs). PSRs are grounded in the sequence of actions and observations of the agent, and hence relate the state representation directly to the agent's experience. In this paper, we present a policy iteration algorithm for finding policies using PSRs. In preliminary experiments, our algorithm produced good solutions. 1 Predictive State Representation We assume that we are given a system consisting of a discrete, finite set of n states 5, a discrete finite set of actions A, and a discrete finite set of observati...

• Type: Conference
• Year: 2003
• Where: IJCAI
• Authors: Masoumeh T. Izadi, Doina Precup
• Keywords: IJCAI 2003, IJCAI 2007, Partially Observable Markov Decision Process, Predictive State Representation, State Representation