Rates of Convergence of Performance Gradient Estimates Using Function Approximation and Bias in Reinforcement Learning

We address two open theoretical questions in Policy Gradient Reinforcement Learning. The first concerns the efficacy of using function approximation to represent the state action value function, . Theory is presented showing that linear function approximation representations of can degrade the rate of convergence of performance gradient estimates by a factor of

• Type: Conference
• Year: 2001
• Where: NIPS
• Authors: Gregory Z. Grudic, Lyle H. Ungar
• Keywords: Action Value Function, Function Approximation, NIPS 2001, NIPS 2007, Policy Gradient Reinforcement