Space-indexed dynamic programming: learning to follow trajectories

We consider the task of learning to accurately follow a trajectory in a vehicle such as a car or helicopter. A number of dynamic programming algorithms such as Differential Dynamic Programming (DDP) and Policy Search by Dynamic Programming (PSDP), can efficiently compute non-stationary policies for these tasks -- such policies in general are well-suited to trajectory following since they can easily generate different control actions at different times in order to follow the trajectory. However, a weakness of these algorithms is that their policies are timeindexed, in that they apply different policies depending on the current time. This is problematic since 1) the current time may not correspond well to where we are along the trajectory and 2) the uncertainty over states can prevent these algorithms from finding any good policies at all. In this paper we propose a method for space-indexed dynamic programming that overcomes both these difficulties. We begin by showing how a dynamical s...

• Type: Conference
• Year: 2008
• Where: ICML
• Authors: J. Zico Kolter, Adam Coates, Andrew Y. Ng, Yi Gu, Charles DuHadway
• Keywords: Dynamic Programming Algorithms, ICML 2008, Machine Learning, Non-stationary Policies, Space-indexed Dynamic Programming