Multidimensional Triangulation and Interpolation for Reinforcement Learning

Dynamic Programming, Q-learning and other discrete Markov Decision Process solvers can be applied to continuous d-dimensional state-spaces by quantizing the state space into an array of boxes. This is often problematic above two dimensions: a coarse quantization can lead to poor policies, and ne quantization is too expensive. Possible solutions are variable-resolution discretization, or function approximation by neural nets. A third option, which has been little studied in the reinforcement learning literature, is interpolation on a coarse grid. In this paper we study interpolation techniques that can result in vast improvements in the online behavior of the resulting control systems: multilinear interpolation, and an interpolation algorithm based on an interesting regular triangulation of d-dimensional space. We adapt these interpolators under three reinforcement learning paradigms: (i) o ine value iteration with a known model, (ii) Q-learning, and (iii) online value iteration with a ...

• Type: Conference
• Year: 1996
• Where: NIPS
• Authors: Scott Davies
• Keywords: Interpolation, Interpolation Techniques, NIPS 1996, NIPS 2007, Reinforcement Learning