A Cost-Shaping LP for Bellman Error Minimization with Performance Guarantees

We introduce a new algorithm based on linear programming that approximates the differential value function of an average-cost Markov decision process via a linear combination of pre-selected basis functions. The algorithm carries out a form of cost shaping and minimizes a version of Bellman error. We establish an error bound that scales gracefully with the number of states without imposing the (strong) Lyapunov condition required by its counterpart in [6]. We propose a path-following method that automates selection of important algorithm parameters which represent counterparts to the "state-relevance weights" studied in [6].

• Type: Conference
• Year: 2004
• Where: NIPS
• Authors: Daniela Pucci de Farias, Benjamin Van Roy
• Keywords: Algorithm, Markov Decision Process, NIPS 2004, NIPS 2007, Pre-selected Basis Functions