Solving Factored MDPs with Exponential-Family Transition Models

Markov decision processes (MDPs) with discrete and continuous state and action components can be solved efficiently by hybrid approximate linear programming (HALP). The main idea of the approach is to approximate the optimal value function by a linear combination of basis functions and optimize it by linear programming. In this paper, we extend the existing HALP paradigm beyond the mixture of beta transition model. As a result, we permit modeling of other transition functions, such as normal and gamma densities, without approximating them. To allow for efficient solutions to the expectation terms in HALP, we identify a rich class of conjugate basis functions. Finally, we demonstrate the generalized HALP framework on a rover planning problem, which exhibits continuous time and resource uncertainty.

• Type: Conference
• Year: 2006
• Where: AIPS
• Authors: Branislav Kveton, Milos Hauskrecht
• Keywords: AIPS 2006, Artificial Intelligence, Basis Functions, Hybrid Approximate Linear, Linear Programming