Sparse Stochastic Finite-State Controllers for POMDPs

Bounded policy iteration is an approach to solving infinitehorizon POMDPs that represents policies as stochastic finitestate controllers and iteratively improves a controller by adjusting the parameters of each node using linear programming. In the original algorithm, the size of the linear programs, and thus the complexity of policy improvement, depends on the number of parameters of each node, which grows with the size of the controller. But in practice, the number of parameters of a node with non-zero values is often very small, and it does not grow with the size of the controller. To exploit this, we develop a version of bounded policy iteration that manipulates a sparse representation of a stochastic finitestate controller. It improves a policy in the same way, and by the same amount, as the original algorithm, but with much better scalability.

• Type: Conference
• Year: 2008
• Where: UAI
• Authors: Eric A. Hansen
• Keywords: Artificial Intelligence, Original Algorithm, Policy Iteration, Stochastic Finitestate Controller, UAI 2008