Optimizing fixed-size stochastic controllers for POMDPs and decentralized POMDPs

POMDPs and their decentralized multiagent counterparts, DEC-POMDPs, offer a rich framework for sequential decision making under uncertainty. Their computational complexity, however, presents an important research challenge. One approach that effectively addresses the intractable memory requirements of current algorithms is based on representing agent policies as finite-state controllers. In this paper, we propose a new approach that uses this representation and formulates the problem as a nonlinear program (NLP). The NLP defines an optimal policy of a desired size for each agent. This new representation allows a wide range of powerful nonlinear programming algorithms to be used to solve POMDPs and DEC-POMDPs. Although solving the NLP optimally is often intractable, the results we obtain using an off-the-shelf optimization method are competitive with stateof-the-art POMDP algorithms and outperform state-of-the-art DEC-POMDP algorithms. Our approach is easy to implement and it opens up ...

• Type: Journal
• Year: 2010
• Where: AAMAS
• Authors: Christopher Amato, Daniel S. Bernstein, Shlomo Zilberstein
• Keywords: AAMAS 2010, Algorithms, Intelligent Agents, Intractable Memory Requirements, Nonlinear Programming