Exact solutions of interactive POMDPs using behavioral equivalence

We present a method for transforming the infinite interactive state space of interactive POMDPs (I-POMDPs) into a finite one, thereby enabling the computation of exact solutions. I-POMDPs allow sequential decision making in multi-agent environments by modeling other agents' beliefs, capabilities, and preferences as part of the interactive state space. Since beliefs are allowed to be arbitrarily nested and are continuous, it is not possible to compute optimal solutions using value iteration as in POMDPs. We present a method that transforms the original state space into a finite one by grouping the other agents' behaviorally equivalent models into equivalence classes. This enables us to compute the complete optimal solution for the I-POMDP, which may be represented as a policy graph. We illustrate our method using the multi-agent Tiger problem and discuss features of the solution.

• Type: Conference
• Year: 2006
• Where: ATAL
• Authors: Bharaneedharan Rathnasabapathy, Prashant Doshi, Piotr J. Gmytrasiewicz
• Keywords: ATAL 2006, Intelligent Agents, Interactive State Space, Optimal Solution, State Space