Solving Factored MDPs via Non-Homogeneous Partitioning

This paper describes an algorithm for solving large state-space MDPs (represented as factored MDPs) using search by successive refinement in the space of non-homogeneous partitions. Homogeneity is defined in terms of bisimulation and reward equivalence within blocks of a partition. Since homogeneous partitions that define equivalent reduced state-space MDPs can have a large number of blocks, we relax the requirement of homogeneity. The algorithm constructs approximate aggregate MDPs from non-homogeneous partitions, solves the aggregate MDPs exactly, and then uses the resulting value functions as part of a heuristic in refining the current best non-homogeneous partition. We outline the theory motivating the use of this heuristic and present empirical results and comparisons.

• Type: Conference
• Year: 2001
• Where: IJCAI
• Authors: Kee-Eung Kim, Thomas Dean
• Keywords: Aggregate Mdps, IJCAI 2001, IJCAI 2007, Non-homogeneous Partition, State-space Mdps