Limits of Multi-Discounted Markov Decision Processes

Markov decision processes (MDPs) are controllable discrete event systems with stochastic transitions. The payoff received by the controller can be evaluated in different ways, depending on the payoff function the MDP is equipped with. For example a mean–payoff function evaluates average performance, whereas a discounted payoff function gives more weights to earlier performance by means of a discount factor. Another well–known example is the parity payoff function which is used to encode logical specifications [14]. Surprisingly, parity and mean–payoff MDPs share two non–trivial properties: they both have pure stationary optimal strategies [4, 15] and they both are approximable by discounted MDPs with multiple discount factors (multi– discounted MDPs) [5, 15]. In this paper we unify and generalize these results. We introduce a new class of payoff functions called the priority weighted payoff functions, which are generalization of both parity and mean–payoff functions. We p...

• Type: Conference
• Year: 2007
• Where: LICS
• Authors: Hugo Gimbert, Wieslaw Zielonka
• Keywords: Computer Science, Discount Factors, LICS 2007, Parity Payoff Function, Payoff Functions