Finite horizon risk sensitive MDP and linear programming

— In the context of standard Markov decision processes (MDPs), the connection between Dynamic Program (DP) and Linear Program (LP) is well understood and is well established under sufficiently general conditions. LP based approach facilitates solving the constrained MDPs. Multiplicative or Risk sensitive MDPs, introduced to control the fluctuations/variations around the expected value, are relatively less studied objects. DP equations are considerably well understood even in the context of Risk MDPs, however the LP connection is not known. We consider a finite horizon risk MDP problem and establish the connections between the DP and LP approaches. We augment the state space with a suitable component, to obtain the optimal policies for constrained risk MDPs. We apply this results to a server selection problem in Ber/M/K/K queues, with a constraint on the utilization of the fast server. We discuss some interesting structural properties of the risk optimal policies.

• Type: Journal
• Year: 2015
• Where: CDC
• Authors: Atul Kumar, Veeraruna Kavitha, N. Hemachandra
• Keywords: CDC 2015, Control Systems