A Hybrid Reinforcement Learning Approach to Autonomic Resource Allocation

— Reinforcement Learning (RL) provides a promising new approach to systems performance management that differs radically from standard queuing-theoretic approaches making use of explicit system performance models. In principle, RL can automatically learn high-quality management policies without an explicit performance model or traffic model, and with little or no built-in system specific knowledge. In our original work [1], [2], [3] we showed the feasibility of using online RL to learn resource valuation estimates (in lookup table form) which can be used to make high-quality server allocation decisions in a multi-application prototype Data Center scenario. The present work shows how to combine the strengths of both RL and queuing models in a hybrid approach, in which RL trains offline on data collected while a queuing model policy controls the system. By training offline we avoid suffering potentially poor performance in live online training. We also now use RL to train nonlinear...

• Type: Conference
• Year: 2006
• Where: ICAC
• Authors: Gerald Tesauro, Nicholas K. Jong, Rajarshi Das, Mohamed N. Bennani
• Keywords: Autonomic Computing, ICAC 2006, Online Rl, Performance Model, RL Training