Reinforcement Learning and the Bayesian Control Rule

We present an actor-critic scheme for reinforcement learning in complex domains. The main contribution is to show that planning and I/O dynamics can be separated such that an intractable planning problem reduces to a simple multi-armed bandit problem, where each lever stands for a potentially arbitrarily complex policy. Furthermore, we use the Bayesian control rule to construct an adaptive bandit player that is universal with respect to a given class of optimal bandit players, thus indirectly constructing an adaptive agent that is universal with respect to a given class of policies.

• Type: Journal
• Year: 2011
• Where: AGI
• Authors: Pedro Alejandro Ortega, Daniel Alexander Braun, Simon J. Godsill
• Keywords: AGI 2011, Artificial Intelligence, Bandit, Planning Problem, Reinforcement