The adaptive k-meteorologists problem and its application to structure learning and feature selection in reinforcement learning

The purpose of this paper is three-fold. First, we formalize and study a problem of learning probabilistic concepts in the recently proposed KWIK framework. We give details of an algorithm, known as the Adaptive k-Meteorologists Algorithm, analyze its sample-complexity upper bound, and give a matching lower bound. Second, this algorithm is used to create a new reinforcement-learning algorithm for factoredstate problems that enjoys significant improvement over the previous state-of-the-art algorithm. Finally, we apply the Adaptive k-Meteorologists Algorithm to remove a limiting assumption in an existing reinforcement-learning algorithm. The effectiveness of our approaches is demonstrated empirically in a couple benchmark domains as well as a robotics navigation problem.

• Type: Conference
• Year: 2009
• Where: ICML
• Authors: Carlos Diuk, Lihong Li, Bethany R. Leffler
• Keywords: ICML 2009, Lower Bound, Machine Learning, Robotics Navigation Problem, Sample-complexity Upper Bound