A theoretical analysis of Model-Based Interval Estimation

Several algorithms for learning near-optimal policies in Markov Decision Processes have been analyzed and proven efficient. Empirical results have suggested that Model-based Interval Estimation (MBIE) learns efficiently in practice, effectively balancing exploration and exploitation. This paper presents the first theoretical analysis of MBIE, proving its efficiency even under worst-case conditions. The paper also introduces a new performance metric, average loss, and relates it to its less "online" cousins from the literature.

• Type: Conference
• Year: 2005
• Where: ICML
• Authors: Alexander L. Strehl, Michael L. Littman
• Keywords: ICML 2005, Machine Learning, Markov Decision Processes, Model-based Interval Estimation, Near-optimal Policies