Bayesian reinforcement learning in continuous POMDPs with application to robot navigation

— We consider the problem of optimal control in continuous and partially observable environments when the parameters of the model are not known exactly. Partially Observable Markov Decision Processes (POMDPs) provide a rich mathematical model to handle such environments but require a known model to be solved by most approaches. This is a limitation in practice as the exact model parameters are often difficult to specify exactly. We adopt a Bayesian approach where a posterior distribution over the model parameters is maintained and updated through experience with the environment. We propose a particle filter algorithm to maintain the posterior distribution and an online planning algorithm, based on trajectory sampling, to plan the best action to perform under the current posterior. The resulting approach selects control actions which optimally trade-off between 1) exploring the environment to learn the model, 2) identifying the system’s state, and 3) exploiting its knowledge in or...

• Type: Conference
• Year: 2008
• Where: ICRA
• Authors: Stéphane Ross, Brahim Chaib-draa, Joelle Pineau
• Keywords: ICRA 2008, Model, Model Parameters, Posterior Distribution, Robotics