Graph Kernels and Gaussian Processes for Relational Reinforcement Learning

RRL is a relational reinforcement learning system based on Q-learning in relational state-action spaces. It aims to enable agents to learn how to act in an environment that has no natural representation as a tuple of constants. For relational reinforcement learning, the learning algorithm used to approximate the mapping between state-action pairs and their so called Q(uality)-value has to be very reliable, and it has to be able to handle the relational representation of state-action pairs. In this paper we investigate the use of Gaussian processes to approximate the Q-values of state-action pairs. In order to employ Gaussian processes in a relational setting we propose graph kernels as a covariance function between state-action pairs. The standard prediction mechanism for Gaussian processes requires a matrix inversion which can become unstable when the kernel matrix has low rank. These instabilities can be avoided, by employing QR-factorization. This leads to better and more stable per...

• Type: Conference
• Year: 2003
• Where: ILP
• Authors: Thomas Gärtner, Kurt Driessens, Jan Ramon
• Keywords: Gaussian Processes, ILP 2003, Relational Reinforcement, State-action Pairs