Internal Rewards Mitigate Agent Boundedness

Abstract--Reinforcement learning (RL) research typically develops algorithms for helping an RL agent best achieve its goals-however they came to be defined--while ignoring the relationship of those goals to the goals of the agent designer. We extend agent design to include the meta-optimization problem of selecting internal agent goals (rewards) which optimize the designer's goals. Our claim is that well-designed internal rewards can help improve the performance of RL agents which are computationally bounded in some way (as practical agents are). We present a formal framework for understanding both bounded agents and the meta-optimization problem, and we empirically demonstrate several instances of common agent bounds being mitigated by general internal reward functions.

• Type: Conference
• Year: 2010
• Where: ICML
• Authors: Jonathan Sorg, Satinder P. Singh, Richard Lewis
• Keywords: ICML 2010, Internal Agent Goals, Internal Reward, Machine Learning, Rl Agents