Learning to overtake in TORCS using simple reinforcement learning

In modern racing games programming non-player characters with believable and sophisticated behaviors is getting increasingly challenging. Recently, several works in the literature suggested that computational intelligence can provide effective solutions to support the development process of NPCs. In this paper, we applied the Behavior Analysis and Training (BAT) methodology to define a behavior-based architecture for the NPCs in The Open Racing Car Simulator (TORCS), a well-known open source racing game. Then, we focused on two major overtaking behaviors: (i) the overtaking of a fast opponent either on a straight stretch or on a large bend; (ii) the overtaking on a tight bend, which typically requires a rather advanced braking policy. We applied simple reinforcement learning, namely Q-learning, to learn both these overtaking behaviors. We tested our approach in several overtaking situations and compared the learned behaviors against one of the best NPC provided with TORCS. Our results ...

• Type: Conference
• Year: 2010
• Where: CEC
• Authors: Daniele Loiacono, Alessandro Prete, Pier Luca Lanzi, Luigi Cardamone
• Keywords: Artificial Intelligence, CEC 2010, Overtaking, Overtaking Behaviors, Sophisticated Behaviors