Why did TD-Gammon Work?

Although TD-Gammon is one of the major successes in machine learning, it has not led to similar impressive breakthroughs in temporal difference learning for other applications or even other games. We were able to replicate some of the success of TD-Gammon, developing a competitive evaluation function on a 4000 parameter feed-forward neural network, without using back-propagation, reinforcement or temporal difference learning methods. Instead we apply simple hill-climbing in a relative fitness environment. These results and further analysis suggest that the surprising success of Tesauro's program had more to do with the co-evolutionary structure of the learning task and the dynamics of the backgammon game itself.

• Type: Conference
• Year: 1996
• Where: NIPS
• Authors: Jordan B. Pollack, Alan D. Blair
• Keywords: Difference Learning Methods, NIPS 1996, NIPS 2007, Similar Impressive Breakthroughs, Temporal Difference Learning