Evolving Memory Cell Structures for Sequence Learning

The best recent supervised sequence learning methods use gradient descent to train networks of miniature nets called memory cells. The most popular cell structure seems somewhat arbitrary though. Here we optimize its topology with a multi-objective evolutionary algorithm. The fitness function reflects the structure’s usefulness for learning various formal languages. The evolved cells help to understand crucial structural features that aid sequence learning.

• Type: Conference
• Year: 2009
• Where: ICANN
• Authors: Justin Bayer, Daan Wierstra, Julian Togelius, Jürgen Schmidhuber
• Keywords: Artificial Intelligence, ICANN 2009, Nets Called Memory, Sequence Learning, Sequence Learning Methods