Credit Assignment through Time: Alternatives to Backpropagation

Learning to recognize or predict sequences using long-term context has many applications. However, practical and theoretical problems are found in training recurrent neural networks to performtasks in which input/output dependencies span longintervals. Starting from a mathematical analysis of the problem, we consider and compare alternative algorithms and architectures on tasks for which the span of the input/output dependencies can be controlled. Results on the new algorithms show performance qualitatively superior to that obtained with backpropagation.

• Type: Conference
• Year: 1993
• Where: NIPS
• Authors: Yoshua Bengio, Paolo Frasconi
• Keywords: Dependencies Span Longintervals, Input/output Dependencies, NIPS 1993, NIPS 2007, Recurrent Neural Networks