Sciweavers

ACSC
2008
IEEE

An investigation of the state formation and transition limitations for prediction problems in recurrent neural networks

14 years 1 months ago
An investigation of the state formation and transition limitations for prediction problems in recurrent neural networks
Recurrent neural networks are able to store information about previous as well as current inputs. This "memory" allows them to solve temporal problems such as language recognition and sequence prediction, and provide memory elements for larger cognitive networks. It is generally understood that there is an (increasing) relationship between the number of nodes (and connections) in a network, the capabilities of the network, and the amount of training required. However the specifics of this relationship are less well understood. In particular, given that the state of a recurrent network is encoded as a real-valued vector of activation levels, even for small networks there are infinitely many states to choose from. What then determines, or limits, the capabilities of the network? In this paper we use dynamical systems techniques to examine this question in regard to temporal lag. We show that for simple delay problems that the network is unable to solve, the system is able to l...
Angel Kennedy, Cara MacNish
Added 12 Oct 2010
Updated 12 Oct 2010
Type Conference
Year 2008
Where ACSC
Authors Angel Kennedy, Cara MacNish
Comments (0)