A new algorithm for learning in piecewise-linear neural networks

Piecewise-linear (PWL) neural networks are widely known for their amenability to digital implementation. This paper presents a new algorithm for learning in PWL networks consisting of a single hidden layer. The approach adopted is based upon constructing a continuous PWL error function and developing an efficient algorithm to minimize it. The algorithm consists of two basic stages in searching the weight space. The first stage of the optimization algorithm is used to locate a point in the weight space representing the intersection of N linearly independent hyperplanes, with N being the number of weights in the network. The second stage is then called to use this point as a starting point in order to continue searching by moving along the single-dimension boundaries between the different linear regions of the error function, hopping from one point (representing the intersection of N hyperplanes) to another. The proposed algorithm exhibits significantly accelerated convergence, as compa...

• Type: Journal
• Year: 2000
• Where: NN
• Authors: Emad Gad, Amir F. Atiya, Samir I. Shaheen, Ayman El-Dessouki
• Keywords: Algorithm, Error Function, Neural Networks, NN 2000, Weight Space