Improvements to the Conventional Layer-by-Layer BP Algorithm

This paper points out some drawbacks and proposes some modifications to the conventional layer-by-layer BP algorithm. In particular, we present a new perspective to the learning rate, which is to use a heuristic rule to define the learning rate so as to update the weights. Meanwhile, to pull the algorithm out of saturation area and prevent it from converging to a local minimum, a momentum term is introduced to the former algorithm. And finally the effectiveness and efficiency of the proposed method are demonstrated by two benchmark examples.

• Type: Conference
• Year: 2005
• Where: ICIC
• Authors: Xu-Qin Li, Fei Han, Tat-Ming Lok, Michael R. Lyu, Guang-Bin Huang
• Keywords: Heuristic Rule, ICIC 2005, Layer-by-layer Bp Algorithm, Learning Rate