Weight initialization methods for multilayer feedforward

In this paper, we present the results of an experimental comparison among seven different weight initialization methods in twelve different problems. The comparison is performed by measuring the speed of convergence, the generalization capability and the probability of successful convergence. It is not usual to find an evaluation of the three properties in the papers on weight initialization. The training algorithm was Backpropagation (BP) with a hyperbolic tangent transfer function. We found that the performance can be improved with respect to the usual initialization scheme.

• Type: Conference
• Year: 2001
• Where: ESANN
• Authors: Mercedes Fernández-Redondo, Carlos Hernández-Espinosa
• Keywords: ESANN 2001, ESANN 2007, Usual Initialization Scheme, Weight Initialization, Weight Initialization Methods