Back Propagation is Sensitive to Initial Conditions

This paper explores the effect of initial weight selection on feed-forward networks learning simple functions with the back-propagation technique. We first demonstrate, through the use of Monte Carlo techniques, that the magnitude of the initial condition vector (in weight space) is a very significant parameter in convergence time variability. In order to further understand this result, additional deterministic experiments were performed. The results of these experiments demonstrate the extreme sensitivity of back propagation to initial weight configuration. 

• Type: Conference
• Year: 1990
• Where: NIPS
• Authors: John F. Kolen, Jordan B. Pollack
• Keywords: Initial Condition Vector, Initial Weight, Initial Weight Selection, NIPS 1990, NIPS 2007