Second-order Learning Algorithm with Squared Penalty Term

This paper compares three penalty terms with respect to the efficiency of supervised learning, by using first- and second-order learning algorithms. Our experiments showed that for a reasonably adequate penalty factor, the combination of the squared penalty term and the second-order learning algorithm drastically improves the convergence performance more than 20 times over the other combinations, at the same time bringing about a better generalization performance.

• Type: Conference
• Year: 1996
• Where: NIPS
• Authors: Kazumi Saito, Ryohei Nakano
• Keywords: Adequate Penalty Factor, NIPS 1996, NIPS 2007, Penalty Term, Second-order Learning Algorithm