Sciweavers

JMLR
2012
12 years 2 months ago
Deep Learning Made Easier by Linear Transformations in Perceptrons
We transform the outputs of each hidden neuron in a multi-layer perceptron network to have zero output and zero slope on average, and use separate shortcut connections to model th...
Tapani Raiko, Harri Valpola, Yann LeCun
CEC
2011
IEEE
13 years 12 days ago
Stochastic Natural Gradient Descent by estimation of empirical covariances
—Stochastic relaxation aims at finding the minimum of a fitness function by identifying a proper sequence of distributions, in a given model, that minimize the expected value o...
Luigi Malagò, Matteo Matteucci, Giovanni Pi...
CVPR
2010
IEEE
13 years 9 months ago
Natural gradients for deformable registration
We apply the concept of natural gradients to deformable registration. The motivation stems from the lack of physical interpretation for gradients of image-based difference measure...
Darko Zikic, Ali Kamen, Nassir Navab
NN
2002
Springer
107views Neural Networks» more  NN 2002»
14 years 3 days ago
Equivariant nonstationary source separation
Most of source separation methods focus on stationary sources, so higher-order statistics is necessary for successful separation, unless sources are temporally correlated. For non...
Seungjin Choi, Andrzej Cichocki, Shun-ichi Amari
GECCO
2009
Springer
137views Optimization» more  GECCO 2009»
14 years 4 months ago
Efficient natural evolution strategies
Efficient Natural Evolution Strategies (eNES) is a novel alternative to conventional evolutionary algorithms, using the natural gradient to adapt the mutation distribution. Unlike...
Yi Sun, Daan Wierstra, Tom Schaul, Jürgen Sch...
GECCO
2010
Springer
230views Optimization» more  GECCO 2010»
14 years 5 months ago
Exponential natural evolution strategies
The family of natural evolution strategies (NES) offers a principled approach to real-valued evolutionary optimization by following the natural gradient of the expected fitness....
Tobias Glasmachers, Tom Schaul, Yi Sun, Daan Wiers...
ICML
2009
IEEE
15 years 1 months ago
Stochastic search using the natural gradient
Daan Wierstra, Jürgen Schmidhuber, Tom Schaul...