Guided by the goal of obtaining an optimization algorithm that is both fast and yields good generalization, we study the descent direction maximizing the decrease in generalizatio...
Nicolas Le Roux, Pierre-Antoine Manzagol, Yoshua B...
We consider the gradient method xt+1 = xt + t(st + wt), where st is a descent direction of a function f : n and wt is a deterministic or stochastic error. We assume that f is Lip...
This paper presents an online support vector machine (SVM) that uses the stochastic meta-descent (SMD) algorithm to adapt its step size automatically. We formulate the online lear...
S. V. N. Vishwanathan, Nicol N. Schraudolph, Alex ...
We present a new method for regularized convex optimization and analyze it under both online and stochastic optimization settings. In addition to unifying previously known firstor...
John Duchi, Shai Shalev-Shwartz, Yoram Singer, Amb...
We apply Stochastic Meta-Descent (SMD), a stochastic gradient optimization method with gain vector adaptation, to the training of Conditional Random Fields (CRFs). On several larg...
S. V. N. Vishwanathan, Nicol N. Schraudolph, Mark ...