An interior-point stochastic approximation method and an L1-regularized delta rule

The stochastic approximation method is behind the solution to many important, actively-studied problems in machine learning. Despite its farreaching application, there is almost no work on applying stochastic approximation to learning problems with general constraints. The reason for this, we hypothesize, is that no robust, widely-applicable stochastic approximation method exists for handling such problems. We propose that interior-point methods are a natural solution. We establish the stability of a stochastic interior-point approximation method both analytically and empirically, and demonstrate its utility by deriving an on-line learning algorithm that also performs feature selection via L1 regularization.

• Type: Conference
• Year: 2008
• Where: NIPS
• Authors: Peter Carbonetto, Mark Schmidt, Nando de Freitas
• Keywords: Information Technology, NIPS 2008, Stochastic Approximation, Stochastic Approximation Method, Stochastic Interior-point Approximation