Analysis and Generalizations of the Linearized Bregman Method

This paper analyzes and improves the linearized Bregman method for solving the basis pursuit and related sparse optimization problems. The analysis shows that the linearized Bregman method has the exact regularization property, namely, it converges to an exact solution of the basis pursuit problem whenever its smooth parameter is greater than a certain value. The analysis is based on showing that the linearized Bregman algorithm is equivalent to gradient descent applied to a certain dual formulation. This result motivates generalizations of the algorithm enabling the use of gradient-based optimization techniques such as line search, Barzilai-Borwein, L-BFGS, and nonlinear conjugate gradient methods. In the numerical simulations, the two proposed implementations, one using Barzilai-Borwein steps with nonmonotone line search and the other using L-BFGS, gave more accurate solutions in much shorter times than the existing basic implementation of the linearized Bregman method (with a so-ca...

• Type: Journal
• Year: 2010
• Where: SIAMIS
• Authors: Wotao Yin
• Keywords: Basis Pursuit, Bregman Method, Line Search, SIAMIS 2010