Parallel Coordinate Descent for L1-Regularized Loss Minimization

We propose Shotgun, a parallel coordinate descent algorithm for minimizing L1regularized losses. Though coordinate descent seems inherently sequential, we prove convergence bounds for Shotgun which predict linear speedups, up to a problemdependent limit. We present a comprehensive empirical study of Shotgun for Lasso and sparse logistic regression. Our theoretical predictions on the potential for parallelism closely match behavior on real data. Shotgun outperforms other published solvers on a range of large problems, proving to be one of

• Type: Journal
• Year: 2011
• Where: CORR
• Authors: Joseph K. Bradley, Aapo Kyrola, Danny Bickson, Carlos Guestrin
• Keywords: Comprehensive Empirical Study, Coordinate Descent Algorithm, CORR 2011, Education, Shotgun