Online Learning for Group Lasso

We develop a novel online learning algorithm for the group lasso in order to efficiently find the important explanatory factors in a grouped manner. Different from traditional batch-mode group lasso algorithms, which suffer from the inefficiency and poor scalability, our proposed algorithm performs in an online mode and scales well: at each iteration one can update the weight vector according to a closed-form solution based on the average of previous subgradients. Therefore, the proposed online algorithm can be very efficient and scalable. This is guaranteed by its low worst-case time complexity and memory cost both in the order of O(d), where d is the number of dimensions. Moreover, in order to achieve more sparsity in both the group level and the individual feature level, we successively extend our online system to efficiently solve a number of variants of sparse group lasso models. We also show that the online system is applicable to other group lasso models, such as the group lass...

• Type: Conference
• Year: 2010
• Where: ICML
• Authors: Haiqin Yang, Zenglin Xu, Irwin King, Michael R. Lyu
• Keywords: Group Lasso, Group Lasso Algorithms, Group Lasso Models, ICML 2010, Machine Learning