Convex optimization techniques for fitting sparse Gaussian graphical models

We consider the problem of fitting a large-scale covariance matrix to multivariate Gaussian data in such a way that the inverse is sparse, thus providing model selection. Beginning with a dense empirical covariance matrix, we solve a maximum likelihood problem with an l1-norm penalty term added to encourage sparsity in the inverse. For models with tens of nodes, the resulting problem can be solved using standard interior-point algorithms for convex optimization, but these methods scale poorly with problem size. We present two new algorithms aimed at solving problems with a thousand nodes. The first, based on Nesterov's first-order algorithm, yields a rigorous complexity estimate for the problem, with a much better dependence on problem size than interior-point methods. Our second algorithm uses block coordinate descent, updating row/columns of the covariance matrix sequentially. Experiments with genomic data show that our method is able to uncover biologically interpretable conne...

• Type: Conference
• Year: 2006
• Where: ICML
• Authors: Onureena Banerjee, Laurent El Ghaoui, Alexandre d'Aspremont, Georges Natsoulis
• Keywords: Empirical Covariance Matrix, ICML 2006, Large-scale Covariance Matrix, Machine Learning, Maximum Likelihood Problem