TAP Gibbs Free Energy, Belief Propagation and Sparsity

The adaptive TAP Gibbs free energy for a general densely connected probabilistic model with quadratic interactions and arbritary single site constraints is derived. We show how a specific sequential minimization of the free energy leads to a generalization of Minka's expectation propagation. Lastly, we derive a sparse representation version of the sequential algorithm. The usefulness of the approach is demonstrated on classification and density estimation with Gaussian processes and on an independent component analysis problem.

• Type: Conference
• Year: 2001
• Where: NIPS
• Authors: Lehel Csató, Manfred Opper, Ole Winther
• Keywords: Free Energy, Minka's Expectation Propagation, NIPS 2001, NIPS 2007, Specific Sequential Minimization