The IM Algorithm: A Variational Approach to Information Maximization

The maximisation of information transmission over noisy channels is a common, albeit generally computationally difficult problem. We approach the difficulty of computing the mutual information for noisy channels by using a variational approximation. The resulting IM algorithm is analagous to the EM algorithm, yet maximises mutual information, as opposed to likelihood. We apply the method to several practical examples, including linear compression, population encoding and CDMA.

• Type: Conference
• Year: 2003
• Where: NIPS
• Authors: David Barber, Felix V. Agakov
• Keywords: Mutual Information, NIPS 2003, NIPS 2007, Noisy Channels, Variational Approximation