Binary Sparse Coding

We study a sparse coding learning algorithm that allows for a simultaneous learning of the data sparseness and the basis functions. The algorithm is derived based on a generative model with binary latent variables instead of continuous-valued latents as used in classical sparse coding. We apply a novel approach to perform maximum likelihood parameter estimation that allows for an efficient estimation of all model parameters. The approach is a new form of variational EM that uses truncated sums instead of factored approximations to the intractable posterior distributions. In contrast to almost all previous versions of sparse coding, the resulting learning algorithm allows for an estimation of the optimal degree of sparseness along with an estimation of the optimal basis functions. We can thus monitor the time-course of the data sparseness during the learning of basis functions. In numerical experiments on artificial data we show that the algorithm reliably extracts the true underlying b...

• Type: Conference
• Year: 2010
• Where: ICA
• Authors: Marc Henniges, Gervasio Puertas, Jörg Bornschein, Julian Eggert, Jörg Lücke
• Keywords: Basis Functions, Data Sparseness, ICA 2010, Signal Processing, Sparse Coding