Sciweavers

JMLR
2012

Noise-Contrastive Estimation of Unnormalized Statistical Models, with Applications to Natural Image Statistics

12 years 2 months ago
Noise-Contrastive Estimation of Unnormalized Statistical Models, with Applications to Natural Image Statistics
We consider the task of estimating, from observed data, a probabilistic model that is parameterized by a finite number of parameters. In particular, we are considering the situation where the model probability density function is unnormalized. That is, the model is only specified up to the partition function. The partition function normalizes a model so that it integrates to one for any choice of the parameters. However, it is often impossible to obtain it in closed form. Gibbs distributions, Markov and multi-layer networks are examples of models where analytical normalization is often impossible. Maximum likelihood estimation can then not be used without resorting to numerical approximations which are often computationally expensive. We propose here a new objective function for the estimation of both normalized and unnormalized models. The basic idea is to perform nonlinear logistic regression to discriminate between the observed data and some artificially generated noise. With th...
Michael Gutmann, Aapo Hyvärinen
Added 27 Sep 2012
Updated 27 Sep 2012
Type Journal
Year 2012
Where JMLR
Authors Michael Gutmann, Aapo Hyvärinen
Comments (0)