Some extensions of score matching

Many probabilistic models are only defined up to a normalization constant. This makes maximum likelihood estimation of the model parameters very difficult. Typically, one then has to resort to Markov Chain Monte Carlo methods, or approximations of the normalization constant. Previously, a method called score matching was proposed for computationally efficient yet (locally) consistentestimationofsuchmodels.Thebasicformofscorematchingisvalid,however,onlyformodelswhichdefineadifferentiable probability density function over Rn. Therefore, some extensions of the framework are proposed. First, a related method for binary variables is proposed. Second, it is shown how to estimate non-normalized models defined in the non-negative real domain, i.e. Rn +. As a further result, it is shown that the score matching estimator can be obtained in closed form for some exponential families. © 2006 Elsevier B.V. All rights reserved.

• Type: Journal
• Year: 2007
• Where: CSDA
• Authors: Aapo Hyvärinen
• Keywords: CSDA 2007, Many Probabilistic Models, Markov Chain Monte Carlo, Maximum Likelihood Estimation