Herding dynamical weights to learn

A new "herding" algorithm is proposed which directly converts observed moments into a sequence of pseudo-samples. The pseudosamples respect the moment constraints and may be used to estimate (unobserved) quantities of interest. The procedure allows us to sidestep the usual approach of first learning a joint model (which is intractable) and then sampling from that model (which can easily get stuck in a local mode). Moreover, the algorithm is fully deterministic, avoiding random number generation) and does not need expensive operations such as exponentiation.

• Type: Conference
• Year: 2009
• Where: ICML
• Authors: Max Welling
• Keywords: ICML 2009, Machine Learning, Moment Constraints, Observed Moments, Random Number Generation