Convergence of the Wake-Sleep Algorithm

The W-S (Wake-Sleep) algorithm is a simple learning rule for the models with hidden variables. It is shown that this algorithm can be applied to a factor analysis model which is a linear version of the Helmholtz machine. But even for a factor analysis model, the general convergence is not proved theoretically. In this article, we describe the geometrical understanding of the W-S algorithm in contrast with the EM (ExpectationMaximization) algorithm and the em algorithm. As the result, we prove the convergence of the W-S algorithm for the factor analysis model. We also show the condition for the convergence in general models.

• Type: Conference
• Year: 1998
• Where: NIPS
• Authors: Shiro Ikeda, Shun-ichi Amari, Hiroyuki Nakahara
• Keywords: Algorithm, Factor Analysis Model, NIPS 1998, NIPS 2007, Simple Learning Rule