Learning dynamics and robustness of vector quantization and neural gas

Various alternatives have been developed to improve the Winner-Takes-All (WTA) mechanism in vector quantization, including the Neural Gas (NG). However, the behavior of these algorithms including their learning dynamics, robustness with respect to initialization, asymptotic results, etc. has only partially been studied in a rigorous mathematical analysis. The theory of on-line learning allows for an exact mathematical description of the training dynamics in model situations. We demonstrate using a system of three competing prototypes trained from a mixture of Gaussian clusters that the Neural Gas can improve convergence speed and achieves robustness to initial conditions. However, depending on the structure of the data, the Neural Gas does not always obtain the best asymptotic quantization error. Key words: Vector quantization, Clustering, Online learning, Winner-Takes-All algorithms, Neural Gas

• Type: Journal
• Year: 2008
• Where: IJON
• Authors: Aree Witoelar, Michael Biehl, Anarta Ghosh, Barbara Hammer
• Keywords: IJON 2008, Neural Gas, Rigorous Mathematical Analysis, Vector Quantization