Phase transitions in Vector Quantization

Abstract. We study Winner-Takes-All and rank based Vector Quantization along the lines of the statistical physics of off-line learning. Typical behavior of the system is obtained within a model where high-dimensional training data are drawn from a mixture of Gaussians. The analysis becomes exact in the simplifying limit of high training temperature. Our main findings concern the existence of phase transitions, i.e. a critical or discontinuous dependence of VQ performance on the training set size. We show how the nature and properties of the transition depend on the number of prototypes and the control parameter of rank based cost functions.

• Type: Conference
• Year: 2008
• Where: ESANN
• Authors: Aree Witoelar, Anarta Ghosh, Michael Biehl
• Keywords: ESANN 2008, Main Findings Concern, Neural Networks, Rank Based Cost, Training Set Size