Sparse Coding in Sparse Winner Networks

This paper investigates a mechanism for reliable generation of sparse code in a sparsely connected, hierarchical, learning memory. Activity reduction is accomplished with local competitions that suppress activities of unselected neurons so that costly global competition is avoided. The learning ability and the memory characteristics of the proposed winner-take-all network and an oligarchy-take-all network are demonstrated using experimental results. The proposed models have the features of a learning memory essential to the development of machine intelligence. Keywords. sparse coding, winner-takes-all, Hebbian learning in sparse structure, hierarchical self-organizing memory

• Type: Conference
• Year: 2007
• Where: ISNN
• Authors: Janusz A. Starzyk, Yinyin Liu, David D. Vogel
• Keywords: Hierarchical Self-organizing Memory, ISNN 2007, Sparse, Sparse Code