Generalized Hebbian Algorithm for Incremental Singular Value Decomposition in Natural Language Processing

An algorithm based on the Generalized Hebbian Algorithm is described that allows the singular value decomposition of a dataset to be learned based on single observation pairs presented serially. The algorithm has minimal memory requirements, and is therefore interesting in the natural language domain, where very large datasets are often used, and datasets quickly become intractable. The technique is demonstrated on the task of learning word and letter bigram pairs from text.

• Type: Conference
• Year: 2006
• Where: EACL
• Authors: Genevieve Gorrell
• Keywords: EACL 2006, Generalized Hebbian Algorithm, Natural Language Processing, Single Observation Pairs, Singular Value Decomposition