An Algorithmic Theory of Learning: Robust Concepts and Random Projection

We study the phenomenon of cognitive learning from an algorithmic standpoint. How does the brain effectively learn concepts from a small number of examples despite the fact that each example contains a huge amount of information? We provide a novel algorithmic analysis via a model of robust concept learning (closely related to "margin classifiers"), and show that a relatively small number of examples are sufficient to learn rich concept classes. The new algorithms have several advantages -- they are faster, conceptually simpler, and resistant to low levels of noise. For example, a robust half-space can be learned in linear time using only a constant number of training examples, regardless of the number of attributes. A general (algorithmic) consequence of the model, that "more robust concepts are easier to learn", is supported by a multitude of psychological studies.

• Type: Conference
• Year: 1999
• Where: FOCS
• Authors: Rosa I. Arriaga, Santosh Vempala
• Keywords: Algorithmic Standpoint, FOCS 1999, Robust Concepts, Small Number, Theoretical Computer Science