On the Expressive Power of Deep Architectures

Deep architectures are families of functions corresponding to deep circuits. Deep Learning algorithms are based on parametrizing such circuits and tuning their parameters so as to approximately optimize some training objective. Whereas it was thought too difficult to train deep architectures, several successful algorithms have been proposed in recent years. We review some of the theoretical motivations for deep architectures, as well as some of their practical successes, and propose directions of investigations to address some of the remaining challenges. 1 Learning Artificial Intelligence An intelligent agent takes good decisions. In order to do so it needs some form of knowledge. Knowledge can be embodied into a function that maps inputs and states to states and actions. If we saw an agent that always took what one would consider as the good decisions, we would qualify the agent as intelligent. Knowledge can be explicit, as in the form of symbolically expressed rules and facts of ex...

• Type: Journal
• Year: 2011
• Where: ALT
• Authors: Yoshua Bengio, Olivier Delalleau
• Keywords: ALT 2011, Animal Brains, Intelligent Knowledge, Machine Learning, Sense Knowledge