Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition

We present an unsupervised method for learning a hierarchy of sparse feature detectors that are invariant to small shifts and distortions. The resulting feature extractor consists of multiple convolution filters, followed by a pointwise sigmoid non-linearity, and a feature-pooling layer that computes the max of each filter output within adjacent windows. A second level of larger and more invariant features is obtained by training the same algorithm on patches of features from the first level. Training a supervised classifier on these features yields 0.64% error on MNIST, and 54% average recognition rate on Caltech 101 with 30 training samples per category. While the resulting architecture is similar to convolutional networks, the layer-wise unsupervised training procedure alleviates the over-parameterization problems that plague purely supervised learning procedures, and yields good performance with very few labeled training samples.

• Type: Conference
• Year: 2007
• Where: CVPR
• Authors: Marc'Aurelio Ranzato, Fu Jie Huang, Y-Lan Boureau, Yann LeCun
• Keywords: Computer Vision, CVPR 2007, Pointwise Sigmoid Non-linearity, Sparse Feature Detectors, Training Samples, Unsupervised Training Procedure, Yields 0.64% Error