Natural Language Processing (almost) from Scratch

We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling, achieving or exceeding state-of-theart performance in each on four benchmark tasks. Our goal was to design a flexible architecture that can learn representations useful for the tasks, thus avoiding excessive taskspecific feature engineering (and therefore disregarding a lot of prior knowledge). Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabelled training data. This work is then used as a basis for building a freely available tagging system with excellent performance while requiring minimal computational resources.

• Type: Journal
• Year: 2011
• Where: CORR
• Authors: Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, Pavel P. Kuksa
• Keywords: CORR 2011, Education, Excessive Taskspecific Feature, Unified Neural Network, Various Natural Language