Over the past decade a large variety of hardware has been designed to exploit the inherent parallelism of the artificial neural network models. This paper presents an overview of neural network hardware. Neural network basics, hardware specification and performance evaluation are introduced. Major categories of neural network architectures are reviewed. Two examples of neurohardware, CNAPS and SYNAPSE-1, and some real-world applications of neural network hardware, are described in detail. The challenges and future of hardware implementation of neural networks are also discussed.