Abstract. In this paper, we focus on a novel NN/HMM architecture for continuous speech recognition. The architecture incorporates a neural feature extraction to gain more discriminative feature vectors for the underlying HMM system. The feature extraction can be chosen either linear or non-linear and can incorporate recurrent connections. With this hybrid system, that is an extension of a state-of-the-art continuous HMM system, we managed to significantly outperform these standard systems. Experimental results show a relative error reduction of about 10% on a remarkably good recognition system based on continuous HMMs for the Resource Management 1000-word continuous speech recognition task.