Non-negative matrix factorization (NMF) is a powerful feature extraction method for finding parts-based, linear representations of non-negative data . Inherently, it is unsupervised learning algorithm. That is to say, the classical NMF algorithm does not respect the class-specific information. This paper presents an improvement of the classical NMF approach by imposing Fisher constraints. This results in a two-step factorization procedure for discriminative feature extraction. Furthermore, weighting factors for each pairwise scatter is introduced to include the confusability information into the between class covariance matrix. The proposed method has been applied to the problem of face and handwritten digit recognition and the experiments give better performance than previous methods.