Kernel principal component analysis (KPCA) extracts features of samples with an efficiency in inverse proportion to the size of the training sample set. In this paper, we develop a novel method to improve KPCA-based feature extraction. The developed method is the first one that is methodologically consistent with KPCA. Experiments on several benchmark datasets illustrate that the feature extraction process derived from the novel method is much more efficient than that associated with KPCA. Moreover, the classification accuracy generated from the developed method is similar to that of KPCA. r 2006 Elsevier B.V. All rights reserved.