We propose a novel, supervised feature extraction procedure, based on an unbiased estimator of the Hilbert-Schmidt independence criterion (HSIC). The proposed procedure can be directly applied to single-label or multi-label data, also the kernelized version can be applied to any data type, on which a positive definite kernel function has been defined. Computer experiments with various classification data sets reveal that our approach can be applied more efficiently than the alternative ones.