In this paper we present a face recognition method based on multiway analysis of color face images, which is robust to varying illumination conditions. Illumination changes cause large variations on color in face images. The main idea is to extract features with minimal color variations but with retaining image spatial information. We construct a tensor of color image ensemble, one of its coordinate reflects color mode, and employ the higher-order SVD (a multiway extension of SVD) of the tensor to extract such features. Numerical experiments show that our method outperforms existing subspace analysis methods including principal component analysis (PCA), generalized low rank approximation (GLRAM) and concurrent subspace analysis (CSA), in the task of face recognition under varying illumination conditions. The superiority is even more substantial in the case of small training sample size.