In this paper, we present a new independent component analysis mixture vector quantization (ICAMVQ) method to summarize the video content. In particular, independent component analysis (ICA) is applied first to explore the characteristics of video data and build a compact 2D feature space. A new ICAMVQ method is then developed to find the optimized quantization codebook in ICA subspace. The optimal codebook size is determined by Bayes information criterion (BIC). The frames that are the nearest neighbors to the quanta in the ICAMVQ codebook are sampled to summarize the video. A 2D kD-tree is employed to index the feature space and accelerate the nearest-neighbor search. Experimental results show that our method is practically effective and computationally efficient to build a video summarization system.