In this paper, we propose a new image clustering algorithm, referred to as Clustering using Local Discriminant Models and Global Integration (LDMGI). To deal with the data points sampled from a nonlinear manifold, for each data point, we construct a local clique comprising this data point and its neighboring data points. Inspired by the Fisher criterion, we use a local discriminant model for each local clique to evaluate the clustering performance of samples within the local clique. To obtain the clustering result, we further propose a unified objective function to globally integrate the local models of all the local cliques. With the unified objective function, spectral relaxation and spectral rotation are used to obtain the binary cluster indicator matrix for all the samples. We show that LDMGI shares a similar objective function with the Spectral Clustering (SC) algorithms, e.g., Normalized Cut (NCut). In contrast to NCut in which the Laplacian matrix is directly calculated based on...