The expectation maximization (EM) algorithm is widely used in the Gaussian mixture model (GMM) as the state-of-art statistical modeling technique. Like the classical EM method, the proposed EM-Information Theoretic algorithm (EM-IT) adapts means, covariances and weights, however this process is not conducted directly on feature vectors but on a smaller set of centroids derived by the information theoretic procedure, which simultaneously minimizes the divergence between the Parzen estimates of the feature vector's distribution within a given Gaussian component and the centroids distribution within the same Gaussian component. The EM-IT algorithm was applied to the speaker verification problem using NIST 2004 speech corpus and the MFCC with dynamic features. The results showed an improvement of the