We propose a novel utterance comparison model based on probability theory and factor analysis that computes the likelihood of two speech utterances originating from the same speaker. The model depends only on a set of statistics extracted from each utterance and can ef ciently compare utterances using these statistics without requiring the inde nite storage of speech features. We apply the model as a distance metric for speaker clustering in the CALLHOME telephone conversation corpus to achieve competitive results compared to three other known similarity measures: the Generalized Likelihood Ratio, Cross-Likelihood Ratio, and eigenvoice distance.