This paper presents an improved distance measure for speaker clustering in speaker diarization systems. The proposed phonetic subspace mixture (PSM) model introduces phonetic information to the BIC distance measure. Therefore, the new PSM model-based BIC distance measure can remove the effect of phonetic content on the diarization results. The typical BIC distance measure can be seen as a special case of the new BIC distance measure. Our experiment results show that the new distance measurement consistently improves the speaker diarization performance on three datasets.