In this paper we propose a classification-based method towards the segmentation of diffusion tensor images. We use Support Vector Machines to classify diffusion tensors and we extend linear classification to the non linear case. To this end, we discuss and evaluate three different classes of kernels on the space of symmetric definite positive matrices that are well suited for the classification of tensor data. We impose spatial constraints by means of a Markov random field model that takes into account the result of SVM classification. Experimental results are provided for diffusion tensor images of human skeletal muscles. They demonstrate the potential of our method in discriminating the different muscle groups.