Abstract. In this paper, we describe an approach for the automatic modality classification in medical image retrieval task of the 2010 CLEF cross-language image retrieval campaign (ImageCLEF). This work is focused on the process of feature extraction from medical images and fusion the different extracted visual feature and textual feature for modality classification. To extract visual features from the images, we used histogram descriptor of edge, gray or color intensity and block-based variation as global features and SIFT histogram as local feature, and the binary histogram of some predefined vocabulary words for image captions is used for textual feature. Then we combine the different features using normalized kernel functions for SVM classification. The proposed algorithm is evaluated by the provided modality dataset by ImageCLEF2010.