Image spam is a new obfuscating method which spammers invented to more effectively bypass conventional text based spam filters. In this paper, we extract local invariant features of images and run a one-class SVM classifier which uses the pyramid match kernel as the kernel function to detect image spam. Experimental results demonstrate that our algorithm is effective for fighting image spam. Categories and Subject Descriptors H.3.3 [Information Storage and Retrieval]: Information Search and Retrieval ?Clustering, Information filtering. General Terms: Algorithms, Security. Keywords Image spam, pyramid match kernel, local invariant features.