We propose using the proximity distribution of vectorquantized local feature descriptors for object and category recognition. To this end, we introduce a novel "proximity distribution kernel" that naturally combines local geometric as well as photometric information from images. It satisfies Mercer's condition and can therefore be readily combined with a support vector machine to perform visual categorization in a way that is insensitive to photometric and geometric variations, while retaining significant discriminative power. In particular, it improves on the results obtained both with geometrically unconstrained "bags of features" approaches, as well as with over-constrained "affine procrustes." Indeed, we test this approach on several challenging data sets, including Graz-01, Graz-02, and the PASCAL challenge. We registered the average performance