Material classification of object surfaces from captured image data is an essential problem in computer vision. The present paper proposes a method for stably classifying the material of an object surface based on the distribution of the degree of polarization (DOP) around specular highlight on the surface. We show that the characteristic of spatial distribution for DOP is remarkably different in metal and dielectric substance. Based on the characteristic, an algorithm is presented for effectively classifying any object surface into two material categories from images captured by a digital camera with rotating a polarization filter. The feasibility of the proposed method is verified on experiments using a variety of object surfaces made of different materials.