We propose a new distance measure for face recognition and human gait recognition. Each probe image (a face image or an average human silhouette image) is represented as a set of local features uniformly sampled over a grid with fixed spacing, and each gallery image is represented as a set of local features sampled at each pixel. We formulate an integer programming problem to compute the distance (referred to as Image-to-Class distance) from one probe image to all the the gallery images belonging to a certain class, in which any feature of the probe image can be matched to only one feature from one of the gallery images. Considering computational efficiency as well as the fact that face images or average human silhouette images are roughly aligned in the preprocessing step, we also enforce a spatial neighborhood constraint by only allowing neighboring features that are within a given spatial distance to be considered for feature matching. The integer programming problem is further tre...