In this paper we present an algorithm for face detection that is based on generic local descriptors (e.g. eyes). A generic descriptor captures the distribution of individual descriptors over a set of samples (training images). This distribution is assumed to be a Gaussian mixture model and is learnt using the minimum description length principle (MDL). A descriptor of an unknown image may then be classified as one of the generic local descriptors. Robustness is achieved by using spatial constraints between locations of descriptors. Experiments show very promising results.