Several methods have been developed for face detection with certain success, however these tend to fail under difficult conditions such as partial occlusions and changes in orientation and illumination. We propose a novel technique for face detection based on a visual grammar. We define a symbol relational grammar for faces, representing the visual elements of a face and their spatial relations. This grammar is transformed into a Bayesian network (BN) structure and its parameters are obtained from data, i.e., positive and negative examples of faces. Then the BN is used for face detection via probabilistic inference, using as evidence a set of weak detectors for different face components. We evaluated our method on a set of sample images of faces under difficult conditions, and contrasted it with a simplified model without relations, and the AdaBoost face detector. The results show a significant improvement when using our method.