This paper presents motion field histograms as a new way of extracting facial features and modeling expressions. Feature are based on local receptive field histograms, which are robust against errors in rotation, translation and scale changes during image alignment. Motion information is also incorporated into the histograms by using difference images instead of raw images. We take the principal components of these histograms of selected facial regions and use the top 20 eigenvectors for compact representation. The eigencoefficients are then used to model the temporal structure of different facial expressions from real-life data in the presence of translational and rotational errors that arise from head-tracking. The results demonstrate a 44% averageperformance increase over traditionalopticflowmethod for expressions extracted from unconstrained interactions.