This article presents an appearance based method to detect automatically facial actions. Our approach focuses on reducing features sensitivity to identity of the subject. We compute from an expressive image a Local Gabor Binary Pattern (LGBP) histogram and synthesize a LGBP histogram approaching the one we would compute on a neutral face. Difference between these two histograms are used as inputs of Support Vector Machine (SVM) binary detectors associated with a new kernel: the Histogram Difference Intersection (HDI) kernel. Experimental results carried out for 16 Action Units (AUs) on the benchmark Cohn-Kanade database can be compared favorably with two state-of-the-art methods.