We present an emotion recognition system based on a probabilistic approach to adaptive processing of Facial Emotion Tree Structures (FETS). FETS are made up of localized Gabor features related to the facial components according to the Facial Action Coding System. The proposed model is an extension of the probabilistic based recursive neural network model applying in face recognition by Cho and Wong [1]. The robustness of the model in an emotion recognition system is evaluated by testing with known and unknown subjects with different emotions. The experiment results shows that the proposed model significantly improved the recognition rate in terms of generalization. Categories and Subject Descriptors I.2.10 [Artificial Intelligence]: Vision and Scene Understanding, Learning. I.5.4 [Pattern Recognition]: Computer vision. E.1 [Data Structures]: Trees. J.4 [Social and Behavioral Sciences]: Psychology. General Terms Algorithms, Measurement, Human Factors, Verification. Keywords facial emot...