We present a novel human action recognition system based on segmented skeletal features which are separated into several human body parts such as face, torso and limbs. Our proposed human action recognition system consists of two steps: (i) automatic skeletal feature extraction and splitting by measuring the similarity in the space of diffusion tensor fields, and (ii) multiple kernel Support Vector Machine based human action recognition. Experimental results on a set of test database show that our proposed method is very efficient and effective to recognize human actions using few parameters, independent of dimensions, shadows, and viewpoints.