In recent years the video event understanding is an active research topic, with many applications in surveillance, security, and multimedia search and mining. In this paper we focus on the human action recognition problem and propose a new Aligned Projection Distance (APD) approach based on the geometry modeling of video appearance manifold and the human action time series statistics on the geometry information. Experimental results on the KTH database demonstrate the solution to be effective and promising.