Abstract This paper proposes a real-time high-precision method to track an unknown face in front of an information system by selecting appropriate model to a video image. Active Appearance Model (AAM) can track a non-rigid object such as a face, because AAM learns the correlation between shape and texture. However, when AAM tracks an unknown face, excessive training data increases tracking errors because of truncating axes which shows characteristics of individuals and of additional local minima generated by the data. In order to increase accuracy of tracking unknown face, we build clusters from training data sets and select a model based on the cluster includes similar face with the unknown face. This paper shows a method of clustering and selecting the cluster based on Mutual Subspace Method, and the result of estimation by leave-one-out. Key words Active Appearance Model, Mutual Subspace Method, Face Tracking