Face recognition using image-set or video sequence as input tends to be more robust since image-set or video sequence provides much more information than single snapshot about the variation in the appearance of the target subject. Usually the distribution of such image-set approximately resides in a low dimensional linear subspace and the distance between image-set pairs can be defined based on the concept of principal angles between the corresponding subspace bases. Inspired by the work of[4, 14], this paper presents a robust framework for image-set based face recognition using boosted global and local principal angles. The original multi-class classification problem is firstly transformed into a binary classification task where the positive class is the principal angle based intra-class subspace “difference” and the negative one is the principal angle based inter-class subspace “difference”. The principal angles are computed not only globally for the whole pattern space b...