Regeneration of biometric templates from match scores has security and privacy implications related to any biometric based authentication system. In this paper, we propose a novel non-iterative scheme to reconstruct face templates from match scores. We use an affine transformation of the images to approximate the behavior of the given face recognition system based on an independent set of face templates termed as "break-in" set. Selected templates from the "break-in" set are matched only once with the enrolled template of the target account and match scores are recorded. These scores are then embedded in the approximating affine space along with break-in set templates to compute the co-ordinates of the target template. The inverse transformation is used to reconstruct the original target template. We present the reconstruction of templates for three different face recognition algorithms: Bayesian intrapersonal/extrapersonal classifier, Elastic Bunch Graph Matching ...