This paper presents a general learning framework for non-rigid registration of MR brain images. Given a set of training MR brain images, three major types of information are particularly learned, and further incorporated into a HAMMER registration algorithm for improving the performance of registration. First, the best features are learned from different types of local image descriptors for each part of brain, thereby the learned best features are consistent on the correspondence points across individual brains, but different on non-correspondence points. Moreover, the statistics of selected best features is learned from the training samples, and used to guide the feature matching during the image registration. Second, in order to avoid the local minima in the registration, the points hierarchically selected to drive image registration are determined by the learned consistency and distinctiveness of their respective best features. Third, deformation fields are adaptively represented by...