This paper proposes a general feature selection approach for real-time image matching systems. To demonstrate the idea's effectiveness, we focus on the issue of rotational invariance. Most current image matching methods compute and align local image patches to a uniform dominant orientation, which are either too computationally expensive for real-time systems or insufficiently robust. In contrast to current approaches, we combine multiple-view training and feature selection into a unified framework. The most invariant features are selected during an offline training stage. Therefore, no additional computation is needed for online processing. Furthermore the proposed Rotation Invariant Feature Selection (RIFS) can be easily adapted to similar image matching problems such as scale invariance improvement and kernel selection in feature description. Experimental results show the effectiveness of RIFS using only a small number of training views. The proposed approach is also successfu...