In this paper we present a method to estimate in real-time the position and orientation of a previously viewed planar patch. The algorithm is based on minimising the sum of squared differences between a selected set of pixels obtained from a previously stored image of the patch and the current image of it. First a linear model for projectively tracking a planar patch is introduced, then, a procedure to accelerate the tracking process is presented. This procedure is based on using a small set of informative points in the tracking process. In the experiments conducted we show the gain in performance and compare different procedures to select the set of points used in tracking.