This paper proposes an algorithm for rejecting false matches (known as outliers) in image pairs acquired with automobile-mounted cameras. Many intelligent vehicle applications require point correspondences for motion estimation and 3D reconstruction which can be affected by outliers. We use the property of automobile motion to reject outliers. Automobile motion mostly represented by two translations and one rotation. The proposed algorithm eliminates the rotational effect and estimates the focus of expansion (FOE). Once the FOE is estimated, the joining line directions with respect to the FOE are used for rejecting the outliers. This algorithm is simple and its computational cost is independent of the outlier percentage. Experimental results show that the proposed algorithm rejects a large number of outliers but retains most inliers when working with synthetic and real image pairs. It works even when the initial matches are contaminated by 80