—To rapidly and accurately search the corresponding points along scan-lines, rectification of stereo pairs are performed so that corresponding epipolar lines are parallel to the horizontal scan-lines and the difference in vertical direction is zero. In this paper, the method to rectify image pairs could be divided into three steps including projective transformation, affine transformation and shearing transformation. The projective transformation matrix is computed under the affine epipolar geometry constraint, and the values of unknown parameters are searched by an algorithm which does not require the relative matrix be positive definite. In this paper, an optimization function is presented to remove the difference in vertical direction and an algorithm is developed to estimate initial values of some parameters such as scale weights and vertical offset.