Multi-frame motion estimation is a novel technique to improve the video coding efficiency. However, the computational complexity in motion search is usually a bottleneck in the video coding system. According to the experiences of conventional techniques, good prediction in selecting an initial search point can further accelerate the convergence speed of fast motion search algorithms. In this paper, two motion vector prediction techniques are developed for the speed-up of multi-frame motion estimation. The proposed method can efficiently increase the prediction accuracy and reduce the search point up to 25% with the similar quality and bit-rate under the New Diamond Search framework in H.264.