In this paper, a novel multi-stage motion vector processing algorithm at the decoder is proposed for motion compensated frame interpolation. We address the problems of discontinuous edges and deformed structures in an interpolated frame by explicitly considering reliability of each received motion vector. By hierarchically refining motion vectors with different block sizes, the proposed method is capable of preserving structure information. Experimental results show that the proposed scheme outperforms other methods in terms of visual quality and PSNR, and it is also robust when video sequences have complex scenes and fast motion.