This paper proposes a data pruning-based compression scheme to improve the rate-distortion relation of compressed images and video sequences. The original frames are pruned to a smaller size before compression. After decoding, they are interpolated to their original size by an edge-directed interpolation. The data pruning is optimized to obtain the minimal distortion in the interpolation phase. Furthermore, a novel high order interpolation is proposed to adapt the interpolation to many edge directions. This high order ltering uses extra surrounding pixels and achieves more robust edge-directed image interpolation. Simulation results are shown for both image interpolation and coding applications.