In this work, we propose a novel saliency-based objective quality assessment metric, for assessing the perceptual quality of decoded video sequences affected by packet loses. The proposed method weights the error at each pixel by the visual saliency of the pixel. Different weighting methods are explored and compared. Our test results show that the predicted scores by the proposed metrics correlate very well with mean subjective scores, significantly better than the mean square error (MSE), mean absolute difference (MAD) or structure similarity (SSIM).