Colorizing grayscale video is a useful technique in scientific computing visualization and entertainment. In this paper, we introduce a method to transfer color from a reference image to the whole video. Although the general problem of adding chromatic values to a grayscale image has no exact, objective solution, the current approach attempts to free user from laborious work, except that the user may contribute their skill to reference image. Rather than choosing RGB colors from a palette to color individual components, we resort to anther color space, called l, which minimizes correlation between channels for many natural scenes. Taking advantage of the correlation between two conjoint frames of video, we track the object and assign the color of it in the preceding frame to that of the posterior one. Experimental results show the algorithm works quite well. Keywords color transfer, image processing, color space, multimedia