Abstract. The mean-shift algorithm is an efficient technique for tracking 2D blobs through an image. Although it is important to adapt the mean-shift kernel to handle changes in illumination for robot vision at outdoor site, there is presently no clean mechanism for doing this. This paper presents a novel approach for color tracking that is robust to illumination changes for robot vision. We use two interleaved mean-shift procedures to track the spatial location and illumination intensity of a blob in an image. We demonstrate that our method enables efficient realtime tracking of the multiple color blobs against changes in illumination, where the illuminace ranges from 58 to 1,300 lx.