In this paper, we present an interactive offline tracking system for generic color objects. The system achieves 60100 fps on a 320 ? 240 video. The user can therefore easily refine the tracking result in an interactive way. To fully exploit user input and reduce user interaction, the tracking problem is addressed in a global optimization framework. The optimization is efficiently performed through three steps. First, from user's input we train a fast object detector that locates candidate objects in the video based on proposed features called boosted color bin. Second, we exploit the temporal coherence to generate multiple object trajectories based on a global best-first strategy. Last, an optimal object path is found by dynamic programming.