Abstract. In this paper, we present an automatic method for estimating the trajectories of Escherichia coli bacteria from in vivo phase-contrast microscopy videos. To address the low-contrast boundaries in cellular images, an adaptive kernel-based technique is applied to detect cells in sequence of frames. Then a novel matching gain measure is introduced to cope with the challenges such as dramatic changes of cells' appearance and serious overlapping and occlusion. For multiple cell tracking, an optimal matching strategy is proposed to improve the handling of cell collision and broken trajectories. The results of successful tracking of Escherichia coli from various phase-contrast sequences are reported and compared with manually-determined trajectories, as well as those obtained from existing tracking methods. The stability of the algorithm with different parameter values is also analyzed and discussed.