We present a fast line segment tracker which does not require any knowledge about the motion of the camera nor the structure of the observed scene. It runs on 320 x 240 pixel images at 30 Hz. We adapted the RAPiD tracker with a new way of handling multiple line hypotheses to deal with the simple model of a single line segment. We discuss the difficulty of using a χ2-test as merging criterion and also present a new approach to overcome it. Furthermore, instead of making assumptions about the camera motion, a constant velocity motion model to predict the line segment position in the following frame is used. We explain how to deal with the instability of the endpoint extraction in this motion model to avoid unintentional motion along the line. Finally, we present results on real world indoor and urban outdoor image sequences.