This paper presents our work on automatically detecting moving rigid text in digital videos. The temporal information is obtained by dividing a video frame into sub-blocks and calculating inter-frame motion vector for each sub-block. Text blocks are then extracted through both intra-frame classification and inter-frame spatial relationship checking. Unlike previous works, our method achieves both detection and tracking of moving text at the same time. The method works very well detecting scrolling text in news clips and movies, and is robust towards low resolution and complex background. The computational efficiency of the method is also discussed.