A three-level method is proposed to achieve robust and real-time ball tracking in soccer videos. It includes object-, intra-trajectory-, and intertrajectory-level processing. Due to much noise and frequent occlusion, it’s difficult to get the solely ball in one frame. Thus, in object level, multiple objects instead of a single one are detected and taken as ball candidates with shape and color features. Then at intra-trajectory level, each ball candidate is tracked by a Kalman filter in successive frames, which results in lots of initial trajectories in a video segment. These trajectories are thereafter scored and filtered according to their length and relationship in a time-line model. With these trajectories, we construct a distance graph, in which a node represents a trajectory, and an edge means distance between two trajectories. We use the Dijkstra algorithm to get the optimal path in the graph at the inter-trajectory level. To smooth the trajectory, we finally apply cubic spline...