This paper proposes an adaptive system for video news story tracking based on the Earth Mover’s Distance (EMD). When an interesting story appears in the news, it is flagged manually as a topic for tracking. Our system then tracks the events as they unfold over time and present accumulated results to the user for feedback. This feedback is used to adapt the topic model to changes in the tracked story. EMD provides the system with a robust way of performing many-to-many matching of news stories independent of the temporal order of their contents. This is particularly suitable in the news genre as stories often are subjected to video editing between shows. Experiments have been run with a range of topics and show promising results.