We describe initial work on a system for augmenting video sequences with 3-D graphics so that they appear to be present within the scene. Our aim is to do this in realtime for sequences captured by uncalibrated ‘live’ cameras, such as a hand-held or wearable. The paper focuses on obtaining 3-D camera motion and depth estimates for these types of sequences using sparse feature tracking and the recursive algorithm developed by Azarbayejani and Pentland [1]. We report experiments which demonstrate that the approach performs well and discuss implementation issues relating to its use in a ‘live’ real-time system.