We present a tracking approach for textured surfaces which recovers the object motion in 6 degrees of freedom. We assume an arbitrary but known surface shape, and an image of the object at a known reference pose. We extend the 2-D tracking framework of Hager et al. [1] to tracking in 3-D and under full perspective projection. The algorithm is evaluated to ground-truth motion and shows high accuracy. Thanks to problem-specific optimizations we achive tracking at video-rate.