This paper presents a simple but robust visual tracking algorithm based on representing the appearances of objects using affine warps of learned linear subspaces of the image space. The tracker adaptively updates this subspace while tracking by finding a linear subspace that best approximates the observations made in the previous frames. Instead of the traditional L2 -reconstruction error norm which leads to subspace estimation using PCA or SVD, we argue that a variant of it, the uniform L2 -reconstruction error norm, is the right one for tracking. Under this framework, we provide a simple and a computationally inexpensive algorithm for finding a subspace whose uniform L2 -reconstruction error norm for a given collection of data samples is below some threshold, and a simple tracking algorithm is an immediate consequence. We show experimental results on a variety of image sequences of people and man-made objects moving under challenging imaging conditions, which include drastic illumin...