We consider a problem central in aerial visual surveillance applications { detection and tracking of small, independently moving objects in long and noisy video sequences. We directly use spatiotemporal image intensity gradient measurements to compute an exact model of background motion. This allows the creation of accurate mosaics over many frames and the de nition of a constraint violation function which acts as an indicator of independent motion. A novel temporal integration method maintains con dence measures over long subsequences without computing the optic ow, requiring object models, or using a Kalman lter. The mosaic acts as a stable feature frame, allowing precise localization of the independently moving objects. We present a statistical analysis of the e ects of image noise on the constraint violation measure and nd a good match between the predicted probability distribution function and the measured sample frequencies in a test sequence.