A motion detection and tracking algorithm for human and car activity surveillance is presented and evaluated by using the Pets'2000 test sequence. Proposed approach uses a temporal fusion strategy by using the history of events in order to improve instantaneous decisions. Normalized indicators updated at each frame summarize history of specific events. For the motion detection stage a fast updating algorithm of the background reference is proposed. The control of the updating at each pixel is based on a stability indicator estimated from inter-frame variations. The tracking algorithm uses a determinist region based approach. A belief indicator representing the tracking consistency for each object allows to solve defined ambiguities at the tracking level. A second specific tracking indicator representing the identity quality of each tracked object is updated by integrating objects interaction. Tracking indicators permit to propagate uncertainties on higher levels of the interpreta...