This paper presents a bottom-up tracking algorithm for surveillance applications where speed and reliability in the case of multiple matches and occlusions are major concerns. The algorithm is divided into four steps. First, moving objects are detected using an accurate hybrid scheme with selective Gaussian modeling. Simple object features balancing speed, reliability, and complexity are then extracted. Objects are matched based on their spatial proximity and feature similarity. Finally, correspondence voting solves multiple match conflicts, segmentation errors, and occlusion cases. This approach is very simple, which makes it suitable for implementation at smart surveillance visual sensing nodes. Moreover, the simulation results demonstrate its robustness in detecting occlusions and correcting segmentation errors without any prior knowledge about the objects models or constraints on the direction of their motion.