This paper describes a method for separating moving objects from temporally varying background in time-lapse confocal microscopy image sequences representing fluorescently tagged moving vesicles. A temporal linear model is considered for backgroundmodeling whose parameters are robustly estimated using asymmetric M-estimators combined with a bias-variance trade-off criterion. Furthermore, we propose an original approach for automatically detecting moving objects in the image sequence. Experimental results demonstrate the interest of this proposed method which can be relevant for biological studies from image sequences.