— A method to detect moving objects on non-stationary background is proposed. The concurrent motions of foreground and background pixels make it extremely difficult to maintain a plausible background model for background subtraction. In our method, motion fields of aligned neighboring frames are fused to reduce parallax effects in moving blob detection. A fused color background model is further developed to refine shapes of detected objects. Finally, moving blob information is incorporated into the adaptation process of background model. Only confidently marked background pixels are adapted into background models with each incoming frame. Experimental results shown robust, well-shaped moving object detection can be obtained under unconstrained scenes. Keywords-object detection; optical flow; background modeling