Particle filter (PF) based object tracking methods have been widely used in computer vision. However, traditional particle filter trackers cannot effectively distinguish the target from the background in complex scenes since they only exploit appearance information of observation to determine the target region. In this paper, we present an adaptive particle filter based on energy field (EPF), which makes good use of moving information of previous frames adaptively to track the target. Besides, we present the mechanism of result rectification to ensure the target region is accurate. Experiment results on several challenging video sequences have verified that the adaptive EPF method is compared very robust and effective with the traditional particle filter in many complicated scenes.