We propose a new method for performing accurate background subtraction in scenes with a door, like a building entrance or a hallway. This kind of scene is common in surveillance applications, yet the sporadic motion of a door causes problems for existing systems that falsely report the door as foreground. Our method models the scene’s appearance by storing a set of gaussian pixel distributions corresponding to a discrete sample of the door’s range of motion. All of the pixels in the image are dependent on the position of the door, so we use the joint probability for all of them to estimate the maximum-likelihood position of the door. We then perform background subtraction using the specific appearance model indexed by our estimated position. We show that our algorithm accurately segments the foreground region in several actual indoor and outdoor surveillance settings.