2 Background Model Background subtraction algorithm is susceptible to both global and local illumination changes such as shadows, sunlight and reflection. These changes sometimes cause failures of object tracking, gesture recognition and posture estimation. In this paper, we propose a method for detecting objects casting shadows in an indoor scene by modeling a pixel value as the total energy received from light sources. Our proposed method can also estimate a rate of illumination change in order to improve background subtraction. The usefulness of the method is demonstrated by experimentalresults.