We propose a method for extracting a gaze region from an observed image by analyzing human's view directions and image information. The view direction of the user, which is represented as a 2D gaze point in the observed image, is obtained by an eye-mark recorder at every imagecapturing timing. All gaze points are translated to one of the images for extracting the gaze region based on the history of the view directions. The system divides all gaze points into several groups by comparing color information etc., and then generates several convex hulls as initial regions. Each initial region is extended based on its color information and the spatial distribution of the gaze points. All regions are finally integrated and regarded as the gaze region.