Along with the ever-growing Web comes the proliferation of objectionable content, such as pornography, violence, horror information, etc. Horror videos, whose threat to childrens health is no less than pornographic video, are sometimes neglected by existing Web filtering tools. Consequently, an effective horror video filtering tool is necessary for preventing children from accessing these harmful horror videos. In this paper, by introducing color emotion and color harmony theories, we propose a horror video scenes recognition algorithm. Firstly, the video scenes are decomposed into a set of shots. Then we extract the visual features, audio features and emotional features of each shot, the video scene is viewed as a bag and each shot is treated as an instance of the corresponding bag. Finally, by combining the three features, the horror video scenes are recognized by the Multiple-Instance learning(MIL). According to the experimental results on diverse video scenes, the proposed schem...