Automatic and real-time identification of unusual incidents is important for event detection and alarm systems. In today's camera surveillance solutions video streams are displayed on-screen for human operators, e.g. in large multiscreen control centers. This in turn requires the attention of operators for unusual events and urgent response. This paper presents a method for the automatic identification of unusual visual content in video streams real-time. In contrast to explicitly modeling specific unusual events, the proposed approach incrementally learns the usual appearances from the visual source and simultaneously identifies potential unusual image regions in the scene. Experiments demonstrate the general applicability on a variety of large-scale datasets including different scenes from public web cams and from traffic monitoring. To further demonstrate the real-time capabilities of the unusual scene detection we actively control a Pan-Tilt-Zoom camera to get close up views ...