This paper investigates the combination of spatial and probabilistic models for reasoning about pedestrian behaviour in visual surveillance systems. Models are learnt by a multi-step unsupervised method and they are used for trajectory labelling and atypical behaviour detection.