Abstract. Artificial systems with a high degree of autonomy require reliable semantic information about the context they operate in. State interpretation, however, is a difficult task. Interpretations may depend on a history of states and there may be more than one valid interpretation. We propose a model for spatio-temporal situations using hidden Markov models based on relational state descriptions, which are extracted from the estimated state of an underlying dynamic system. Our model covers concurrent situations, scenarios with multiple agents, and situations of varying durations. To evaluate the practical usefulness of our model, we apply it to the concrete task of online traffic analysis.