If an organization depends on the service quality provided by another organization it often enters into a bilateral service level agreement (SLA), which mitigates outsourcing risks by associating penalty payments with poor service quality. Once these agreements are entered into, it becomes necessary to monitor their conditions, which will commonly relate to timeliness, reliability and request throughput, at runtime. We show how these conditions can be translated into timed automata. Acceptance of a timed word by a timed automaton can be decided in quadratic time and because the timed automata can operate while messages are exchanged at runtime there is effectively only a linear run-time overhead. We present an implementation to derive on-line monitors for web services automatically from SLAs using an Eclipse plugin. We evaluate the efficiency and scalability of this approach using a large-scale case study in a service-oriented computational grid. Categories and Subject Descriptors D.2...