Although Web services are intended for short term, ad hoc collaborations, in practice many Web service compositions are offered longterm to customers. While the Web services making up the composition may vary, the structure of the composition is rather fixed. For companies managing such Web service compositions, however, challenges arise which go far beyond simple bilateral contract monitoring. It is not only important to determine whether or not a component (i.e., Web service) in a composition is performing properly, but also to understand what the impact of its performance is on the overall service composition. In this paper we show which challenges emerge and we provide an approach on determining the impact each Web service has on the composition at runtime.