This paper studies the dynamic Web service selection problem in a failure-prone environment, which aims to determine a subset of Web services to be invoked at runtime so as to successfully orchestrate a composite Web service. We observe that both the composite and constituent Web services often constrain the sequences of invoking their operations and, therefore, we propose using a finite state machine to model the permitted invocation sequences of Web service operations. We assign each state of execution an aggregated reliability to measure the probability that the given state will lead to successful execution in the context where each Web service may fail with some probability. We show that the computation of aggregated reliabilities is equivalent to eigenvector computation and adopt the power method to efficiently derive aggregated reliabilities. In orchestrating a composite Web service, we propose two strategies to select Web services that are likely to successfully complete the exe...