High-Availability Algorithms for Distributed Stream Processing

Stream-processing systems are designed to support an emerging class of applications that require sophisticated and timely processing of high-volume data streams, often originating in distributed environments. Unlike traditional dataprocessing applications that require precise recovery for correctness, many stream-processing applications can tolerate and benefit from weaker recovery guarantees. In this paper, we study various recovery guarantees and pertinent recovery techniques that can meet the correctness and performance requirements of stream-processing applications. We discuss the design and algorithmic challenges associated with the proposed recovery techniques and describe how each can provide different guarantees with proper combinations of redundant processing, checkpointing, and remote logging. Using analysis and simulations, we quantify the cost of our recovery guarantees and examine the performance and applicability of the recovery techniques. We also analyze how the knowle...

• Type: Conference
• Year: 2005
• Where: ICDE
• Authors: Alex Rasin, Jeong-Hyon Hwang, Magdalena Balazinska, Michael Stonebraker, Stanley B. Zdonik, Ugur Çetintemel
• Keywords: Database, ICDE 2005, Pertinent Recovery Techniques, Various Recovery Guarantees, Weaker Recovery Guarantees