Probabilistic Failure Detection for Efficient Distributed Storage Maintenance

Distributed storage systems often use data replication to mask failures and guarantee high data availability. Node failures can be transient or permanent. While the system must generate new replicas to replace replica lost to permanent failures, it can save significant replication costs by not replicating following transient faults. Given the unpredictability of network dynamics, however, distinguishing permanent and transient failures is extremely difficult. Traditional timeout approaches are difficult to tune and can introduce unnecessary replication.1 In this paper, we propose Protector, an algorithm that addresses this problem using network-wide statistical prediction. Our algorithm drastically improves prediction accuracy by making predictions across aggregate replica groups instead of single nodes. These estimates of the number of "live replicas" can guide efficient data replication policies. We prove that given data on node down times and the probability of permanent ...

• Type: Conference
• Year: 2008
• Where: SRDS
• Authors: Jing Tian, Zhi Yang, Wei Chen, Ben Y. Zhao, Yafei Dai
• Keywords: Data Replication, Distributed And Parallel Computing, Node Failures, Permanent Failures, SRDS 2008