A latency and fault-tolerance optimizer for online parallel query plans

We address the problem of making online, parallel query plans fault-tolerant: i.e., provide intra-query fault-tolerance without blocking. We develop an approach that not only achieves this goal but does so through the use of different fault-tolerance techniques at different operators within a query plan. Enabling each operator to use a different faulttolerance strategy leads to a space of fault-tolerance plans amenable to cost-based optimization. We develop FTOpt, a cost-based fault-tolerance optimizer that automatically selects the best strategy for each operator in a query plan in a manner that minimizes the expected processing time with failures for the entire query. We implement our approach in a prototype parallel query-processing engine. Our experiments demonstrate that (1) there is no single best fault-tolerance strategy for all query plans, (2) often hybrid strategies that mix-and-match recovery techniques outperform any uniform strategy, and (3) our optimizer correctly ide...

• Type: Journal
• Year: 2011
• Where: SIGMOD
• Authors: Prasang Upadhyaya, YongChul Kwon, Magdalena Balazinska
• Keywords: Database, Database Management Systems, Fault Tolerance Techniques, Hybrid Strategies, SIGMOD 2011