Predicting Multiple Metrics for Queries: Better Decisions Enabled by Machine Learning

One of the most challenging aspects of managing a very large data warehouse is identifying how queries will behave before they start executing. Yet knowing their performance characteristics -- their runtimes and resource usage -- can solve two important problems. First, every database vendor struggles with managing unexpectedly long-running queries. When these long-running queries can be identified before they start, they can be rejected or scheduled when they will not cause extreme resource contention for the other queries in the system. Second, deciding whether a system can complete a given workload in a given time period (or a bigger system is necessary) depends on knowing the resource requirements of the queries in that workload. We have developed a system that uses machine learning to accurately predict the performance metrics of database queries whose execution times range from milliseconds to hours. For training and testing our system, we used both real customer queries and quer...

• Type: Conference
• Year: 2009
• Where: ICDE
• Authors: Archana Ganapathi, Harumi A. Kuno, Umeshwar Dayal, Janet L. Wiener, Armando Fox, Michael I. Jordan, David A. Patterson
• Keywords: Database, Database Queries, Extensions Mimic Queries, ICDE 2009, Long-running Queries, Real Customer Queries, Test Queries