BI batch manager: a system for managing batch workloads on enterprise data-warehouses

Modern enterprise data warehouses have complex workloads that are notoriously difficult to manage. An important problem in workload management is to run these complex workloads `optimally'. Traditionally this problem has been studied in the OLTP (Online Transaction Processing) context where MPL (Multi Programming Level) is used as a knob to achieve optimality. However, MPL is a tricky knob in a BI (Business Intelligence) scenario, since a low MPL can easily result in underload and a high MPL can easily result in overload and `thrashing'. In this work we present BI Batch Manager, a workload management system to run batches of queries `optimally' on an Enterprise Data Warehouse (EDW). It is comprised of three components: an admission control component, a scheduler and an execution control component. In order to automatically avoid underload and overload, we introduce a novel execution control mechanism, PGM (Priority Gradient Multiprogramming). In PGM, a priority gradient...

• Type: Conference
• Year: 2008
• Where: EDBT
• Authors: Abhay Mehta, Chetan Gupta, Umeshwar Dayal
• Keywords: BI Batch Manager, BI Workloads, Complex Workloads, Database, EDBT 2008