Parallel Querying with Non-Dedicated Computers

We present DITN, a new method of parallel querying based on dynamic outsourcing of join processing tasks to non-dedicated, heterogeneous computers. In DITN, partitioning is not the means of parallelism. Data layout decisions are taken outside the scope of the DBMS, and handled within the storage software; query processors see a “Data In The Network” image. This allows gradual scaleout as the workload grows, by using non-dedicated computers. A typical operator in a parallel query plan is Exchange [7]. We argue that Exchange is unsuitable for non-dedicated machines because it poorly addresses node heterogeneity, and is vulnerable to failures or load spikes during query execution. DITN uses an alternate intra-fragment parallelism where each node executes an independent select-projectjoin-aggregate-group by block, with no tuple exchange between nodes. This method cleanly handles heterogeneous nodes, and well adapts during execution to node failures or load spikes. Initial experiments ...

• Type: Conference
• Year: 2005
• Where: VLDB
• Authors: Vijayshankar Raman, Wei Han, Inderpal Narang
• Keywords: Database, Gradual Scaleout, Load Spikes, Parallel Query Plan, VLDB 2005