PMJoin: Optimizing Distributed Multi-way Stream Joins by Stream Partitioning

Abstract. In emerging data stream applications, data sources are typically distributed. Evaluating multi-join queries over streams from different sources may incur large communication cost. As queries run continuously, the precious bandwidths would be aggressively consumed without careful optimization of operator ordering and placement. In this paper, we focus on the optimization of continuous multi-join queries over distributed streams. We observe that by partitioning streams into substreams we can significantly reduce the communication cost and hence propose a novel partitionbased join scheme - PMJoin. A few partitioning techniques are studied. To generate the query plan for each substream, a heuristic algorithm is proposed based on a rate-based model. Results from an extensive experimental study show that our techniques can sufficiently reduce the communication cost.

• Type: Conference
• Year: 2006
• Where: DASFAA
• Authors: Yongluan Zhou, Ying Yan, Feng Yu, Aoying Zhou
• Keywords: Communication Cost, DASFAA 2006, Database, Large Communication Cost, Multi-join Queries