Optimization Rules for Programming with Collective Operations

We study how several collective operations like broadcast, reduction, scan, etc. can be composed efficiently in complex parallel programs. Our specific contributions are: (1) a formal framework for reasoning about collective operations; (2) a set of optimization rules which save communications by fusing several collective operations into one; (3) performance estimates, which guide the application of optimization rules depending on the machine characteristics; (4) a simple case study with machine experiments.

• Type: Conference
• Year: 1999
• Where: IPPS
• Authors: Sergei Gorlatch, Christoph Wedler, Christian Lengauer
• Keywords: Collective Operations, Complex Parallel Programs, Distributed And Parallel Computing, IPPS 1999, Optimization Rules