Internally deterministic parallel algorithms can be fast

The virtues of deterministic parallelism have been argued for decades and many forms of deterministic parallelism have been described and analyzed. Here we are concerned with one of the strongest forms, requiring that for any input there is a unique dependence graph representing a trace of the computation annotated with every operation and value. This has been referred to as internal determinism, and implies a sequential semantics—i.e., considering any sequential traversal of the dependence graph is sufficient for analyzing the correctness of the code. In addition to returning deterministic results, internal determinism has many advantages including ease of reasoning about the code, ease of verifying correctness, ease of debugging, ease of defining invariants, ease of defining good coverage for testing, and ease of formally, informally and experimentally reasoning about performance. On the other hand one needs to consider the possible downsides of determinism, which might include...

• Type: Journal
• Year: 2012
• Where: PPOPP
• Authors: Guy E. Blelloch, Jeremy T. Fineman, Phillip B. Gibbons, Julian Shun
• Keywords: Dependence Graph, Distributed And Parallel Computing, PPOPP 2012, Programming Paradigms, Sequential Semantics