GPU-SM: shared memory multi-GPU programming

Discrete GPUs in modern multi-GPU systems can transparently access each other’s memories through the PCIe interconnect. Future systems will improve this capability by including better GPU interconnects such as NVLink. However, remote memory access across GPUs has gone largely unnoticed among programmers, and multi-GPU systems are still programmed like distributed systems in which each GPU only accesses its own memory. This increases the complexity of the host code as programmers need to explicitly communicate data across GPU memories. In this paper we present GPU-SM, a set of guidelines to program multi-GPU systems like NUMA shared memory systems with minimal performance overheads. Using GPU-SM, data structures can be decomposed across several GPU memories and data that resides on a different GPU is accessed remotely through the PCI interconnect. The programmability benefits of the shared-memory model on GPUs are shown using a finite difference and an image filtering applicatio...

• Type: Journal
• Year: 2015
• Where: PPOPP
• Authors: Javier Cabezas, Marc Jordà, Isaac Gelado, Nacho Navarro, Wen-mei W. Hwu
• Keywords: Distributed And Parallel Computing, PPOPP 2015