Low-overhead message passing is critical to the performance of many applications. Active Messages[27] reduce the software overhead for message handling: messages are run as handlers instead of as threads, which avoids the overhead of thread management and the unnecessary data copying of other communication models. Scheduling the execution of Active Messages is typically done by disabling and enabling interrupts, or by polling the network. This primitive scheduling control, combined with the fact that handlers are not schedulable entities, puts severe restrictions on the code that can be run in a message handler. This paper describes a new software mechanism, Optimistic Active Messages(OAM), that eliminates these restrictions; OAMs allow arbitrary user code to execute in handlers, and also allow handlers to block. Despite this gain in expressiveness,OAMs perform as well as Active Messages. We used OAM as the base for an RPC system, Optimistic RPC (ORPC), for the Thinking Machines CM-5 ...
Deborah A. Wallach, Wilson C. Hsieh, Kirk L. Johns