Distributed Computing with Load-Managed Active Storage

One approach to high-performance processing of massive data sets is to incorporate computation into storage systems. Previous work has shown that this active storage model is effective for a variety of problems. This paper explores opportunities to use active storage as a basis for exploiting asymmetric parallelism in applications using a streaming computation model on collections of fixed-size records. This model is the basis for much of the research in I/O-efficient algorithms, which deals with an important class of massive data problems not studied in previous work on active storage. We present an extension of a streaming computation model for an external memory toolkit to support a flexible mapping of computations to storage-based processors. Our approach enables load-managed active storage: it exposes parallelism, ordering constraints, and primitive computation units to the system, which can configure the application to balance load and make the best use of available processi...

• Type: Conference
• Year: 2002
• Where: HPDC
• Authors: Rajiv Wickremesinghe, Jeffrey S. Chase, Jeffrey Scott Vitter
• Keywords: Active Storage, Active Storage Model, Distributed And Parallel Computing, HPDC 2002, Load-managed Active Storage