A strategy for cost efficient distributed data storage for in-memory OLAP

With the availability of inexpensive blade servers featuring 32 GB or more of main memory, memory-based engines such as the SAP NetWeaver Business Warehouse Accelerator are coming into widespread use for online analytic processing (OLAP) of terabyte data volumes. Data storage for such engines is often implemented in standard storage technologies like storage area network (SAN) or network attached storage (NAS) with high hardware costs. Given the access pattern, storage costs can be reduced by using a distributed persistence layer based on commodity architecture. We discuss an example of an in-memory OLAP engine with a focus on storage architecture. We then present an implementation of a distributed persistence layer that is optimized for the access pattern of such engines. Finally, we show the cost-saving potential and discuss the performance impact compared to SAN systems. KEYWORDS Storage issues, Distributed storage, Cost efficient storage, OLAP

• Type: Journal
• Year: 2009
• Where: IADIS
• Authors: Olga Mordvinova, Oleksandr Shepil, Thomas Ludwig 0002, Andrew Ross
• Keywords: Access Pattern, Distributed Persistence Layer, Engines, IADIS 2009, Internet Technology