Flexible and scalable storage management for data-intensive stream processing

Data Stream Management Systems (DSMS) operate under strict performance requirements. Key to meeting such requirements is to efficiently handle time-critical tasks such as managing internal states of continuous query operators, traffic on the queues between operators, as well as providing storage support for shared computation and archived data. In this paper, we introduce a general purpose storage management framework for DSMSs that performs these tasks based on a clean, loosely-coupled, and flexible system design that also facilitates performance optimization. An important contribution of the framework is that, in analogy to buffer management techniques in relational database systems, it uses information about the access patterns of streaming applications to tune and customize the performance of the storage manager. In the paper, we first analyze typical application requirements at different granularities in order to identify important tunable parameters and their corresponding value...

• Type: Journal
• Year: 2009
• Where: EDBT
• Authors: Irina Botan, Gustavo Alonso, Peter M. Fischer, Donald Kossmann, Nesime Tatbul
• Keywords: Database, EDBT 2009, Performance, Storage Management, Storage Manager