Managing Statistical Behavior of Large Data Sets in Shared-Nothing Architectures

—Increasingly larger data sets are being stored in networked architectures. Many of the available data structures are not easily amenable to parallel realizations. Hashing schemes show promise in that respect for the simple reason that the underlying data structure can be decomposed and spread among the set of cooperating nodes with minimal communication and maintenance requirements. In all cases, storage utilization and load balancing are issues that need to be addressed. One can identify two basic approaches to tackle the problem. One way is to address it as part of the design of the data structure that is used to store and retrieve the data. The other is to maintain the data structure intact but address the problem separately. The method that we present here falls in the latter category and is applicable whenever a hash table is the preferred data structure. Intrinsically attached to the used hash table is a hashing function that allows one to partition a possibly unbounded set of...

• Type: Journal
• Year: 1998
• Where: TPDS
• Authors: Isidore Rigoutsos, Alex Delis
• Keywords: Data Structure, Hashing Function, Larger Data Sets, TPDS 1998