Iteration Aware Prefetching for Remote Data Access

1 Although processing speed, storage capacity and network bandwidth are steadily increasing, network latency remains a bottleneck for scientists accessing large remote data sets. This problem is most acute with n-dimensional data. Grid researchers have only recently begun to develop tools for efficient remote access to n-dimensional data sets. Within the context of the Granite Scientific Database system, we show that latency penalties can be dramatically reduced using explicit knowledge of a user’s access pattern represented as an Iterator. The iterator not only performs an n-dimensional iteration for the user, but also communicates the access pattern to Granite so that a prefetching cache can be constructed that is tuned to the user’s access pattern. We experimentally evaluate a scenario for incorporating Granite’s prefetching mechanism into the Grid, demonstrating extraordinary performance gains. In light of these results, we describe planned additions to existing Grid servic...

• Type: Conference
• Year: 2005
• Where: ESCIENCE
• Authors: Philip J. Rhodes, Sridhar Ramakrishnan
• Keywords: Access Pattern, ESCIENCE 2005, N-dimensional Data, User’s Access Pattern