The File Mover: high-performance data transfer for the grid

The exploration in many scientific disciplines (e.g., High-Energy Physics, Climate Modeling, and Life Sciences) involves the production and the analysis of massive data collections, whose archival, retrieval, and analysis require the coordinated usage of high capacity computing, network, and storage resources. To obtain satisfactory performance, these applications require the availability of a high-performance, reliable data transfer mechanisms, able to minimize the data transfer time that often dominates their execution time. In this paper we present the File Mover, an efficient data transfer system specifically tailored to the needs of data-intensive applications, that exploits the overlay networks paradigm to provide superior performance with respect to conventional file transfer systems. An extensive experimental evaluation, carried out by means of a proof-of-concept implementation of the File Mover for a variety of network scenarions, shows the ability of the File Mover to outper...

• Type: Journal
• Year: 2008
• Where: CONCURRENCY
• Authors: Cosimo Anglano, Massimo Canonico
• Keywords: CONCURRENCY 2008, Conventional File Transfer, File Mover, Massive Data Collections