Bandwidth Challenge
SC 2007

Tools used in SC07

FAST TCP - developed by Professor Steven Low of Caltech's computer science department is an alternative congestion control algorithm in TCP. It is designed for high speed data transfers over large distance, e.g., tens of gigabyte files across the Atlantic.

FDT - One of the key advances in this demonstration was Fast Data Transport (FDT; http://monalisa.cern.ch/FDT), a Java application developed by the Caltech team in close collaboration with the Polytehnica Bucharest team. FDT runs on all major platforms and uses the NIO libraries to achieve stable disk reads and writes coordinated with smooth data flow across long-range networks. The FDT application streams a large set of files across an open TCP socket, so that a large data set composed of thousands of files, as is typical in high-energy physics applications, can be sent or received at full speed, without the network transfer restarting between files. FDT works with Caltechs MonALISA system to dynamically monitor the capability of the storage systems as well as the network path in real-time, and sends data out to the network at a moderated rate that achieves smooth data flow across long range networks.

dCache - The goal of this project is to provide a system for storing and retrieving huge amounts of data, distributed among a large number of heterogenous server nodes, under a single virtual filesystem tree with a variety of standard access methods. Depending on the Persistency Model, dCache provides methods for exchanging data with backend (tertiary) Storage Systems as well as space management, pool attraction, dataset replication, hot spot determination and recovery from disk or node failures. Connected to a tertiary storage system, the cache simulates unlimited direct access storage space. http://www.dcache.org

SRM - Designed in Fermi National Lab Chicago - Storage Resource Managers (SRMs) are middleware components whose function is to provide dynamic space allocation and file management on shared storage components on the Grid. http://srm.fnal.gov