GridFTP: Success Stories
"One Australian user left nearly 1TB of data on our systems that we had been struggling to transfer
via standard FTP for several weeks. The typical data rate using multiple sessions of scp was ~200 KB/s.
Using GridFTP we are now moving data at 6 MB/s, quite a significant boost in performance!"
- Brian Tieman and Francesco De Carlo, Advanced Photon Source, Argonne National Laboratory - Mar, 2008
"Using GridFTP, we moved about 1.5 TB of data from University of Wisconsin, Milwaukee to
Hannover, Germany at a sustained rate of 80 MB/s. We were quite pleased with 80 MB/s."
- Scott Koranda, LIGO Project - June, 2008
"Getting large datasets to our GPFS filesystem can be greatly speeded
up by using striped transfers with GridFTP. For example transferring a
14GB file from the UC/ANL TeraGrid site to the CI GPFS cluster using
scp took almost 28 minutes at an average 8.6MB/s. That same file
transferred via GridFTP took 6.5 minutes for an average of 37.3MB/s or
a 4.3x increase over scp. And lastly that same file transferred via a
striped GridFTP took just over 3 minutes for an average of 75.8MB/s or
a 2.0x increase over non-striped GridFTP or an 8.8x increase over scp."
- Ti Leggett,
Senior HPC systems administrator
Computation Institute, University of Chicago - July, 2008
"Using GridFTP, we currently get up to 200 MB/s on the TeraGrid network between Texas Advanced Computing Center (TACC) and San Diego Supercomputing Center (SDSC)."
- Yifeng Cui, San Diego Supercomputer Center - July, 2008
"ChinaGrid's data service uses GridFTP as its underly data transfer tool. Its
third-party data transfer capability enables us to use a centrally managed
service to control all the data transfer tasks, simplifying our data service's
design, configuration and management greatly. ChinaGrid applications such as
BioInformatics Grid, Computational Fluid Dynamics Grid, Massive Information
Processing Grid, Image processing Grid move huge volumes of data all over China
using GridFTP. GridFTP's parallel data transfer and other optimizations to
improve the speed enables the ChinaGrid applications to get better data transfer
rates."
- Yongwei Wu, Tsinghua University, China - Jan 2009
"The availability of several striped data nodes and parallel streams provided
by GridFTP allows us at LRZ to circumvent disk and network interface bottlenecks and to avoid long latencies between Grid computing centres. Moreover, the disk block size parameter of GridFTP has been useful to improve performance. We achieved over 300 MB/s data transfer speeds fairly easily inside Europe with the
TCP-based GridFTP protocol."
- Helmut Heller, Leibniz Supercomputing Center (LRZ), Germany - Feb 2009
"The Dark Energy Survey Data Management (DESDM) team has relied on GridFTP as
the underlying file transfer protocol for rapidly transferring large quantities
of raw and reduced data between a collection of heterogeneous sites. The project
has needed to transfer data between TeraGrid sites (NCSA and SDSC), between NCSA
and FermiLab (a DOE site), from DESDM team dedicated servers to shared
supercomputer platforms, from parallel file systems to tape archives. For all of
these use cases, GridFTP has provided the flexible and high-performance
solution required by the DESDM project."
- Greg Daues, National Center for Supercomputing Applications,
USA - Mar 2009
"GridFTP has been a pivotal part of our application development, providing
the fast, reliable and secure communication that underpins our applications for
the large-scale service infrastructures we have created. Its use has been
particularly significant in our work with the British Broadcasting Corporation
(BBC) in prototyping next generation on-demand media infrastructures. In these
applications GridFTP has been used to provide a media streaming service and has
been embedded in set-top boxes to provide fast on-demand download of video
content. GridFTP has proven more effective than the traditional proprietary
tools that are more often used in a broadcast infrastructure. For our BBC
applications, GridFTP has been used to move thousands of hours of video content
between sites and to users."
- Terence Harmer, Belfast e-Science Center, UK - Mar 2009
"High performance file transfer has always been critical to the success of
WestGrid. Right from the initial phase of WestGrid in 2003, some compute
resources were more than 500 miles away from our primary storage server; now
WestGrid spans larger distances. Due to this, appropriate tools are needed to
enable high perfromance files transfers across WestGrid's private network over
these distances. In addition, due to participation in world wide high energy
physics experiments, large amounts data needs to be transferred between sites
in US and Europe, and WestGrid systems. In all cases GridFTP has been the
primary tool used to enable high performance file transfers we rely on. While we
have explores other high performance file transfer tools over the past 5 years,
the GridFTP implemention in the Globus toolkit is the only one that we have
found t be fully robust and remains the primary file tranfer tool used within
WestGrid."
- Rob Simmonds, University of Calgary, Canada - Apr 2009
"The data rates available with SSH authenticated globus-url-copy are
roughly double (100MB/sec vs 50MB/sec) what we were achieving with
other high speed data movement tools at a fraction of the system resource
utilization. Having an SSH authenticated GridFTP has allowed us to quickly
deploy globus-url-copy using our existing infrastructure."
- Chandin Wilson, National Oceanic and Atmospheric Administration - Sep, 2009
"We now have 2 beamlines routinely automating data movement using GridFTP as
part of the acquisition workflow with plans for 2 more to begin working this way
in the next 1-2 months. The ability to script data movement 3rd party is really
working well for us. Not only are we moving data ~4 times faster than before,
the ability to pipe it during acquisition rather than burst it all at the end
has been a big boon!"
- Brian Tieman, Advanced Photon Source, Argonne National
Laboratory - Feb, 2010