The petabytes produced and consumed by exascale computers must often be moved among elements of the international scientific infrastructure. Scientists have long relied upon Globus for efficient, reliable, and secure data movement, and indeed as of 10/18/2021, Globus has been used to move more than 1018 bytes (1,343,546,072,100,387,000,000 bytes to be precise) among more than 20,000 labs worldwide.
On March 7th of 2016, a couple of members of the Globus team started working on replacing the globusonline-transfer-api-client package. We debated what to call it. Was it the globus-client? Just globus-python? "What about SDK -- Standard Development Kit?" "Yeah, SDK is good."
The National Institutes of Health (NIH) provide leadership and direction for research programs aimed at enhancing health, lengthening life, and reducing illness and disability. Data is an important ingredient for—and a product of—these projects. Historically, each program has managed the data it produces independently. Among the NIH institutes, a program called the Common Fund sponsors a portfolio of ambitious initiatives, each of which generates biomedical data of high value to the research community.
During the month of July Globus transferred over 800 Terabytes of data on Comet, a dedicated NSF funded eXtreme Science and Engineering Discovery Environment (XSEDE) cluster at the San Diego Supercomputer Center (SDSC). While Comet is now officially retired, users will still have access to the Comet filesystems via Globus to facilitate data migration.
Over the past few years Globus Labs has participated in two student summer lab programs at the University of Chicago - the Center for Data and Computing (CDAC) Summer Lab Program, and the BigDataX Research Experiences for Undergraduates (REU) Program, which is funded by the NSF. This year the two programs include 65 students, and five of the students are working with Globus Labs.
Data volumes are exploding, and the need to efficiently store and share data quickly, reliably and securely—along with making the data discoverable—is increasingly important. Deliberate planning and execution are necessary to properly collect, curate, manage, protect and disseminate the data that are the lifeblood of the modern research enterprise.
In order to leverage the ever-increasing vast amounts of data, breaking down research silos and integrating research services becomes essential. Key to accomplishing this goal is to simplify data access by using existing identities. Globus Auth - a platform service offered by Globus - enables resources like NCAR, XSEDE, and Petrel to do just that. In the five years since its launch Globus Auth has grown to over 231,000 unique users, issued over one billion access tokens, and now issues over one million new tokens daily.