AGU23
- Rachana Ananthakrishnan
- Ian Foster
- San Francisco, CA and Online
Attending AGU23? Join Monday’s talk on Leveraging Globus for data management and analysis in Earth Science
Title: Leveraging Globus for data management and analysis in Earth Science
Date: December 11, 2023 - 16:00 - 16:03
Location: eLightning Theater IV, Hall D - South
The Earth System Grid Federation (ESGF, esgf.io) is an international multi-institutional initiative that builds and operates a platform for management, distribution and analysis of climate model and observational data. The DOE funded ESGF2 project is upgrading the infrastructure services to better support the growing data volumes, reduce barriers for participation in the federation, broaden access to data and analysis, and increase the robustness and availability of the platform.
The project leverages Globus (globus.org) a data management and compute platform used by universities, research organizations, national laboratories, and government facilities worldwide. Operated by the University of Chicago, the Globus service enables secure, reliable file transfer, sharing, and data management automation throughout the research lifecycle. The Globus compute platform enables users to execute functions on diverse remote systems, from laptops to campus clusters, clouds, and supercomputers.
In this talk, we will provide an overview of the Globus platform capabilities, and present how it is leveraged by ESGF2 project to deliver a scalable and robust system. We will discuss design patterns that can be used to deliver high performant data distribution services, and automated data publication pipelines to serve scientific data.