Globus platform services enable developers to create applications that leverage the Globus identity, profile, file transfer, and data sharing capabilities. Services currently available to developers include Globus Auth and Globus Transfer, as well as a number of web helper pages to accelerate application development.
Research applications often require sophisticated data management capabilities, for example to move large amounts of data between acquisition, storage, analysis and archival. However, doing so efficiently and reliably is often a difficult undertaking, requiring significant expertise to maximize available bandwidth, restart transfers that fail (due to network unreliability), validate that files have been transferred correctly, and securely transfer data between resources with different security configurations. Globus Transfer addresses these challenges by providing an easy to use "fire-and forget" data transfer mechanism that can be integrated by developers into their own applications.
Globus Transfer provides high performance, secure, third-party data movement and synchronization between endpoints, as well as in-place sharing of files directly on existing storage systems. Globus Transfer handles all the difficult aspects of data transfer, allowing application users to easily start and manage transfers between endpoints, while automatically tuning parameters to maximize bandwidth usage, managing security configurations, providing automatic fault recovery, and notifying users of completion and problems. With in-place sharing of data, researchers can share large datasets directly from their existing storage repositories on which the data resides.
Developers can leverage Globus Transfer to provide a high-performance and reliable upload or download mechanism for large (or small) datasets. This allows application users to asynchronously start uploads or downloads and rely on Globus Transfer to ensure the data is moved efficiently and reliably. By integrating Globus Transfer into an application, developers gain access to over 10,000 Globus endpoints.
Explore the Globus Platform
The Globus Search service enables storage of metadata with fine grained access control on creation, modification, and visibility, and metadata retrieval through search queries. Globus Search is a set of capabilities that can be incorporated into your data management applications, either in conjunction with, or independently of, other Globus features such as file transfer and data sharing. The starting point is to provide Globus Search with metadata. Unlike many other services, there is no prescribed "metadata schema"; Globus Search will ingest metadata of arbitrary structure and make it available for faceted search. The primary element of the service is the index, which is the vehicle for storing metadata, setting permissions, and performing searches.
Globus Search metadata are organized using "subjects" and "entries". Entries allow association of multiple distinct pieces of metadata with a single subject, making it possible to have, for example, multiple sources contributing metadata independently of each other or different visibility policies on different entries (such as may be required in an environment that includes both open and protected data). Queries in Globus Search utilize ElasticSearch and allow for both basic text matching (e.g. a search field in a web application) and more advanced syntax that incorporates ranges, regular expressions, matching on particular fields, and other more sophisticated capabilities (e.g. to facilitate more targeted discovery in data portals and science gateways).
Globus Auth is a foundational identity and access management platform service designed to address unique needs of the science and engineering community. It serves to broker authentication and authorization interactions between end-users, identity providers, resource servers (services), and clients (including web, mobile, desktop, and command line applications, and other services).
Globus Auth thus makes it easy, for example, for a researcher to authenticate with one credential, connect to a specific remote storage resource with another identity, and share data with colleagues based on another identity. By eliminating friction associated with the frequent need for multiple accounts, identities, credentials, and groups when using distributed cyberinfrastructure, Globus Auth streamlines the creation, integration, and use of advanced research applications and services. Globus Auth builds upon the OAuth 2 and OpenID Connect specifications to enable standards-compliant integration using existing client libraries. It supports identity federation models that enable diverse identities to be linked together, while also providing delegated access tokens via which client services can obtain short term delegated tokens to access other services.
Many research communities have adopted Globus Auth to facilitate access to their resources and services, including the JetStream cloud, XSEDE, NCAR's Research Data Archive, and FaceBase.
The Globus identifier service enables users to quickly and easily associate a persistent identifier (e.g., DOI or ARK) with arbitrary objects.
When working with research data it is important to associate an unambiguous name with an object rather than rely on potentially volatile references to mutable storage locations. By enabling such naming, data can be uniquely referenced and located, and they can then be easily shared. The Globus Identifier service addresses this need by enabling anyone to create and associate a unique persistent identifier with an object.
The Globus Identifier service allows users to manage identifiers within a namespace. Namespaces abstract use of an external persistent identifier (PID) provider and a valid account (or shoulder) within that provider. Identifiers minted within a namespace use the external PID provider to create a persistent identifier. Namespaces may also define policies regarding their use, such as which users (or groups of users) are able to create identifiers and what visibility policies are imposed on identifiers created in the namespace.
The Globus Identifier service provides a simple REST API for creating and managing namespaces and identifiers. It also implements landing pages for all identifiers in both human (HTML) and machine-readable (JSON) forms.
The Globus identifier service is currently in limited production. If you are interested in helping us pilot the service please contact email@example.com.
Web Helper Pages
The Globus platform provides several web helper pages to enable use of common Globus functions within your web applications. Web helper pages simplify integration by allowing you to embed endpoint browsing, group selection, and login/logout pages into the application workflow with minimal code. After the user performs their action in Globus, the flow will redirect the user back to the application. For example, the endpoint selection page allows users to preselect a source endpoint and path. The user is then presented only with a destination selection interface to transfer a datasets. This approach provides a simple way for developers to use advanced Globus Transfer functionality without having to develop their own user interfaces.