In the Transfer Web Interface (Web GUI), when you first access an endpoint you are prompted for your MyProxy username and passphrase. Once you provide this information, you can browse your files and perform transfers to and from the endpoint. This blog post is about what happens behind the scenes to make that happen.
In order for Globus Online to perform endpoint operations like directory listing and file transfer on a user's behalf, our service needs a copy of the user's credentials. In the academic grid, credential management is typically done using MyProxy servers. The MyProxy server, usually run by the same organization as the GridFTP server, accepts the user's username and passphrase as input, and returns a time-limited X.509 credential representing that user. Globus Online stores only this time-limited credential. This process of giving access to a user's credential is called endpoint activation, because the credential is associated with a specific endpoint.
When you activate an endpoint with the Web GUI, it's actually calling the RESTful Transfer API behind the scenes. The Transfer API exposes activation as a two step process:
- Client requests a list of what methods of activation are supported and what information is required.
- Client collects any required information from the user, and sends the filled in values back to the API.
This mechanism allows us to support other activation methods in the future without changing the API. It also allows the client to dynamically create a dialog box, without hard coding the requirements.
In the sense of required user interaction, endpoint activation is an expensive operation. Prompting the user for their MyProxy username and passphrase every time they want to do a directory listing or submit a transfer would make the interface unusable. The most obvious optimization is to not activate the endpoint if it's already activated and does not expire for a sufficient amount of time. This is exposed in the API as the if_expires_in query parameter to the activation resource. However, this still requires entering MyProxy credentials twice to transfer between two TeraGrid endpoints using the same MyProxy server.
To further optimize activation, the API introduces a new concept: auto-activation. When a new endpoint is selected in the Web GUI, it actually performs the following steps:
- Auto activate the endpoint with an appropriate minimum lifetime.
- If the endpoint can't be automatically activated, the server returns the activation requirements, and the Web GUI creates a dialog to prompt the user for the username and passphrase.
- When the user submits the dialog, manual activation is performed.
You can think of auto-activation as lazy credential sharing between endpoints. If you activate one TeraGrid endpoint, other TeraGrid endpoints can be auto-activated using the same credential without requiring any user interaction. This also works if you sign in to Globus Online using a MyProxy credential. The minimum lifetime parameter applies to auto-activation as well - it will only share the credential if it has at least the requested lifetime.
An important aspect of activation is that different endpoints may require different credentials for the same user. I use my UChicago CI credentials to access PADS, and my TeraGrid credentals to access TeraGrid endpoints. Auto-activation takes this into account by only sharing credentials between endpoints with the same default MyProxy server.
The Transfer API is public, and communities and individual users are welcome to write their own clients to integrate with Globus Online. Most clients will want to follow the same workflow as the Web GUI: always perform auto activation first, and prompt the user only if that fails.
For more information about the Transfer API, head to the homepage at http://transfer.api.globusonline.org. The documentation has a more detailed information about activation, and you can sign up for the mailing list to ask questions and stay informed about new features and changes to the API. In the coming months we will be adding direct X509 proxy delegation as a new activation method, for when the client already has access to the user's credential. A 1.0 release is also in the works for later this summer.