When any successful project attracts a large community, there will be growing pains. Since its launch in 2015, the experimental cloud computing testbed Chameleon has realized its vision of becoming a shared scientific instrument for computer science research and education. The project, led by Argonne Senior Scientist and UChicago CASE Affiliate Kate Keahey, has supported over 800 projects and 6,000 users from around the world and developed several new applications and deployments.
But until 2020, it was still using its original, basic identity and access management system, which created unnecessary hurdles for adoption and use. A new system would ideally allow for single sign-on across the Chameleon ecosystem as well as federated identity support — similar to the option of using Google, Facebook, or a University account to log into various websites and apps. Changing the sign-on system for a project this large without disruption is no small feat; in a new paper with former Chameleon DevOps lead Jason Anderson, Keahey described it as “tantamount to rebuilding the foundation under a skyscraper with thousands of inhabitants.”
The writeup of this epic moving operation received a Best Paper Award at the 2022 Practice and Experience in Advanced Research Computing (PEARC) conference, one of the leading supercomputing meetings. The honor recognizes that their approach doesn’t just benefit Chameleon and its users, but could help other projects make similar changes with minimal disruption.
“We were five years into operating the system, and our users created thousands of artifacts — images, orchestration templates, datasets — that were tied to their identity,” Keahey said. “We asked, how do we now port it to a completely different identity management system and preferably while the testbed is still operating? We tried a few solutions, but they were all either inefficient or brittle or did not scale.”
The team eventually arrived at a two-tiered architecture, combining a single sign-on solution built using the open source software Keycloak with a federated identity system provided by fellow University of Chicago project Globus. The former allows Chameleon users to use a single username and password to access the system through multiple routes, including Jupyter notebooks and the command-line interface. The latter allows for those login details to be connected with the user’s host institution login or providers such as Google, ORCiD, and the project’s original identity system from the Texas Advanced Computing Center.