This blog post is a summary of my project at the European Organization for Nuclear Research (CERN) where I worked in their Identity and Access Management (IAM) team.
My project was to introduce Two-Factor Authentication (2FA) in the Keycloak system. CERN started migration to the Keycloak Identity Provider (IdP) as part of the MALT project which aimed to move away from Microsoft products. The project was cancelled in late 2021, however, some services were allowed to continue development. The IAM team was allowed to continue development for the CERN SSO, which was based on Keycloak.
We developed a custom 2FA implementation with Keycloak which allowed users to optionally login from the SSO login page. Internally, this setup posed a few drawbacks as it was provided using a separate realm in Keycloak. Firstly, it did not enforce users to use 2FA, and only 10% of total CERN accounts were using 2FA in this old approach. Secondly, this approach required creating a separate realm in Keycloak, thus duplicating all users, which added an overhead on the software.
This flow allowed users to login with 2FA using a separate button in the SSO login portal, as shown below.
In the new approach, 2FA is tied to the user’s account, thus making it mandatory for each login. The motivation behind this approach came from the CERN annual (simulated) phishing campaign, where each year the CERN Computer Security team sends thousands of emails containing phishing links, which redirects users to a web form asking for their password. Although no actual data is collected, this gives us an indication of how many users will get compromised in a real-world scenario. In the 2022 CERN phishing campaign, around 2000 users gave away their passwords. In a real phishing incident this would have meant 2000 compromised accounts, unless they were protected with Two-Factor Authentication (2FA).
Always-on 2FA Approach
In late 2020, the CERN Computer Security team took a mandate to make 2FA mandatory and couple it with the user account. This new flow requires users to provide their 2FA token for each login, if enabled for their account. In addition to improving security, since the account became protected for all SSO access, this solved two main problems:
- Users were no longer required to re-login when they wanted to access a 2FA-protected application.
- The complex Keycloak setup and login flow could be simplified.
The new 2FA flow is tied to the user’s account rather than with the end application in a single realm. A realm in Keycloak is a space which includes users, applications, roles, and groups. This is made possible by providing 2FA from the cern realm in Keycloak, contrary to the 2FA provided in mfa realm, as done previously. A migration script migrate-users-2fa is responsible for migrating users from the old to the new 2FA flow. A comparison of the 2FA systems is shown below.
New vs. Old 2FA Flow
Supported MFA Methods
Keycloak natively supports Time-based one-time password (TOTP) and WebAuthn protocols. For WebAuthn we distribute Yubikeys throughout the CERN campus. The IAM team has developed a custom web portal where users can register their 2FA token. In the future we also plan to add an option to register multiple 2FA tokens.
OTP and WebAuthn MFA
Keycloak Configuration
We introduced several customisations in Keycloak, which were provided by implementing the Keycloak Service Provider Interfaces (SPIs). This included adding a custom role in the user token if they logged in with 2FA. We also changed the default Keycloak authentication flow to check for a specific role in the user profile and to prompt for a 2FA based on its presence. A check was also added for the user password reset flag in our database and to display a form to reset the user password.
Custom Extensions (SPIs)
Keycloak allows modifying the default authentication flow and add a custom check before the user logs in. In our custom 2FA login flow, we check if a user has the 2FA migrated role present in their token. We present the 2FA login screen only if this role is present. Other SPIs we developed include adding a custom role in the user token and checking if the user password has been compromised.
In the old login flow, 2FA login is provided using a separate button which redirects users to login using a different login screen. The new system incorporates 2FA and 1FA logins within the same login flow.
Note that in the figure below there is the addition of a 2FA Migrated Role which indicates whether a user is using the new 2FA flow or not. This is a temporary solution which allows both optional and Always-on 2FA systems to co-exist while we run the migration campaign.
2FA Authentication Flow
Migration Strategy
The CERN Computer Security team started rolling out the new 2FA flow in February, 2021. Initially, users from the IT department were migrated to the new flow and were asked for their feedback. Gradually, we asked for volunteers from other departments and currently have around 3,000 users in the new 2FA flow, while the total user count is above 10,000. The majority of unmigrated users are externally affiliated university employees at CERN and can be migrated at once.
The Computer Security team also maintains a list of critical accounts which are obliged to use 2FA, and do not have the ability to opt-out (i.e. disable both their 2FA tokens). The migration is controlled using an internal group which is open for subscription from any CERN account. A migration script is responsible for reading users from this group and 1) adding the required roles in their Keycloak profile and 2) copying their 2FA tokens from the old to the new realm. The chart below shows a detailed overview of migrated users in Keycloak.
2FA Migration Timeline
The diagram below shows which departments are fully migrated, partially migrated, and pending migration.
2FA Migration Progress
Command-Line Tool
The authentication team has developed a CLI tool in Python to migrate users and generate statistics on the migration. This tool runs as a cron job to periodically synchronize users in the 2FA group and adds the migrated role to their Keycloak profile. We currently have the following cron jobs in place:
migrate-users-2fa: Moves users to the new 2FA flow and copies their 2FA credentials.remove-users-from-mfa-role: Removes users from the new 2FA flow which are in the bypass list.get-users-2fa: Returns a list of users pending migration to the new 2FA flow.
This tool is also installed on a VM which is accessible by the CERN Computer Security team to generate their own statistics.
Critical Users
The Computer Security team has defined a list of critical users which are obliged to use 2FA for each login. These users do not have the ability to disable both their 2FA methods. This is made possible by adding these users to a special authorization group. In the API, we check if the user is part of this authorization group and deny their request to disable both their 2FA methods.
Kerberos Configuration
To simplify SSO access we also offer logins with Kerberos. All CERN-managed Windows machines get a valid Kerberos ticket on login and can login without re-entering their username and password. We provide Kerberos logins using a separate Identity Provider in both the cern and mfa realm. The reason to create a separate IdP was to avoid automatic logins if the user already had a valid Kerberos ticket. With this flow, users can choose to authenticate with Kerberos by pressing a separate button on the SSO login page, as shown below.
2FA Kerberos
Compromised Password Check
We added a check in the login page to search for the user’s password hash in a large database of compromised passwords. This database was obtained from the Have I Been Pwned website, and is hosted internally at CERN.
To achieve this check we developed a custom SPI in Keycloak which overrides the default authentication flow. This allowed us to intercept the user password and search it in our database. In case a match is found, we flag the user account and immediately notify them via email. The user has a grace period of 14 days to change their password, after which their account is blocked and a password reset is mandatory to access their account.
Presentations
The 2FA migration project at CERN was presented at these conferences.
Conclusion
We plan to finish the 2FA migration and remove all logic relating with optional 2FA flow. There have been numerous requests from users to add new features to our system; these include the ability to register multiple 2FA tokens and add support for step-up authentication for applications requiring 2FA. These features are now natively supported in newer versions of Keycloak and thus will be made available to our users. The timeline for providing this new functionality has yet to be determined.