Last month, attackers compromised a Twitter team member’s access to an internal administrative panel in order to take over high-profile accounts. Full details of the breach are still pending, but Twitter has shared that the attackers stole credentials through a coordinated spear phishing attack.
The attackers convinced a team member to share login permissions, giving the attackers the ability to access the Twitter control plane. Once authenticated, they sent password reset flows to email accounts they controlled in order to hijack the Twitter accounts.
Administrative panels like Twitter’s are a rich target for phishing attacks because they give attackers a backdoor to privileged systems. Customer-facing teams at SaaS companies rely on these administrative panels to update end-user data and troubleshoot user account issues. If an attacker can compromise a single team member’s account they can potentially impact thousands of end users.
We have our own administrative panel at Cloudflare and we’ve deployed a number of safeguards over the last several years to keep it secure from phishing attacks. However, we had no way to enforce the security feature we think would most insulate us from phishing attacks: physical hard keys.
With hard keys, users can only login when they use a physical device - one that does not produce codes that could be shared over chat. Google notably eliminated all employee phishing cases by rolling out their own hard keys. We issued them to every team member who had to login to our identity provider to reach the admin panel, but our identity provider allowed users to to fall back to a less secure option for MFA.
To solve that problem, we moved the hard key requirement into Cloudflare’s network. Using Cloudflare Access, we can now restrict the ability to reach our admin panel only to team members who authenticate with a hard key. Starting today, we’re making that feature available to all teams.
Securing our own internal tools
Our admin panel gives our team members the ability to turn on features, manage settings, and investigate issues for our customers. The tool itself maintains its own set of application-level permissions that control the actions that administrators can take. Some customer accounts are scoped to specific team members; other accounts cannot be modified without the customer’s explicit approval.
We layer other safeguards on top of those permissions. Users who are inactive for a number of days need to be manually re-enabled. We regularly audit who can use the tool and trim back the list.
We used to lean on a VPN to keep the front door to the admin panel secure. Team members would connect to our VPN using a client on their device. Once on the VPN, they could reach the login page of the tool.
The VPN was painful for end users and a source of concern for our security team. Two issues, in particular, motivated us to find a better solution.
Segmentation was difficult. Not every user in the company needs to reach the admin panel, but any user on the VPN could connect to the login page. We wanted to limit that access to users in specific permission groups as part of a defense-in-depth strategy.
We could not add additional signals. Users could connect to the private network with just a password and MFA code. We could not limit VPN connections to corporate laptops, healthy devices, or other more granular restrictions.
About two years ago, we migrated that admin panel’s security perimeter to Cloudflare Access. Access gave us a zero-trust alternative to our VPN. Instead of being able to reach the admin panel because you are on the private network, Access continuously checks every request to the tool for identity against a list of allowed users.
We could use Access to limit who could reach certain tools, and it became a foundation for adding other sources of signal down the road. Access also gave us a new level of visibility. Since all requests pass through Cloudflare’s edge, Access provides our team with visibility into every request to every path. Without making any server-side code changes, Access can log every request and attribute it to an authenticated user. We can export those to our SIEM and create a comprehensive audit trail.
Hard keys without weak fallbacks
We integrated Cloudflare Access with our identity provider, which supports multifactor authentication (MFA). To login through Cloudflare Access, users would need to authenticate with their password and a MFA option. The ability to choose an option meant a less secure method could be selected.
Those fallback options were subject to a higher risk of phishing attack. SMS-based codes can be vulnerable to SIM swapping attacks. App-based time-based one-time-pin (TOTP) codes can linger on forgotten devices and, more dangerously, be transmitted as part of an attack.
Hard keys stand out because they rely on control of a physical item. With hard keys, users login with their password and then have to tap an actual key (typically in the form of a USB device). That tap presents a certificate, that only lives on the key, to the service configured to trust it. A user with the hard key could not inadvertently share that code over the phone or in a chat with an attacker.
We distribute hard keys to all team members at Cloudflare. However, we could not require team members to use them on an app-by-app basis. If I don’t have my hard key around, I always have the option to fall back to a TOTP code. We needed a filtering engine that could combine multiple sources of identity signal, which Cloudflare Access provided.
How Cloudflare Access solved our own problem
If you remember going to bars or clubs before the pandemic, Cloudflare Access might seem familiar. You had to show your identity card at the door to a bouncer. The bouncer (and the establishment) did not issue that card - your government did. However, they know what it should look like and how to use its information.
Once checked, the bouncer stamped the back of your hand. You could put your ID back in your wallet and the stamp became proof that the bartenders inside knew to trust.
Cloudflare Access works like that bouncer. When users connect to a resource secured by Cloudflare Access, we check for their ID by sending them to login to their identity provider like Okta or Azure AD. Users authenticate and their identity provider sends Cloudflare Access details like who they are and, for certain providers, what MFA method they used.
Like that stamp, Cloudflare Access uses the external identity information to create a distinct badge that we trust. Access generates a JSON Web Token (JWT) for the user and stores it in their browser. That token becomes the user’s badge for the rest of their session. Cloudflare Access looks for that JWT on every request the user makes to the application and enforces rules that an administrator configures about who can proceed.
Cloudflare Access can store more than just the user’s identity in the JWT. If the identity provider captures the MFA method used by a team member, Access can read that value and store it as an additional field in the JWT. RFC 8176, Authentication Method Reference Values, standardizes these values and how they are shared between systems.
We can use that standard to introduce an MFA option into the JWT created by Cloudflare Access. Access could then add an additional check that evaluated both the user’s identity and the MFA method they used to login.
The policy flexibility gave us what we needed to work with our security team to solve this problem. By adding a new rule that layers on top of the identity rule, we could immediately require that every team member logging in to our admin panel do so with their software-secured key.
In our case, that includes any hard keys supported by WebAuthN and FIDO2 or keys tied to a physical device like Apple Touch ID and Windows Hello. Access would reject the attempt if they (or an attacker) used the fallback TOTP code - even if the identity provider allowed the login.
How to get started
You can begin using the same feature our own security team needed today at no additional cost. You’ll need an identity provider that supports Authenticated Method Reference, or amr settings. Today, that consists of Okta and Azure Active Directory. We expect others will add support and we will update our documentation as they do.
To get started, navigate to an application you have in Cloudflare Access or create a new one. In the rules that determine who is allowed to reach the application, add a rule in the “Require” section. Select “MFA” from the dropdown and then choose which options you want to require.
What’s next?
Cloudflare Access, part of Cloudflare for Teams, is available today. You can follow the documentation here to add the additional rule. All accounts can use 50 seats of Cloudflare Access for free, including the hard key requirement feature.