Humanity wastes about 500 years per day on CAPTCHAs. It’s time to end this madness

Select all the buses. Click on bikes. Does this photo have traffic lights? As ridiculous as these questions are, you’re almost guaranteed to have seen one recently. They are a way for online services to separate humans from bots, and they’re called CAPTCHAs. CAPTCHAs strengthen the security of online services. But while they do that, there’s a very real cost associated with them.

Based on our data, it takes a user on average 32 seconds to complete a CAPTCHA challenge. There are 4.6 billion global Internet users. We assume a typical Internet user sees approximately one CAPTCHA every 10 days.

This very simple back of the envelope math equates to somewhere in the order of 500 human years wasted every single day — just for us to prove our humanity.

Today, we are launching an experiment to end this madness. We want to get rid of CAPTCHAs completely. The idea is rather simple: a real human should be able to touch or look at their device to prove they are human, without revealing their identity. We want you to be able to prove that you are human without revealing which human you are! You may ask if this is even possible? And the answer is: Yes! We’re starting with trusted USB keys (like YubiKey) that have been around for a while, but increasingly phones and computers come equipped with this ability by default.

Today marks the beginning of the end for fire hydrants, crosswalks, and traffic lights on the Internet.

In many instances, businesses need a way to tell whether an online user is human or not. Typically those reasons relate to security, or abuse of an online service. Back at the turn of the century, CAPTCHAs were created to do just that. The first one was developed back in 1997, and the term ("Completely Automated Public Turing test to tell Computers and Humans Apart") was coined in 2003 by Luis von Ahn, Manuel Blum, Nicholas J. Hopper, and John Langford.

By their very nature, the challenge-response nature of CAPTCHAs have to be automated: so they can scale across both humans and the bots they need to catch.

Put simply: we all hate them.

The best we’ve been able to do to date has been to minimize them. For example, at Cloudflare, we’ve continuously improved our Bot management solution to get as smart as possible about when to serve a CAPTCHA to the user. However, over the years the web moved from simple CAPTCHAs based on text recognition against backgrounds to OCRing old books to identifying objects from pictures as AI has improved (see Google paper on Street Numbers). This creates some real problems for the human users of the Internet:

1. Productivity: Time is lost — as is focus on the task at hand — and often in exchange for some frustration.

2. Accessibility: Users are assumed to have the physical and cognitive capabilities required to solve the tests, which may not be the case. A visual disability, for example, may make it impossible to perform a CAPTCHA-solving task.

3. Cultural Knowledge: The people on the planet who have seen a US fire hydrant are in the minority, as are the number who speak English. Cabs are yellow in New York City, and black in London — heck, ‘cabs’ are only cabs in a few places, and ‘taxis’ everywhere else!

4. Interactions on Mobile Devices: Phones and mobile devices are the primary — and most often only — means of Internet access for a large part of the world. CAPTCHAs put a strain on their data plans and battery usage, in addition to being more difficult on small screens.

In fact, the World Wide Web Consortium (W3C) worked on multiple drafts — as early as 2003 — pointing out the inaccessibility of CAPTCHAs.

And this is just from the user side. Inflicting all these costs on users has very real costs for businesses, too. There’s a reason why businesses spend so much time  optimizing the performance and layout of their websites and applications. That work stops users from bouncing when you want them to register. It stops shopping carts getting abandoned when you want them to end in the checkout. In general, you want to stop customers from getting frustrated and simply not come back.

CAPTCHAs are effectively businesses putting friction in front of their users, and as anyone who has managed a high performing online business will tell you, it’s not something you want to do unless you have no choice.

We started tackling these issues when we moved from Google reCAPTCHA to hCAPTCHA. Today, we are going further.

Hardware security keys are devices with an embedded secret that can connect to your computer or your phone

From a user perspective, a Cryptographic Attestation of Personhood works as follows:

1. The user accesses a website protected by Cryptographic Attestation of Personhood, such as cloudflarechallenge.com.

2. Cloudflare serves a challenge.

3. The user clicks I am human (beta) and gets prompted for a security device.

4. User decides to use a Hardware Security Key.

5. The user plugs the device into their computer or taps it to their phone for wireless signature (using NFC).

6. A cryptographic attestation is sent to Cloudflare, which allows the user in upon verification of the user presence test.

Completing this flow takes five seconds. More importantly, this challenge protects users' privacy since the attestation is not uniquely linked to the user device. All device manufacturers trusted by Cloudflare are part of the FIDO Alliance. As such, each hardware key shares its identifier with other keys manufactured in the same batch (see Universal 2nd Factor Overview, Section 8). From Cloudflare’s perspective, your key looks like all other keys in the batch.

There are at most three clicks required to complete a Cryptographic Attestation of Personhood. There is no looping, where a user is asked to click on buses 10 times in a row.

While there is a variety of hardware security keys, our initial rollout is limited to a set of USB and NFC keys that are both certified by the FIDO alliance and have no known security issues according to the FIDO metadata service (MDS). Our demo only includes support for YubiKeys, which we had the chance to use and test; HyperFIDO keys; and Thetis FIDO U2F keys.

“Driving open authentication standards like WebAuthn has long been at the heart of Yubico’s mission to deliver powerful security with a delightful user experience,” said Christopher Harrell, Chief Technology Officer at Yubico. “By offering a CAPTCHA alternative via a single touch backed by YubiKey hardware and public key cryptography, Cloudflare’s Cryptographic Attestation of Personhood experiment could help further reduce the cognitive load placed on users as they interact with sites under strain or attack. I hope this experiment will enable people to accomplish their goals with minimal friction and strong privacy, and that the results will show it is worthwhile for other sites to consider using hardware security for more than just authentication.”

The Cryptographic Attestation of Personhood relies on Web Authentication (WebAuthn) Attestation. This is an API that has been standardized at the W3C and is already implemented in most modern web browsers and operating systems. It aims to provide a standard interface to authenticate users on the web and use the cryptography capability of their devices.

As the need for stronger security with improved usability increases, we envision the deployment instances of WebAuthn to rise.

Platform

Compatible Browsers

iOS 14.5

All browsers

Android 10 and later

Chrome

Windows

All browsers

macOS

All browsers

Ubuntu

All browsers

Assuming you are using a hardware device with a compatible configuration, you might be wondering what is happening behind the scenes.

The short version is that your device has an embedded secure module containing a unique secret sealed by your manufacturer. The security module is capable of proving it owns such a secret without revealing it. Cloudflare asks you for proof and checks that your manufacturer is legitimate.

The longer version is that this verification involves public-key cryptography and digital certificates.

Public-key cryptography provides a way to produce unforgeable digital signatures. A user generates a signing key that can sign messages and a verification key that can be used by anyone to verify a message is authentic. This is akin to a signet ring, where the imprint of the ring is the signature and the ring itself the signing key.

Signature schemes are used widely to prove authenticity. Right now, your browser has verified that the server claiming to be “blog.cloudflare.com” is legitimate by verifying a signature made by someone who has a signing key associated with “blog.cloudflare.com”. To show the verification key is legitimate, the server provides a certificate that links the verification key to “blog.cloudflare.com”, itself signed by another verification key in another certificate. This chain goes all the way up to a root certificate from a Certificate Authority built into your browser.

Let's take another example. Alice owns a laptop with a secure module embedded. This module holds a signing key, sk_a. Alice says she sent a love letter to Bob yesterday. However, Bob is suspicious. Despite the letter stating "Hi Bob, it's Alice", Bob would like to be sure this letter comes from Alice. To do so, Bob asks Alice to provide her signature for the following message "musical-laboratory-ground". Since Bob chooses the message, if Alice can provide a signature associated with her verification key (pk_a), Bob would be convinced the love letter is from Alice. Alice does provide the signature, sk_a(“musical-laboratory-ground”). Bob confirms sk_a(“musical-laboratory-ground”) is associated with pk_a. He can now securely engage in their cryptographer relationship.

Thinking back to the Cryptographic Attestation of Personhood, you now know that your hardware key embeds a signing key. However, Cloudflare does not and cannot know the signing keys of all users of the Internet. To alleviate this problem, Cloudflare requests a different kind of proof. When asked if you are a human, we ask you to prove you are in control of a public key signed by a trusted manufacturer. When shipping devices with a secure module, manufacturers sign the associated attestation public key with a digital certificate.

Digital certificates usually contain a public key, information about the organization they are provisioned for, a validity period, the allowed usage, and a signature from a Certificate Authority making sure the certificate is legitimate. They allow metadata to be associated with a public key and therefore provide information about the issuer of a signature.

So when Cloudflare asks you to provide a signature, it verifies your public key has been signed by the public key of a manufacturer. Since manufacturers have multiple levels of certificates, your device provides a chain of certificates that Cloudflare is able to verify. Each link in the chain is signed by its predecessor and signs its successor. Cloudflare trusts the root certificate of manufacturers. Because their numbers are limited, we have the capacity to verify them manually.

Designing a challenge asking users to prove they are in control of a key from a certain manufacturer comes with a privacy and security challenge.

The privacy properties of the Cryptographic Attestation of Personhood are summarized in the following table.

Property

Cloudflare Could

Cloudflare Does

Get your fingerprints or face

NO

N/A

Know the manufacturer of your key

YES - limited to the number of keys in your batch*

YES

Associate a unique ID to your key

YES**

NO

* There must be 100,000 or more keys per batch (FIDO UAF Protocol Specification #4.1.2.1.1). However, self-signed keys and keys from certain manufacturers have been found to not meet this requirement.

**This would require that we set a separate and distinct cookie to track your key. This is antithetical to privacy on the Internet, and to the goals of this project. You can learn more about how we are removing cookies like __cfduid here.

The aim of this project: we want to know that you’re human. But we’re not interested in which human you are.

Happily, the WebAuthn API goes a long way to take care of this for us. Not that we want it, but the WebAuthn API prevents the collection of biometrics, such as a fingerprint. When your device asks for a biometric authentication — such as via a fingerprint sensor — it all happens locally. The verification is meant to unlock the secure module of your device, which provides a signature associated with your platform.

For our challenge, we leverage the WebAuthn registration process. It has been designed to perform multiple authentications, which we do not have a use for. Therefore, we do assign the same constant value to the required username field. It protects users from deanonymization.

A common use of CAPTCHA is to label datasets that AI has difficulty identifying. This could be for books, street numbers, or fire hydrants. While this is useful for science, it has also been used as a way for companies to leverage human recognition ability for commercial gain without their users’ knowledge.

With the Cryptographic Attestation of Personhood, this does not happen. We have more flexibility designing the user flow, as we are not constrained by the CAPTCHA challenge model anymore.

While the Cryptographic Attestation of Personhood has a lot of upside in terms of privacy, it is not perfect. Cloudflare still needs to know your manufacturer to let you in. As WebAuthn works with any certificate, we need to make sure Cloudflare receives certificates from untampered hardware keys. We would prefer to not have that information, further preserving your privacy.

We have worked on privacy standards in the past, leading the efforts with Privacy Pass for instance. Privacy Pass allows you to solve a challenge once and provide a proof you passed it, meaning you don’t have to solve multiple CAPTCHAs. It greatly improved the user experience of VPN users, who face more challenges than other Internet users.

For the Cryptographic Attestation of Personhood, we dig into an emerging field in cryptography called Zero Knowledge proofs (ZK proof). It allows our users to prove their manufacturer is part of a set of manufacturers trusted by Cloudflare. Using a  ZK proof, the devices from a single manufacturer become indistinguishable from each other, as well as from devices from other manufacturers. This new system requires more technical details and deserves a dedicated blog post. Stay tuned.

Designing a challenge aimed at protecting millions of Internet properties is no easy task. In the current setup, we believe Cryptographic Attestation of Personhood offers strong security and usability guarantees compared to traditional CAPTCHA challenges. During a preliminary user study, users indicated a strong preference for touching their hardware key over clicking on pictures. Nevertheless, we know that this is a new system with room for improvements.

This experiment will be available on a limited basis in English-speaking regions. This allows us to have diversity in the pool of users and test this process in various locations. However, we recognize this is insufficient coverage and we intend to test further. If you have specific needs, feel free to reach out.

Another issue that we keep a close eye on is security. The security of this challenge depends on the underlying hardware provided by trusted manufacturers. We have confidence they are secured. If any breach were to occur, we would be able to quickly deauthorize manufacturers’ public keys at various levels of granularity.

We also have to consider the possibility of facing automated button-pressing systems. A drinking bird able to press the capacitive touch sensor could pass the Cryptographic Attestation of Personhood. At best, the bird solving rate matches the time it takes for the hardware to generate an attestation. With our current set of trusted manufacturers, this would be slower than the solving rate of professional CAPTCHA-solving services, while allowing legitimate users to pass through with certainty. In addition, existing Cloudflare mitigations would remain in place, efficiently protecting Internet properties.

For Cloudflare, it always comes back to: helping build a better Internet. The very idea that we’re all wasting 500 years per day on the Internet — that nobody had revisited the fundamental assumptions of CAPTCHAs since the turn of the century — seemed absurd to us.

We’re very proud of the work we’ve done here to release the Cryptographic Attestation of Personhood. This challenge has been built with a user-first approach while maintaining a high level of security for accessing Internet properties sitting behind Cloudflare’s global network. We’re now in the process of augmenting our existing humanity challenge with the Cryptographic Attestation of Personhood. You should expect to see it more frequently as time passes. You can try it out today at cloudflarechallenge.com.

We want to acknowledge the work of other teams at Cloudflare. While this work is led by the Research team, we have been extremely privileged to get support from all across the company. If you want to help us build a better Internet, we are hiring.

Finally: we’re excited to bring about the demise of the fire hydrant on the Internet. It’s no longer needed.

As this is currently an experimental project from the Cloudflare Research Team only USB or NFC security keys work today. We’re happy for the feedback and will look into adding other authenticators as soon as possible. If you use a non-supported device then you’re likely to get a somewhat difficult to understand error message from your browser. On Google Chrome you would see:

If you would like to give us feedback on the Cryptographic Attestation of Personhood please fill out our Google Form.