TLS Certificate Optimization: The Technical Details behind "No Browser Left Behind"

Back in early December we announced our "no browser left behind" initiative to the world. Since then, we have served well over 500 billion SHA-1 certificates to visitors that otherwise would not have been able to communicate securely with our customers’ sites using HTTPS. All the while, we’ve continued to present newer SHA-2 certificates to modern browsers using the latest in elliptic curve cryptography, demonstrating that one does not have to sacrifice security to accommodate all the world’s Internet users. (If you weren’t able to acquire a SHA-1 certificate before CAs ceased issuing them on 2015/12/31, you can still sign up for a paid plan and we will immediately generate one to serve to your legacy visitors.)

Shortly after we announced these new benefits for our paid Universal SSL customers, we started hearing from other technology leaders who were implementing (or already had implemented) similar functionality. At first glance, the logic to identify incoming connections that only support SHA-1 seems straightforward, but as we spoke with our friends at Facebook, Twitter, and Mozilla, I realized that everyone was taking a slightly different approach. Complicating the matter even further was the fact that at CloudFlare we not only wanted to optimize between SHA-1 and SHA-2, but also between RSA and the newer, but less universally supported ECDSA certificates. Solve the "optimal certificate" question incorrectly, and the TLS handshake will fail — or get explicitly aborted by browsers that have deprecated SHA-1 entirely; solve it correctly, and the client and server will establish the most performant, secure connection available between the two endpoints.

After several trillion requests, we’re confident that our approach works quite well for CloudFlare’s customers and their visitors. If you have taken an alternative approach to implementation, or have found any exceptions/potential refinements to our logic, please chime in below. We remain committed to withdrawing SHA-1 support if, as our CEO said, "a vulnerability is discovered [in our certificate optimization logic] which allows some form of downgrade attack—where a modern browser can be tricked into receiving a certificate signed with an insecure protocol—and the vulnerability cannot be patched".

TLS Handshake

Before your web browser can securely exchange "application data" such as HTTP GET or POST requests and responses with a web server, it must first establish the cryptographic parameters of the secure session. This well-choreographed dance, known as the SSL/TLS handshake, commences as soon as you click, type, or get redirected to a URL containing the "https://" scheme. (The process described below also applies to connections from any user agent — not just browsers—so substitute "mobile app", "command-line utility", or anything else that can communicate via HTTPS.)

Note on embedded, traversable packet captures

Each figure below contains an actual SSL/TLS record embedded from CloudShark, a browser based version of WireShark. To view details of the handshake, expand the bottom-most row labeled "Secure Sockets Layer" and drill down until you see, e.g., "Handshake Protocol: Client Hello". Note that some extensions will render as "Unknown" due to CloudShark using an old version of WireShark, but these are immaterial to understanding our certificate switching logic.

Step 1 - Client Hello

After establishing a TCP connection to port 443 on the server, your browser transmits a ClientHello message containing the most recent version of TLS it supports; randomly generated data to be used for key exchange; the hostname of the website the user wants to browse; other capabilities or extensions it supports; and a rank ordered list of supported encryption algorithms.

If the session had previously been established with a server that supports session IDs or session tickets, this resumption data would have been sent in an attempt to abbreviate the process and avoid a “full” handshake; the steps below assume a full handshake, however, not an abbreviated one.

_Figure 1 - ClientHello message sent by Chrome to www.cloudflare.com\_

Step 2 - Server Hello

If the basic parameters of the ClientHello are acceptable to the web server, it responds with a ServerHello containing its own random data, the subset of the client-sent extensions it supports, and the selection it made from each of the options—(maximum supported) TLS version, cipher suite, and method of compression—provided by the client. Those interested in the cipher suites that CloudFlare’s edge supports can find them listed on GitHub.

_Figure 2 - ServerHello message sent by www.cloudflare.com to Chrome_

Step 3 - (Server) Certificate

Immediately after sending the ServerHello, i.e., without waiting for a response from the client, the server sends the Certificate message. This message contains — at minimum — the leaf certificate matching the requested site, but it also can contain other certificates in the chain such as the CA intermediate(s).

We'll come back to this message in a moment, as by this point any server-side logic to determine the optimal certificate to present to the browser — SHA-2 ECDSA, SHA-2 RSA, or SHA-1 RSA — must have been executed.

Figure 3 - Certificate message sent by www.cloudflare.com to Chrome

Step 4 - Certificate Chain Validation

While not an explicit step in most TLS handshake diagrams, it's important to pause here and review the process in the context of SHA-1 deprecation. When browsers like Chrome, Firefox, and Internet Explorer drop (or plan to drop) support for SHA-1 certificates, it is this certificate validation logic that they are modifying to explicitly reject such certificates[1]. Conversely, when we consider browsers that can only utilize SHA-1 certificates, it is here where they would encounter critical, handshake-aborting errors if presented with a SHA-2 chain for validation. (Even worse, the web server would almost surely not be able to distinguish between SHA-2 validation errors and intermittent network issues.)

After receiving the Certificate message, the browser[1:1] must first build a chain between the leaf certificate (i.e., the one valid for the hostname of the requested site) and the specific CA root that the system explicitly trusts due to inclusion in the root store. This process is fast and unambiguous when connecting to HTTPS providers—such as CloudFlare—that bundle the full chain together: leaf and intermediate(s). But for those that do not, the process can be slower and error-prone as the user agent is sometimes forced to decide between alternate paths with insufficient and imperfect information.

Then, for each certificate in the assembled chain, the browser extracts the signature provided by the issuing certificate authority (CA) and verifies it using the issuer’s public key. To do so, the browser uses the specified hashing algorithm — typically SHA-1 or SHA-256 — to create a digest of the data contained within the To Be Signed (TBS) Certificate section of the X509 structure (e.g., expiration date, valid hostnames, etc.). It then takes that digest, along with the issuer's public key and signature, and runs it through the "verify" function for the signature algorithm. If the output of this verify function is "true", the browser knows the structure is intact and unchanged from when the CA vetted it, as only someone in possession of the corresponding private key could have produced a valid signature.

Figure 4 - Validating a certificate signature

Remaining Steps (5+)

The handshake steps remaining after certificate verification—ClientKeyExchange, ChangeCipherSpec and Finished, followed by ChangeCipherSpec and Finished from the server—are not important to our certificate optimization logic, so they are not discussed here in detail. To summarize, the same primary key is generated on either end (ClientKeyExchange) and is used to encrypt traffic (ChangeCipherSpec) with a fast, symmetric, block cipher algorithm such as AES.

Now that we understand how user agents establish TLS sessions with servers, let’s turn our attention to how CloudFlare specifically determines which certificate to present and, relatedly, which cipher suite to use. As we’ll soon see, this logic must execute prior to the ServerHello and Certificate messages being sent to the client. Fortunately, we have everything we need to act after parsing the ClientHello.

Below is the logic tree that our edge traverses to craft the appropriate response to the client's ClientHello. We’ve labeled each decision point with a number that corresponds to the headings below so that you can understand each step in the process. Before you attempt to verify this logic with your own zone, please remember that i) this flow only applies to paid plans—free plans always see the SHA-256/ECDSA certificate—and ii) you may have Legacy Browser Support disabled, so make sure to confirm on the Crypto tab in your CloudFlare dashboard.

Figure 5 - Certificate Optimization Logic for Paid plans

Decision #0: Plan type

The first check is quite simple: if the zone is using the Free plan, we abort the rest of the checks and immediately return the SHA-256/ECDSA certificate. The reason we take this shortcut is that only one certificate is generated for free zones during onboarding, so there are no other candidates to consider besides SHA-256/ECDSA.

Decision #1: Presence of signature_algorithm extension

The next check that is performed is for the presence of the signature_algorithm extension (0x000d). This extension, added to TLS in version 1.2, “indicate[s] to the server which signature/hash algorithm pairs may be used in digital signatures.” The specification (RFC 5246) also reads:

The semantics of this extension are somewhat complicated because the cipher suite indicates permissible signature algorithms but not hash algorithms. If the client supports only the default hash and signature algorithms (listed in this section), it MAY omit the signature_algorithms extension. If the client does not support the default algorithms, or supports other hash and signature algorithms (and it is willing to use them for verifying messages sent by the server, i.e., server certificates and server key exchange), it MUST send the signature_algorithms extension, listing the algorithms it is willing to accept.

While many clients appear to support only the default set of signature algorithms ("sigalgs"), we have yet to see one TLS 1.2 ClientHello omit this extension (perhaps because the RFC is unclear on the default enumerations of hashes and signatures). If we don't see any sigalgs, we head to decision #5, while we head to #2 if we do.

Decisions #2-4: Inclusion of specific signature_algorithms and shared cipher suites

At this point we know we are dealing with TLS 1.2 so we begin by checking if SHA-256/ECDSA is present in the signature_algorithm array.[1:2] If it is, we also know that the client is able to validate a certificate signed with SHA-256 ECDSA, so we set that certificate as our top candidate and proceed to the next check.

With our optimal certificate in mind, we confirm, as §7.4.2 of RFC 5246 requires, that "the certificate MUST be appropriate for the negotiated cipher suite's key exchange algorithm and any negotiated extensions." Bear in mind that we haven't yet chosen the cipher suite—the ServerHello message is yet to be sent—so we take the opportunity here to filter the client-provided candidates to those that i) are compatible with our SHA-256/ECDSA certificate and ii) overlap with our preferred list as configured in NGINX.[1:3] After filtering this list, we try to choose a cipher suite that supports ECDSA keys, such as those that use ECDH(E)_ECDSA key exchange rather than ECDH(E)_RSA; assuming one is found, we select it and respond with the ServerHello and Certificate messages. If we’re unable to find a cipher suite that can be used with the zone’s SHA-2/ECDSA certificate, we look to see if SHA-256/RSA is an acceptable signature algorithm and return that certificate (and compatible cipher suite) to the user agent.

Note that adhering to this §7.4.2 check has helped us work around bugs in popular browsers such as Google Chrome. For example, during our implementation we discovered that Chrome incorrectly reports it supports SHA-2/ECDSA on Windows XP, even though that operating system has zero support for ECDSA signatures nor can it use the SHA-2 family of hashing algorithms without SP3 installed. (While Google acknowledged and initially committed code to fix the bug, they later determined that because i) cipher suites were still adjusted accordingly and ii) Windows XP support will be removed in the not too distant future, the fix was not worth merging.)

Decision #5: Presence of server_name_indication extension

Note that this check was added after initial launch, to reduce the frequency of SHA-2 capable clients receiving SHA-1. If we discover other ways to optimize our logic, we’ll push those as well.

If our edge code was unable to determine SHA-2 support by any other means (i.e., decision points #2-4 failed to converge on a certificate), we check for the presence of the server_name extension ("SNI") as a last resort before serving SHA-1. The reason this check works is due to assumptions we have made—and confirmed—on a few important dates: when the SHA-2 standard was published by NIST (5/2001), when the TLS extensions (6/2003) and TLS 1.1 (4/2006) RFCs were published, and how quickly vendors implemented these new specifications.

Outside of some extreme corner cases, we have been unable to find any instances of user agents that support SNI—defined in RFC 3546 in 2003, but not shipped by most major browsers until they added TLS 1.1 after 2006—but not SHA-2, which was published by NIST in 2001. Therefore, if we see SNI in the inbound request, we are confident that the user agent can validate a SHA-2/RSA signed certificate, so send it along with a compatible cipher suite. If we don’t, we’ve exhausted all of our signals that indicate the capability to use SHA-2, and move onto decision #6.

Decision #6: Setting of Legacy Browser Support in CloudFlare dashboard

If we reach this step we believe—based on the information conveyed in the ClientHello—that the user agent we are speaking with cannot support SHA-2 based certificates. But before serving a SHA-1 certificate, we first check to make sure that our customer (i.e., the owner of the site being requested), has not explicitly disabled support for Legacy Browsers.

While the vast majority of our customers are happy to support as many browsers as possible and thus leave this support on by default, some may be prohibited from doing so by their own information security policies. It is for reasons like this that we built a switch in the CloudFlare dashboard so customers can prevent our certificate optimization logic from falling all the way back to SHA-1 certificates. In the future, we plan to expand this UI control to allow clients to disable (or warn on) other protocols, such as TLS 1.0, 1.1, etc.

If you’d like to verify that the certificates you’ve uploaded are optimized appropriately for the browser’s cryptographic capabilities, you can run the following commands. Before doing so, verify that you have a version of OpenSSL that supports TLS 1.2 (i.e., at least 1.0.1).

Emulating a modern browser that supports TLS 1.2 with SHA-2 ECDSA certificates

$ openssl s_client -connect www.cloudflare.com:443 -servername www.cloudflare.com -tls1_2 </dev/null 2>/dev/null | openssl x509 -noout -text | egrep "DNS:|Signature Algorithm:"|head -n2
    
Signature Algorithm: ecdsa-with-SHA256
DNS:cloudflare.com, DNS:www.cloudflare.com

Emulating a browser that supports TLS 1.2 and SHA-2 but only RSA (not ECDSA)

$ openssl s_client -connect www.cloudflare.com:443 -servername www.cloudflare.com -tls1_2 -sigalgs RSA+SHA256 </dev/null 2>/dev/null | openssl x509 -noout -text | egrep "DNS:|Signature Algorithm:"|head -n2

Signature Algorithm: sha256WithRSAEncryption
DNS:cloudflare.com, DNS:www.cloudflare.com`

Emulating a legacy browser that supports TLS 1.1, SHA-2, and SNI

$ openssl s_client -connect www.cloudflare.com:443 -servername www.cloudflare.com -tls1_1 </dev/null 2>/dev/null | openssl x509 -noout -text | egrep "DNS:|Signature Algorithm:"|head -n2

Signature Algorithm: sha256WithRSAEncryption
DNS:cloudflare.com, DNS:www.cloudflare.com

Emulating a legacy browser that supports only TLS 1.0 and SHA-1 (no SNI)

$ openssl s_client -connect www.cloudflare.com:443 -tls1 </dev/null 2>/dev/null | openssl x509 -noout -text | egrep "DNS:|Signature Algorithm:"|head -n2

Signature Algorithm: sha1WithRSAEncryption
DNS:cloudflare.com, DNS:www.cloudflare.com

Thank you to Zi Lin and Nick Sullivan for their assistance in designing, implementing, deploying, and testing our certificate optimization logic.

Would you like to work on solving interesting problems like this for hundreds of millions of website visitors? If so, you're in luck: we're hiring for a wide range of engineering and product management positions and welcome you to apply through the CloudFlare Careers page.

1. The format used in this config file is the shorthand expected by OpenSSL. As such, each colon-delimited value potentially represents many individual cipher suites. To enumerate this list, try running: $ for cs in $(curl -s https://raw.githubusercontent.com/cloudflare/sslconfig/master/conf|grep ssl_ciphers|awk '{print $2}'|sed -e 's/:/ /g; s/!.*//; s/.$//'); do echo "expanding $cs:" && openssl ciphers -V $cs; done. ↩︎ ↩︎ ↩︎ ↩︎