Subscribe to receive notifications of new posts:

The Hidden Costs of Heartbleed

2014-04-17

2 min read

A quick followup to our last blog post on our decision to reissue and revoke all of CloudFlare's customers' SSL certificates. One question we've received is why we didn't just reissue and revoke all SSL certificates as soon as we got word about the Heartbleed vulnerability? The answer is that the revocation process for SSL certificates is far from perfect and imposes a significant cost on the Internet's infrastructure.

Today, after having done a mass reissuance and revocation, we have a tangible sense of that cost. To understand it, you need to understand a bit about how your browser checks if an SSL certificate has been revoked.

OCSP & CRL

When most browsers visit web pages over HTTPS they perform a check using one of two certificate revocation methods: Online Certificate Status Protocol (OCSP) or Certificate Revocation List (CRL). For OCSP, the browser pings the certificate authority and asks whether a particular site's certificate has been revoked. For CRL, the browser pings the certificate authority (CA) and downloads a complete list of all the certificates that have been revoked by that CA.

There are pluses and minuses to both systems. OCSP imposes a lighter bandwidth cost, but a higher number of requests and backend lookups. CRL doesn't generate as many requests, but, as the CRL becomes large, can impose a significant bandwidth burden. These costs are borne by visitors to websites, whose experience will be slower as a result, but even more so by the CAs who need significant resources in place to handle these requests.

Technical Costs of Revocation

Yesterday, CloudFlare completed the process of reissuing all the SSL certificates we manage for our customers. Once that was complete, we revoked all previously used certificates. You can see the spike in global CRL activity we generated:

What you can't see is the spike in bandwidth that imposed. Globalsign, who is CloudFlare's primary CA partner, saw their CRL grow to approximately 4.7MB in size from approximately 22KB on Monday. The activity of browsers downloading the Globalsign CRL generated around 40Gbps of net new traffic across the Internet. If you assume that the global average price for bandwidth is around \$10/Mbps, just supporting the traffic to deliver the CRL would have added \$400,000USD to Globalsign's monthly bandwidth bill.

Lest you think that’s an overestimate, to make the total costs more accurate, we ran the numbers using AWS’s CloudFront price calculator using a mix of traffic across regions that approximates what we see at CloudFlare. The total cost to Globalsign if they were using AWS’s infrastructure, would be at least \$952,992.40/month. Undoubtedly they’d give some additional discounts above the pricing they list publicly, but any way you slice it the costs are significant.

Beyond the cost, many CAs are not setup to be able to handle this increased load. Revoking SSL certificates threatens to create a sort of denial of service attack on their own infrastructures. Thankfully, CloudFlare helps power Globalsign's OCSP and CRL infrastructure. We were able to bear the brunt of the load, allowing us to move forward with revocation as quickly as we did. And, no, we didn’t charge them anything extra.

So, if you're wondering why some people are dragging their feet on mass certificate revocation, now you know why — it imposes a real cost. And if you're a CA who's wondering what you're going to do when you inevitably have to revoke all the certs you've issued over the last year, we're happy to help.

Cloudflare's connectivity cloud protects entire corporate networks, helps customers build Internet-scale applications efficiently, accelerates any website or Internet application, wards off DDoS attacks, keeps hackers at bay, and can help you on your journey to Zero Trust.

Visit 1.1.1.1 from any device to get started with our free app that makes your Internet faster and safer.

To learn more about our mission to help build a better Internet, start here. If you're looking for a new career direction, check out our open positions.
OCSPHTTPSVulnerabilitiesReliabilitySSLOpenSSLSecurity

Follow on X

Matthew Prince|@eastdakota
Cloudflare|@cloudflare

Related posts

October 09, 2024 1:00 PM

Improving platform resilience at Cloudflare through automation

We realized that we need a way to automatically heal our platform from an operations perspective, and designed and built a workflow orchestration platform to provide these self-healing capabilities across our global network. We explore how this has helped us to reduce the impact on our customers due to operational issues, and the rich variety of similar problems it has empowered us to solve....

October 08, 2024 1:00 PM

Cloudflare acquires Kivera to add simple, preventive cloud security to Cloudflare One

The acquisition and integration of Kivera broadens the scope of Cloudflare’s SASE platform beyond just apps, incorporating increased cloud security through proactive configuration management of cloud services. ...

October 06, 2024 11:00 PM

Enhance your website's security with Cloudflare’s free security.txt generator

Introducing Cloudflare’s free security.txt generator, empowering all users to easily create and manage their security.txt files. This feature enhances vulnerability disclosure processes, aligns with industry standards, and is integrated into the dashboard for seamless access. Strengthen your website's security today!...

October 02, 2024 1:00 PM

How Cloudflare auto-mitigated world record 3.8 Tbps DDoS attack

Over the past couple of weeks, Cloudflare's DDoS protection systems have automatically and successfully mitigated multiple hyper-volumetric L3/4 DDoS attacks exceeding 3 billion packets per second (Bpps). Our systems also automatically mitigated multiple attacks exceeding 3 terabits per second (Tbps), with the largest ones exceeding 3.65 Tbps. The scale of these attacks is unprecedented....