2013: Rebuild the Engine; 2014: Step on the Gas

by Matthew Prince.

It's been a busy 2013 here at CloudFlare. By all external measures it was a terrific year. We grew page views, revenue and traffic across our network – all by more than 400%. We added terrific partners and high profile customers. We continued to hire great people, doubling team size and opening our second office in London near St. Paul's Cathedral.

However, 2013 was not without its challenges. For instance, the size of the denial of service attacks we saw over the course of the year reached record levels. To stay ahead of those attacks we upgraded our per server infrastructure from a 1Gbps platform to a 10Gbps platform. In the process we learned a lot about the capabilities — and limitations — of network gear vendors.

At the same time, CloudFlare's code base started to show its age. Lee originally wrote the first lines of code for CloudFlare's beta in early 2009. At the time I asked Lee how long he thought the architecture would hold up. "Three to four years," he estimated. Turned out he was exactly right.

We Can Rebuild Him, We Have the Technology

This last year was our "refactoring" year. Rather than adding new data centers we upgraded our existing facilities – adding equipment and network redundancy. We rebuilt our entire DNS infrastructure from scratch in Go. We cleaned up and simplified the pipeline that processes every HTTP request through our network, standardizing around Lua for its speed and agility. And we created a new, fully customizable, rules-based WAF to augment our original heuristics-based WAF.

Behind the scenes we've been cleaning up and rebuilding our APIs and getting ready to roll out a major upgrade to our website. And, we're close to a new logging infrastructure that will keep better pace with the more than 100GB of log data we generate every minute in order to offer our customers real-time intelligence into the legitimate and threat traffic coming to their sites.

Stepping on the Gas in 2014

So what's in store for 2014? If last year was about quietly working behind the scenes to refactor our existing systems, the coming year is about taking our new streamlined machine and stepping on the gas. Over the next 12 months we will significantly expand our data centers, adding facilities in regions we currently lack coverage: Latin America, the Middle East, Africa, and parts of Asia. China is our second largest market, Brazil is our third: in 2014 we’ll be significantly expanding our network to better serve these customers.

From a product perspective, one of the biggest requests we get is to expand the control developers can have over CloudFlare's platform. You should be able to access any feature and tweak any setting through a consistent, RESTful API. We realized the only way to do that was to eat our own dogfood so we're building our new website atop the APIs that we'll be opening to all our users.

Finally, we've been deeply troubled by some of the disclosures around government surveillance that came to light in 2013. From a technical perspective, we're committed to making available state-of-the-art encryption technologies to protect our customers' data on our network. To that end, in 2014 we will be rolling out SSL with perfect forward secrecy support to all our customers, even those at the free tier. That is a significant challenge for a number of reasons but we believe it's disappointing that there are only about 2 million SSL-protected sites online today. One day in 2014 we plan to double that. We think it's one of the most important things we can do to further our mission of building a better web.

Happy New Year!

We learned a lot in 2013. We learned what 10Gbps switches can (and cannot) keep up CloudFlare-scale traffic. We learned how to double, and then double again, the maximum number of requests per second a server on our network could handle. And we learned a lot about Brazilian import controls and customs regulations.

One of the things I think we've traditionally done well is share what we've learned. While it's been quiet on our blog, know that we’ve been heads down working hard to solve the challenges that arose throughout 2013. Now that we’ve done that, we're excited in the New Year to pull back the covers and share what we learned.

So, from everyone at the CloudFlare team, thanks for your support in 2013. Happy New Year! And stay tuned for an incredible 2014 ahead.

comments powered by Disqus