We've been making a lot of improvements deep within CloudFlare that you'll start to see the benefits of over the coming weeks. As the changes graduate from our tests to full production, we'll try and give you a sense of what they are and why they're important. Today I wanted to tell you about one that I'm particularly excited about: end-to-end Keep Alives.
TCP Latency
CloudFlare acts as a reverse proxy. What that means is traffic passes from a visitor through our network of global data centers and then to a website's origin server. CloudFlare used to open a new connection from its network to the origin server for each item requested. While this is the standard behavior for most reverse proxies, it isn't particularly efficient. Each new TCP connection has a certain amount of overhead. By opening new connections for each resource, the TCP overhead can add up and add latency.
Ideally, what you'd want to do is open a connection to the origin server and, assuming that server is getting a constant stream of traffic, hold it open and pass all new requests over it. This not only lessens the TCP overhead, but also decreases the load on the origin server from having to setup and tear down a number of new connections -- something that can particularly slow in Apache.
Johnny 5 Is Alive
So that's exactly what we build. For the last month we've been testing the new Keep Alive setup in Singapore where it performed extremely well. This week we rolled it out across our entire network. For sites that get a constant stream of traffic, there is a significant improvement in the time to the first byte. Even for sites that don't get enough traffic to always hold open a persistent connection there is benefit because most page loads request multiple resources which are all pulled down over a single connection.
Overall our testing shows that this has improved CloudFlare's overall snappiness on average by another 10% or so. There is nothing you need to change in your settings; if your origin server supports Keep Alive connections you're already seeing the benefit. If your CloudFlare powered site seemed a bit faster the last few days, now you know why.
Stay tuned... more cool stuff to come!