Over the past few years, the CloudFlare blog has covered a great range of different topics, drilling down into the technology we use to both protect websites from attack, and optimise them so that they load faster for visitors.
One thing we haven't spent enough time talking about so far though also happens to be at the core of the way our service, as well as any service on the Internet works: DNS. CloudFlare offers DNS services for free- you don't even need to use any of our other free performance or security features to take advantage of our DNS. One of the side effects of building a network such as ours is that our DNS has properties unlike that of nearly all other DNS providers: even within the cutting edge world of Content Delivery Networks. These unique properties allow you to do things that would be much harder or impossible with other providers.
A bit about DNS
First though, a recap for the uninitiated. DNS (Domain Name System) is the way in which the human-readable addresses such as www.cloudflare.com are turned into the IP addresses that computers use to communicate with each other, such as
198.41.213.157 or 2400:cb00:2048:1::c629:d59d.
You can think of it as the world's greatest phone book, turning human-readable names such Joe Bloggs into the string of numbers you need to call them. DNS entries (or 'records', in the correct parlance) are announced by DNS 'nameservers', which hold the records for a particular domain name or list of domains, such as cloudflare.com. (As a side note, a common misconception is that the domain name includes the www at the beginning: the address www.cloudflare.com is actually a subdomain of the cloudflare.com domain.) Anyone can run a DNS nameserver, and become part of the DNS network.
When you enter the web site name www.cloudflare.com into your web browser and hit enter, the first thing it does is ask a DNS 'resolver' server (usually provided by your ISP) to find the machine-readable IP address that corresponds with the human-readable domain name. In order to do that, much like a postman searching for the correct house to post a letter, the DNS resolver starts at the end of the address, and works its way to the beginning.
First, it asks the 'root' nameservers for the location of the .com records. Once that's found, it asks the .com nameservers for the location of the nameservers for the cloudflare.com domain. Once we have that, it can finally ask the cloudflare.com nameservers for the IP address of www.cloudflare.com. The web browser can then carry on with the web request, now that it has the right IP address to send it to.
Note that in reality it's not necessary to find the root server and then the .com server every time because the DNS server will cache that information.
In theory, there is no difference between theory and practice. In practice, there is.
All great in theory, but there are a few ways in which this process is hampered. Firstly, let's say you run a web site with a DNS nameserver in London, and someone from San Francisco wants to visit your site. To find the correct IP address, they need to make a request all the way from San Francisco to London. A distance of 8,611km is going to add some latency to the request, and slow down the load time of your web site. Secondly, the DNS records have a property known as Time To Live (or TTL). This specifies the length of time that a DNS name should be cached with an ISP's nameservers before being refreshed. This caching means that a commonly accessed name like www.google.com isn't constantly being sent to Google's servers. The lower the TTL (it's usually specified in seconds) the more often DNS resolvers will ask the nameserver for the DNS records of a particular domain name.
Often to save on traffic and processing power, web hosts and other DNS providers will set this to hours or even days. That means that if you want to make some changes to your DNS, it can often take hours or days for the change to be seen by visitors to your web site, as the resolvers don't know to check for changes to your domain. (Web hosts often refer to this as 'propagation time', though the idea that DNS records need to 'propagate-out' is largely a myth: in most cases, it's the TTL set by the web host or DNS provider that causes delays in DNS changes.) Also like all servers, DNS nameservers will fail from time to time, and when they do your web site will drop off the internet.
Anycast DNS
The way CloudFlare routes traffic allows us to get around these issues, and provide interesting advantages to boot. As with the rest of CloudFlare's networking, our DNS works over Anycast. That means that instead of having to make requests all the way back to the web host, who could be on a different continent, the request is instead made to the nearest of CloudFlare's 24 data centres worldwide. For example, I'm based in the UK, but cloudflare.com is hosted in California. Even so, when I perform a traceroute to CloudFlare's DNS nameserver dns2.cloudflare.com, it's only 7 short hops to reach a server in our London data centre.
$ traceroute dns2.cloudflare.com traceroute to dns2.cloudflare.com (173.245.58.99), 64 hops max, 52 byte packets 1 10.0.1.1 (10.0.1.1) 1.572 ms 1.875 ms 2.047 ms 2 lo0-central10.pcl-ag07.plus.net (195.166.128.188) 16.613 ms 16.897 ms 15.741 ms 3 link-a-central10.pcl-gw01.plus.net (212.159.2.184) 15.757 ms 15.601 ms 15.641 ms 4 xe-9-0-0.pcl-cr01.plus.net (212.159.0.216) 15.787 ms 16.378 ms 15.766 ms 5 ae1.ptw-cr01.plus.net (195.166.129.0) 15.888 ms 15.742 ms 15.549 ms 6 195.66.225.179 (195.66.225.179) 46.333 ms * * 7 dns2.cloudflare.com (173.245.58.99) 16.925 ms 15.713 ms 15.698 ms
This can significantly reduce the time that your web site takes to load for visitors, anywhere in the world. Not only that, but because there's no one single physical DNS server, the chances that DNS would fail for the domain are greatly reduced.
Short TTLs
To solve the issue of having to wait ages for new DNS records to set correctly, CloudFlare has TTLs of 5 minutes on all DNS records by default. That means that if you make a change to, or add a new DNS record, you can expect visitors to your site to see the change in under 5 minutes (2 and a half minutes on average).
As well as never having to wait for DNS propagation again, this allows you to do things with DNS records that you wouldn't be able to do with other providers. Let's say you've been working on a replacement for your current website in a staging area on a different IP address, and it's now ready to go live. All you need to do is change the IP address set through your account with us and seconds later the new site will be live and publicly accessible.
Alternatively, let's say you have a server at home which contains personal files such as documents, music and other things you might want to access from work or a coffee shop. Some DNS providers including CloudFlare offer an API so you can write a script to constantly update an address such as home.example.com to point to your server at home, similar to a service such as DynDNS but with your own domain name. With a TTL of hours or days, an update to your home's IP address would make such a script pretty unreliable, as you would need to wait for your domain's records to change. With our DNS though, any downtime for that address wouldn't ever last more than a handful of minutes, allowing you to access your files remotely at all times.
Low TTLs can have more serious uses as well: let's say you run an enterprise-level service, and near-100% uptime is a major concern. If one of your servers stopped working, a quick DNS change to the IP address of your fail-over server is all that's need to get it back up again. As before, this process is scriptable, and can be set to happen automatically if the script detects issues connecting to the primary server.
Another advantage of low TTLs is it means we can move the IP addresses used for customers around at short notice. This is really important for CloudFlare as an attack mitigation service. When an attack reaches our network, our primary goal is to isolate it so that it doesn't affect any other customers, and 'null-route' it so that it doesn't go anywhere. Lowering the TTLs for DNS records even further allows us to do this on-the-fly, allowing us to react to attacks within a few seconds of them hitting our network edge.
Conclusion
Looking to the future, we are investigating other things we can do with the network we've built, and other DNS-related services we can add on top of the ones we already offer. One interesting feature would be the ability for traffic to be sent to different IP addresses depending on where the visitor is located, allowing customers with servers in multiple data centres to direct requests based on where it comes from. For the time-being though, our DNS offers some of the best reliability and speed in the industry, and can help to speed up your website, for free, regardless of whether you use any of the other services CloudFlare offers.
Sam Howson is a member of the Support team for CloudFlare, based in London. When he isn't helping customers to take advantage of CloudFlare's awesome features, he enjoys irritating his housemates with the sounds of his violin and mandolin, going to gigs, concerts & plays and exploring London on his bike.
CloudFlare is hiring! Do you love helping people, and want to work in one of the fastest growing and most exciting companies in tech? If so, check out our careers page. We are looking for team members for both our London and San Francisco offices, in a variety of roles including Technical Support.