Highly available and highly scalable Cloudflare tunnels

Starting today, we’re thrilled to announce you can run the same tunnel from multiple instances of cloudflared simultaneously. This enables graceful restarts, elastic auto-scaling, easier Kubernetes integration, and more reliable tunnels.

I work on Cloudflare Tunnel, a product our customers use to connect their services and private networks to Cloudflare without poking holes in their firewall. Tunnel connections are managed by cloudflared, a tool that runs in your environment and connects your services to the Internet while ensuring that all its traffic goes through Cloudflare.

Say you have some local service (a website, an API, or a TCP server), and you want to securely expose it to the Internet using a Cloudflare Tunnel. First, download cloudflared, which is a “connector” that connects your local service to the Internet through Cloudflare. You can then connect that service to Cloudflare and generate a DNS entry with a single command:

cloudflared tunnel create --name mytunnel --url http://localhost:8080 --hostname example.com

This creates a tunnel called “mytunnel”, and configures your DNS to map example.com to that tunnel. Then cloudflared connects to the Cloudflare network. When the Cloudflare network receives an incoming request for example.com, it looks up the cloudflared running mytunnel and proxies the request there. Then cloudflared proxies those requests to localhost:8080.

With Tunnel, your origin server no longer needs to allow any incoming traffic. In fact, it doesn’t even need a publicly reachable IP address. This is significant because that means that no one can simply bypass Cloudflare to reach your resource either.

Traditionally, Cloudflare customers onboard their sites to our platform with a simple nameserver change. By changing your nameserver, Cloudflare receives any queries to your resource first and leverages this as an opportunity to block malicious traffic and enforce policies and rules you define for your resource within the Cloudflare dashboard. However, if attackers discover your origin IP, they could bypass Cloudflare and your policies and rules.

Instead, with Tunnel, requests for your Internet property are proxied through the already-established outgoing connections from cloudflared to the Cloudflare network. This way any traffic entering your site will have to go through Cloudflare, where you can enforce more granular control with policies for caching, page rewrites, or Zero Trust security (e.g. only users with an @example.com email can view the page).

One feature request we’ve heard quite often is that our users need their software systems to scale. Their database must be scalable. Their web servers must be scalable. And of course, in turn, cloudflared must be scalable, because without cloudflared, our users can’t receive traffic.

For reliability purposes, cloudflared opens connections to four different Cloudflare servers (two in each of two different data centers, for redundancy in case a data center goes down). This way if one goes down, the other three will serve traffic while it reconnects. But what if the cloudflared process itself goes down?

Well, ideally, we would be able to scale or replicate cloudflared itself.

Previously, scaling cloudflared required using Cloudflare Load Balancer to spread traffic across multiple unique tunnels. Each tunnel then had to be manually authenticated, configured, and connected. That poses a challenge for teams who need to autoscale instances, like the resources in a Kubernetes cluster, without manual intervention.

Starting today, you can now create and configure an instance of cloudflared once and run it as multiple different processes in a replica model.

You can still point a DNS record or Cloudflare Load Balancer to a tunnel using its unique ID, but that tunnel is now represented by one or more identical instances of cloudflared - each with a unique connector ID.

When you run your tunnel, cloudflared will log its connector ID.

2021-03-29T18:40:17Z INF Starting tunnel tunnelID=610a53bd-ed0c-4afe-92b5-ca0238153410
2021-03-29T18:40:17Z INF Version 2021.3.5
2021-03-29T18:40:17Z INF GOOS: darwin, GOVersion: go1.16.2, GoArch: amd64
2021-03-29T18:40:17Z INF Generated Connector ID: 14e2e624-0d32-4a21-a88c-64acf9484dac

There’s a new command, cloudflared tunnel info , to show you each cloudflared running your tunnel.

$ cloudflared tunnel info mytunnel
NAME:     mytunnel
ID:       610a53bd-ed0c-4afe-92b5-ca0238153410
CREATED:  2021-03-26 19:29:34.291328 +0000 UTC

CONNECTOR ID                         CREATED              ARCHITECTURE VERSION  ORIGIN IP     EDGE         
71490dec-190f-4652-a70a-cd001fe6fdcf 2021-03-26T19:29:47Z darwin_amd64 2021.3.3 104.13.170.35 2xDFW, 2xMCI 
a0737d55-51f5-4fe0-8b53-c25989453c43 2021-03-26T19:29:58Z darwin_amd64 2021.3.3 104.13.170.35 2xDFW, 2xMCI 

At the moment, there’s a limit of 100 simultaneous connectors per tunnel. We think that should be enough for any customer, but if your use-case requires more, please reach out to support and we can raise it for your account.

Don’t let cloudflared become a bottleneck in your system. When your incoming traffic spikes, your origin servers should scale up, and your cloudflareds should scale up too.

With this launch, your team can dynamically start more instances of cloudflared without changing your DNS or Load Balancer configuration. The tunnel will distribute traffic between instances of cloudflared without the need to manually create and enroll new instances.

Right now it can be painful to change your cloudflared configuration. You’ll change the config file (or environment variables), then restart cloudflared. The problem is, this restart causes downtime while cloudflared stops accepting requests, restarts, reconnects to the edge, and starts accepting requests again.

Today’s announcement enables zero-downtime config changes. Instead of restarting cloudflared, simply start a second instance. The new instance will read the new configuration from the file. Once it’s connected to the edge and accepting traffic, you can stop the old cloudflared instance. The old instance will stop accepting new connections, wait for existing connections to finish, then terminate. Now 100% of your traffic is going through the new configuration, with zero downtime.

Cloudflared tunnels were previously incompatible with two of the most common Kubernetes scenarios:

1. Scaling up a service by adding another pod with identical configuration

2. Gracefully upgrading a service by adding another pod with the new version/configuration, waiting for it to become healthy, then removing the old pod

Unfortunately, neither of these worked with cloudflared, because the new cloudflared pod would fail to start. Instead, it would output an error message, saying it couldn’t run because its tunnel was already running somewhere else.

But now you can run many cloudflared pods, each running the same tunnel. We suggest the easiest way to use cloudflared with Kubernetes is to have your origin server encapsulated in a Kubernetes Service, and then use a separate Kubernetes Deployment for cloudflared. Configure cloudflared’s ingress rules to point at the origin Service. Now you can scale cloudflared and your origin service up or down, independently of each other.

In a modern distributed system, it’s important to avoid having a bottleneck or a single point of failure. Unfortunately, cloudflared sometimes became a single point of failure. Previously, you could mitigate this by running two cloudflareds, with different tunnels but otherwise identical configuration, and load balancing across these tunnels.

However, today you can simply run the same cloudflared multiple times -- whether in the same data center or two different continents -- and avoid the anxiety that comes from relying on a single program to keep your traffic flowing in.

If you’re interested in trying it out for yourself, check out our tutorial to get started today!