Custom Load Balancing With Cloudflare Workers

The following is a guest post by Jayaprabhakar Kadarkarai, Developer of Codiva.io, an Online IDE used by computer science students across the world. He works full stack to deliver low latency and scalable web applications.

Have you launched your website? Getting a lot of traffic? And you are planning to add more servers? You’ll need load balancing to maintain the scalability and reliability of your website. Cloudflare offers powerful Load Balancing, but there are situations where off-the-shelf options can’t satisfy your specific needs. For those situations, you can write your own Cloudflare Worker.

In this post, we’ll learn about load balancers and how to set them up at a low cost with Cloudflare Service Workers.

This post assumes you have a basic understanding of JavaScript, as that’s the language used to write a Cloudflare Worker.

The basic pattern starts with adding ‘fetch’ event listener to intercept the requests. You can configure which requests to intercept on the Cloudflare dashboard or using the Cloudflare API.

Then, modify the hostname of the URL and send the request to the new host.

addEventListener('fetch', event => {
  var url = new URL(event.request.url);

  // https://example.com/path/ to https://myorigin.example.com/path
  url.hostname = 'myorigin.' + url.hostname
  
  event.respondWith(fetch(url));
});

This doesn’t do anything useful yet, but this is the basic pattern that will be used in the rest of the examples.

When you have a list of origin servers, pick a random host to route to.

This is a very basic load balancing technique to evenly distribute the traffic across all origin servers.

var hostnames = [
  "0.example.com",
  "1.example.com",
  "2.example.com"
];

addEventListener('fetch', event => {
  var url = new URL(event.request.url);

  // Randomly pick the next host 
  url.hostname = hostnames[getRandomInt(hostnames.length)];
  
  event.respondWith(fetch(url));
});

function getRandomInt(max) {
  return Math.floor(Math.random() * max);
}

What about when a host is down? A simple fallback strategy is to route the request to a different host. Use this only if you know the requests are idempotent. In general, this means GET requests are okay, but you might wish to handle POST requests another way.

addEventListener('fetch', event => {

  // Randomly pick the primary host
  var primary = getRandomInt(hostnames.length);

  var primaryUrl = new URL(event.request.url);
  primaryUrl.hostname = hostnames[primary];

  var timeoutId = setTimeout(function() {
    var backup;
    do {
        // Naive solution to pick a backup host
        backup = getRandomInt(hostnames.length);
    } while(backup === primary);

    var backupUrl = new URL(event.request.url);
    backupUrl.hostname = hostnames[backup];

    event.respondWith(fetch(backupUrl));
  }, 2000 /* 2 seconds */);

  fetch(primaryUrl)
    .then(function(response) {
        clearTimeout(timeoutId);
        event.respondWith(response);
    });
});

Cloudflare adds CF-IPCountry header to all requests once Cloudflare IP Geolocation is enabled.

You can access it using:

var countryCode = event.request.headers.get(‘CF-IPCountry’);

We can use the countryCode to route requests from different locations to different servers in different regions.

For example, 80% of the traffic to Codiva.io is from the US and India. So, I have servers in two different regions (Oregon, USA; and Mumbai, India). Requests from India and  other countries near it are routed to servers in India. All other requests are routed to the US data center.

const US_HOST = "us.example.com"
const IN_HOST = "in.example.com"

var COUNTRIES_MAP = {
  IN: IN_HOST,
  PK: IN_HOST,
  BD: IN_HOST,
  SL: IN_HOST,
  NL: IN_HOST
}
addEventListener('fetch', event => {
  var url = new URL(event.request.url);

  var countryCode = event.request.headers.get('CF-IPCountry');
  if (COUNTRIES_MAP[countryCode]) {
    url.hostname = COUNTRIES_MAP[countryCode];
  } else {
    url.hostname = US_HOST;
  }
  
  event.respondWith(fetch(url));
});

Now, let us combine the geographic routing, random load balancing and fallback into a single worker:

const US_HOSTS = [
  "0.us.example.com",
  "1.us.example.com",
  "2.us.example.com"
];

const IN_HOSTS = [
  "0.in.example.com",
  "1.in.example.com",
  "2.in.example.com"
];

var COUNTRIES_MAP = {
  IN: IN_HOSTS,
  PK: IN_HOSTS,
  BD: IN_HOSTS,
  SL: IN_HOSTS,
  NL: IN_HOSTS
}
addEventListener('fetch', event => {
  var url = new URL(event.request.url);

  var countryCode = event.request.headers.get('CF-IPCountry');
  var hostnames = US_HOSTS;
  if (COUNTRIES_MAP[countryCode]) {
    hostnames = COUNTRIES_MAP[countryCode];
  }
  // Randomly pick the next host 
  var primary = hostnames[getRandomInt(hostnames.length)];

  var primaryUrl = new URL(event.request.url);
  primaryUrl.hostname = hostnames[primary];

  // Fallback if there is no response within timeout
  var timeoutId = setTimeout(function() {
    var backup;
    do {
        // Naive solution to pick a backup host
        backup = getRandomInt(hostnames.length);
    } while(backup === primary);

    var backupUrl = new URL(event.request.url);
    backupUrl.hostname = hostnames[backup];

    event.respondWith(fetch(backupUrl));
  }, 2000 /* 2 seconds */);

  fetch(primaryUrl)
    .then(function(response) {
        clearTimeout(timeoutId);
        event.respondWith(response);
    });  
});

function getRandomInt(max) {
  return Math.floor(Math.random() * max);
}

In this article, you saw the power of Cloudflare workers and how simple it is to use it. Before implementing custom load balancer with workers, take a look at Cloudflare’s load balancer.

For more examples, take a look at the recipes on the developer docs page.