Subscribe to receive notifications of new posts:

Custom Load Balancing With Cloudflare Workers

2018-10-03

2 min read

The following is a guest post by Jayaprabhakar Kadarkarai, Developer of Codiva.io, an Online IDE used by computer science students across the world. He works full stack to deliver low latency and scalable web applications.

Have you launched your website? Getting a lot of traffic? And you are planning to add more servers? You’ll need load balancing to maintain the scalability and reliability of your website. Cloudflare offers powerful Load Balancing, but there are situations where off-the-shelf options can’t satisfy your specific needs. For those situations, you can write your own Cloudflare Worker.

In this post, we’ll learn about load balancers and how to set them up at a low cost with Cloudflare Service Workers.

This post assumes you have a basic understanding of JavaScript, as that’s the language used to write a Cloudflare Worker.

The Basic Pattern

The basic pattern starts with adding ‘fetch’ event listener to intercept the requests. You can configure which requests to intercept on the Cloudflare dashboard or using the Cloudflare API.

Then, modify the hostname of the URL and send the request to the new host.

addEventListener('fetch', event => {
  var url = new URL(event.request.url);

  // https://example.com/path/ to https://myorigin.example.com/path
  url.hostname = 'myorigin.' + url.hostname
  
  event.respondWith(fetch(url));
});

This doesn’t do anything useful yet, but this is the basic pattern that will be used in the rest of the examples.

Load Balancer with Random Routing

When you have a list of origin servers, pick a random host to route to.

This is a very basic load balancing technique to evenly distribute the traffic across all origin servers.

var hostnames = [
  "0.example.com",
  "1.example.com",
  "2.example.com"
];

addEventListener('fetch', event => {
  var url = new URL(event.request.url);

  // Randomly pick the next host 
  url.hostname = hostnames[getRandomInt(hostnames.length)];
  
  event.respondWith(fetch(url));
});

function getRandomInt(max) {
  return Math.floor(Math.random() * max);
}

Load Balancer with Fallback

What about when a host is down? A simple fallback strategy is to route the request to a different host. Use this only if you know the requests are idempotent. In general, this means GET requests are okay, but you might wish to handle POST requests another way.

addEventListener('fetch', event => {

  // Randomly pick the primary host
  var primary = getRandomInt(hostnames.length);

  var primaryUrl = new URL(event.request.url);
  primaryUrl.hostname = hostnames[primary];

  var timeoutId = setTimeout(function() {
    var backup;
    do {
        // Naive solution to pick a backup host
        backup = getRandomInt(hostnames.length);
    } while(backup === primary);

    var backupUrl = new URL(event.request.url);
    backupUrl.hostname = hostnames[backup];

    event.respondWith(fetch(backupUrl));
  }, 2000 /* 2 seconds */);

  fetch(primaryUrl)
    .then(function(response) {
        clearTimeout(timeoutId);
        event.respondWith(response);
    });
});

Geographic Routing

Cloudflare adds CF-IPCountry header to all requests once Cloudflare IP Geolocation is enabled.

You can access it using:

var countryCode = event.request.headers.get(‘CF-IPCountry’);

We can use the countryCode to route requests from different locations to different servers in different regions.

For example, 80% of the traffic to Codiva.io is from the US and India. So, I have servers in two different regions (Oregon, USA; and Mumbai, India). Requests from India and  other countries near it are routed to servers in India. All other requests are routed to the US data center.

const US_HOST = "us.example.com"
const IN_HOST = "in.example.com"

var COUNTRIES_MAP = {
  IN: IN_HOST,
  PK: IN_HOST,
  BD: IN_HOST,
  SL: IN_HOST,
  NL: IN_HOST
}
addEventListener('fetch', event => {
  var url = new URL(event.request.url);

  var countryCode = event.request.headers.get('CF-IPCountry');
  if (COUNTRIES_MAP[countryCode]) {
    url.hostname = COUNTRIES_MAP[countryCode];
  } else {
    url.hostname = US_HOST;
  }
  
  event.respondWith(fetch(url));
});

Putting it all together

Now, let us combine the geographic routing, random load balancing and fallback into a single worker:

const US_HOSTS = [
  "0.us.example.com",
  "1.us.example.com",
  "2.us.example.com"
];

const IN_HOSTS = [
  "0.in.example.com",
  "1.in.example.com",
  "2.in.example.com"
];

var COUNTRIES_MAP = {
  IN: IN_HOSTS,
  PK: IN_HOSTS,
  BD: IN_HOSTS,
  SL: IN_HOSTS,
  NL: IN_HOSTS
}
addEventListener('fetch', event => {
  var url = new URL(event.request.url);

  var countryCode = event.request.headers.get('CF-IPCountry');
  var hostnames = US_HOSTS;
  if (COUNTRIES_MAP[countryCode]) {
    hostnames = COUNTRIES_MAP[countryCode];
  }
  // Randomly pick the next host 
  var primary = hostnames[getRandomInt(hostnames.length)];

  var primaryUrl = new URL(event.request.url);
  primaryUrl.hostname = hostnames[primary];

  // Fallback if there is no response within timeout
  var timeoutId = setTimeout(function() {
    var backup;
    do {
        // Naive solution to pick a backup host
        backup = getRandomInt(hostnames.length);
    } while(backup === primary);

    var backupUrl = new URL(event.request.url);
    backupUrl.hostname = hostnames[backup];

    event.respondWith(fetch(backupUrl));
  }, 2000 /* 2 seconds */);

  fetch(primaryUrl)
    .then(function(response) {
        clearTimeout(timeoutId);
        event.respondWith(response);
    });  
});

function getRandomInt(max) {
  return Math.floor(Math.random() * max);
}

Recap

In this article, you saw the power of Cloudflare workers and how simple it is to use it. Before implementing custom load balancer with workers, take a look at Cloudflare’s load balancer.

For more examples, take a look at the recipes on the developer docs page.

Cloudflare's connectivity cloud protects entire corporate networks, helps customers build Internet-scale applications efficiently, accelerates any website or Internet application, wards off DDoS attacks, keeps hackers at bay, and can help you on your journey to Zero Trust.

Visit 1.1.1.1 from any device to get started with our free app that makes your Internet faster and safer.

To learn more about our mission to help build a better Internet, start here. If you're looking for a new career direction, check out our open positions.
Cloudflare WorkersServerlessDevelopersJavaScriptDeveloper Platform

Follow on X

Cloudflare|@cloudflare

Related posts

October 31, 2024 1:00 PM

Moving Baselime from AWS to Cloudflare: simpler architecture, improved performance, over 80% lower cloud costs

Post-acquisition, we migrated Baselime from AWS to the Cloudflare Developer Platform and in the process, we improved query times, simplified data ingestion, and now handle far more events, all while cutting costs. Here’s how we built a modern, high-performing observability platform on Cloudflare’s network....

October 24, 2024 1:05 PM

Build durable applications on Cloudflare Workers: you write the Workflows, we take care of the rest

Cloudflare Workflows is now in open beta! Workflows allows you to build reliable, repeatable, long-lived multi-step applications that can automatically retry, persist state, and scale out. Read on to learn how Workflows works, how we built it on top of Durable Objects, and how you can deploy your first Workflows application....