Version and Stage Configuration Changes with HTTP Applications in Beta

Today, we are announcing a closed beta of HTTP Applications: a new way to safely test and deploy changes to your HTTP traffic. HTTP Applications introduce versioning of configuration and the ability to control when changes rollout to HTTP traffic on Cloudflare’s global edge network. Enterprise customers looking for greater control should reach out to their Customer Success Manager to get access.

Since the very first days of Cloudflare, management of websites and web applications has been done through what we called a Zone, which comes from the concept of a DNS Zone. While this model has served customers well over the years, it does create difficulties in managing edge configuration, namely:

1. Manual effort is required by customers to setup a staging environment.

2. Risk of drift in configuration between production and staging.

In software development, you want to test changes in a safe environment to validate them before they go to production or affect live traffic. In many common software development lifecycles, this means deploying changes to a staging or pre-production environment for testing and validation. The most common way customers do this today on Cloudflare is through the use of two Zones denoted by the hostnames of those zones, for example: one for staging named staging.example.com and one for production named example.com. This solves the core problem, as it provides insulation of changes. Errors in the staging zone will not affect production traffic.

However, in order to apply to production, changes that have been successfully verified in staging, the customer must manually copy those changes — or build an automation through the use of Cloudflare’s Terraform Provider. For many, this follows a manual ‘change request’ process in which a ticket is logged with the changes to be made. Then, someone (often a different person) picks up the ticket and must accurately reproduce the same changes based on manually provided instructions. This is an error-prone process; and an error in this process can lead to an outage, depending on the change involved. Additionally, a drift in the configuration between staging and production configurations could lead to further complications.

We want to provide customers with safety and reliability in managing their services on Cloudflare. In order to solve the aforementioned problems, we are announcing HTTP Applications along with Routing Rules.

HTTP Applications are a way to manage edge configuration by use case, rather than by hostname. Each HTTP Application has a purpose, whether that is handling the configuration of your marketing website or an internal application. Each HTTP Application consists of versions of configurations where each version represents a snapshot of settings for managing traffic — page rules, firewall rules, cache settings, etc.  Each version of configuration inside of the HTTP Application is independent of the others, but when a new version is created, it is initialized as a copy of the version that preceded it.

Unlike zones, each version of an HTTP Application is independent of any specific hostname. So if versions are not tied to a hostname, like zones, then how do you decide which version of an HTTP Application will affect a specific set of traffic? The answer is Routing Rules. With Routing Rules, you get to decide which version of an HTTP Application is applied to which traffic, aka hostnames. Routing Rules are powered by Cloudflare’s Ruleset Engine and rely on the use of conditional “if then” rules to map hostnames controlled in your Cloudflare account to a version of configuration. As an example, if a request’s hostname matches `www.example.com`, then apply version 2 of my Marketing HTTP Application. When this rule executes at our edge, instead of applying the regular zone configuration of www.example.com, the edge will instead use the configuration defined in version 2 of the HTTP Application.

Routing Rules supports two kinds of rules — staging rules and production rules. Both take a list of hostnames, as described, but when creating a staging rule, we additionally apply a filter such that the rule will only execute when traffic is sent to specific IPs at our edge. This means that you can safely test changes by sending traffic to www.example.com at the staging IPs while customers are unaffected. Even better, once you have validated your changes with the creation of a production routing rule, the exact same configuration will be applied in production for all your customers.

But that’s enough talk — let’s see it in action!

For this walkthrough, I am going to play the role of an existing customer. I have an existing zone serving customers and I want to make some changes to transform rules such that I can rewrite my assets location. However, I am not very good with regex and making a mistake will likely break the site for all of my customers! Instead of making the change directly in the zone, we are going to use HTTP Applications and Routing Rules to make, test and roll out the change.

First, I log into the Cloudflare Dashboard. After selecting my account, I see HTTP Applications available in the sidebar. Upon selecting that, I am given the option to create my first HTTP Application.

The Cloudflare Dashboard showing the empty state page for HTTP Applications

To create my first HTTP Application, I need to give it a name and select a pre-existing zone, in this case example.com. Cloudflare will use that zone to initialize the configuration of the HTTP Application’s first version. By copying over the existing settings from the zone, I have a safe copy to work from, and I don’t need to rebuild the configuration manually.

The “Create an Application” screen showing that an HTTP Application will be created named “Example Application” and initialized from example.com.

After selecting create, I have my first HTTP Application! Now, the first version is in the process of being created. Behind the scenes, Cloudflare will take the existing configuration of example.com and copy it to Version 1 of this HTTP Application. Once successfully copied, I can begin making edits to the configuration.

The Version list of the Example Application, showing Version 1 being created.

I can make edits to this version, just like I would with any zone. There are two important distinctions, however. First: any change I make right now will not affect any live traffic at Cloudflare’s edge, because we have not yet created any routing rules to send traffic to this version’s configuration. Second: we don’t allow everything associated with a zone to be controlled via an HTTP Application, namely: DNS records, SSL Certificates, Spectrum, or Load Balancing.

Transform rules of Version 1 showing no rules have been created.

In the Rules section under Transform Rules, I create a new rule to rewrite the path of assets to their correct location. For any request to example.com/assets/*, we will rewrite the path to be example.com/internal/files/assets/*.

Creating a transform rule for Version 1 named “Rewrite Assets”. This rule replaces the path for requests starting with “/assets/*” with “internal/files/assets/*”.

Transform rules of Version 1 showing a new rule named “Rewrite Assets” has been created.

At this point, I have made my change, but now I want to test it. To do that, I can leave the version editing section and head over to Routing Rules for this HTTP Application. Here I can create rules that will allow my existing traffic to be routed through this version’s configuration.

An empty list of Routing Rules for the Example Application.

I will create a staging rule, as I want to be the only one testing these changes without affecting any customers. Note, that when creating a staging rule, the IPs that can be used to test this version will be shown in the rule creation screen.

Creating a staging Routing Rule that will match when requests match example.com and the edge IP is 192.168.1.1 or 192.168.2.2 and apply the configuration of Version 1

After creating the rule, I can configure my computer to send requests to those IPs for example.com. Rackspace has a comprehensive guide for how to change your local machine’s host file to do this. Now, when I visit example.com, the new transform rule is executed, but for everyone else visiting the site, nothing has changed.

Routing Rules for the Example Application showing one rule for staging has been created.

Once I’m confident that my changes work, I can create a production Routing Rule that will apply these changes for all traffic to example.com — and I am done!

Routing Rule creation screen showing the creation of a production rule that will apply Version 1 when requests match example.com

Once updated, the path to assets for my site will be rewritten for all requests to example.com.

Routing Rules for the Example Application showing both a staging and production rule for Version 1.

What happens next? When I am ready to make another set of changes, I can go to my HTTP Application and clone Version 1 to create Version 2. Initially, Version 2 will exactly match the configuration of Version 1 but since it has no matching Routing Rules, it won’t be applied to any traffic.

The list of versions for the Example Application showing Version 1 being applied to staging and production, and Version 2 ready to edit, but not being used anywhere.

I can now safely edit Version 2, like I did previously with Version 1. This time I want to make some changes to firewall rules, so that I can prevent some potentially malicious traffic from accessing my site. Making the changes in Version 2 will not modify any traffic at the edge until I update my staging Routing Rule to use Version 2. This allows me to make changes with confidence, and then safely test them.

The list of routing rules for the Example Application showing Version 2 being used for staging, while Version 1 is still used in production.

After validating this new set of changes, I can push Version 2 to all traffic by updating the production Routing Rule to use Version 2. The same process can be done for each subsequent set of changes I want to make.

With the power of HTTP Applications and Routing Rules, customers now have greater control over how and when configuration changes are made. This alleviates the concern of making a bad change that might break your site. This capability is available in a closed beta for enterprise customers, but if you are interested, reach out to your Cloudflare account team to learn about how to get access!