Transform Rules:"Requests, Transform and Roll Out!"

Applications expect specific inputs in order to perform optimally. Techniques used to shape inputs to meet an application's requirements might include normalizing the URLs to conform to a consistent formatting standard, rewriting the URL’s path and query based on different conditions and logic, and/or modifying headers to indicate an application’s specific information. These are expensive to run and complex to manage. Cloudflare can help you to offload the heavy lifting of modifying requests for your servers with Transform Rules. In this blog we will cover the nuts and bolts of the functionality.

Origin server? : Thank you so much for offloading that for me, Cloudflare

Cloudflare edge servers? : No problem, buddy, I have taken care of that for you

When it comes to modifying an HTTP/HTTPS request with normalization, rewriting the URLs, and/or modifying headers, Cloudflare users often use Cloudflare Workers, code they craft that runs on Cloudflare’s edge.

Cloudflare Workers open the door to many possibilities regarding the amount of work that can be done for your applications, close to where your end users are located. It provides a serverless execution environment that allows you to create application functionality without configuring or maintaining infrastructure. However, using a Worker to modify the request is kind of like wearing a diving suit in a kiddie pool. Therefore, a simple tool to modify requests without Workers has long been wanted.

It’s in this context that we looked at the most common request modifications that customers were making, and built out Transform Rules to cover them. Once Transform Rules were announced we anticipated they’d become the favourite tool in our customers’ tool box.

• URL Normalization: normalizes HTTP requests to a standard format which then allows you to predictably write security rule filters.

• URL Rewrite: static and dynamic rewrites of the URL’s path and/or query string based on various attributes of the request.

• Header Modify: add or remove static or dynamic headers (based on Cloudflare specific attributes) to requests based on different various attributes of the request.

Bad actors on the Internet often encode your URLs to attack your applications because encoded URLs can bypass some security rules. URL Normalization transforms the request URL from encoded to unencoded before Cloudflare’s security features, so no one can bypass the firewall rules you configure.

For example, say you had a rate limiting rule for the path "/login" but someone sent the request as “/%6cogin”. Illustrated below:

You?: Rate Limiting for https://www.example.com/login to avoid brute force attacks.

Attacker?: You think you can stop me? I will issue massive requests to https://www.example.com/%6cogin to bypass your rate limiting rule.

Without URL Normalization, the request would bypass the rate limiting rule, but with URL Normalization the request is converted from the URL path /%6cogin to /login before the rule is applied.

By default, URL Normalization is enabled for all the zones on Cloudflare at Cloudflare’s edge, and disabled when going to origins. This means incoming URLs will be decoded to standard format before any Cloudflare security execution. When going back to the origins, we will use the original URL. In this way, no encoded URL can bypass security features and the origin also can see the original URL.

The default settings are flexible to adjust if you need. This FAQ page has more information about URL Normalization.

When talking about URL Rewrites, we always want to distinguish them from URL Redirects. They are like twins. Rewrites is a server-side modification of the URL before it is fully processed by the web server. This will not change what is seen in the user’s browser. Redirects forward URLs to other locations via a 301 or 302 HTTP status code. This will change what is seen in the user’s browser. You can do a URL Redirect with "Forwarding URL" in Cloudflare Pages Rules. Page Rules trigger actions whenever a request matches one of the URL patterns you define_._

Transform Rules come into play when we need to use URL Rewrite. This allows you to rewrite a URL’s path and/or query string based on the logic of your applications. The rewrite can be a fixed string (which we call ‘static’) or a computed string (called ‘dynamic’) based on various attributes of a request, like the country it came from, the referrer, or parts of the path. These rewrites come before products such as Firewall Rules, Page Rules, and Cloudflare Workers.

When visiting www.example.com with a cookie of version=v1, you want to rewrite the URL to www.example.com/v1 when going to the origin server. In this case, the end-user facing URL will not change, but the content will be the /v1’s content. This is a static URL rewrite. It only does rewrites when end users visit the URL www.example.com with cookie version=v1. It can help you to do A/B testing when rolling out new content.

When visiting any URL of www.example.com with a cookie of version=v1, you want to rewrite the request by adding /v1/ to the beginning of the URL for v1 content, when going to the origin server.

In this use case, when end users visit www.example.com/Literaturelibrary/book1314 with cookie version=v1, Cloudflare will rewrite the URL to www.example.com/v1/Literaturelibrary/book1314.

When end users visit www.example.com/fictionlibrary/book52/line43/universe with cookie version=v1, Cloudflare will rewrite the URL to www.example.com/v1/fictionlibrary/book52/line43/universe.

In this case, the URL visible in the client’s browser will not change, but the content returned will be from the /v1 location. This is a dynamic URL rewrite, so it applies the rewrite to all URLs when end users visit with the cookie.

When visiting any URL of www.example.com with a cookie of version=v1 and query string of page=1 that has /global in the beginning of the URL, this rule rewrites the request by replacing /global in the beginning for the URL with /v1 and updates the query string to newpage=1, when going to the origin server.

When end users visit www.example.com/global/Literaturelibarary/book1013?page=1 with cookie of version=v1, Cloudflare will rewrite the URL to www.example.com/v1/Literaturelibarary/book1013?newpage=1.

And when end users visit www.example.com/global/fictionlibarary/book52/line43/universe?page=1 with cookie of version=v1, Cloudflare will rewrite the URL to www.example.com/v1/fictionlibarary/book52/line43/universe?newpage=1.

In this case, the end-user facing URLs will not change, but the content will be v1’s content. This is a dynamic URL rewrite, so it applies the rewrite to all URLs when end users visit with the cookie of version=v1 and a query string of page=1.

Adding/removing request headers of the requests when going to origin servers. This is one of the most requested features of customers using Cloudflare Workers, especially those sending the Bot Score as a request header to origin. You can use this feature to add/remove strings and non-strings, and static or dynamic request header values.

Set Static header: Adds a static header and value to the request and sends it to the origin.

For example, add a request header such as foo: bar only when the requests have the hostname of www.example.com.

With the above setting, Cloudflare appends a static header Foo: bar to your origin when this rule triggers. Here is what the origin should see.

Set Dynamic header : Adds a dynamic header value from the computed field, like the end user’s geolocation.

The dynamic request headers are added.

Set Dynamic Bot Management headers: Cloudflare Bot Management protects applications from bad bot traffic by scoring each request with a “bot score” from 1 to 99. With Bot Management enabled, we can send the bot score to the origin server as a request header via Transform Rules.

The bot score header is added.

With Transform Rules, you can modify the request with URL Normalization, URL Rewrites, and HTTP Header Modification with easy settings to power your application. There’s no script required for Cloudflare to offload those duties from your origin servers. Just like Optimus Prime says “Autobots, transform and roll out!", Cloudflare says “Requests, transform and roll out!”.

Try out the latest Transform Rules yourself today.