Introducing hostname and ASN lists to simplify WAF rule creation

If you’re responsible for creating a Web Application Firewall (WAF) rule, you’ll almost certainly need to reference a large list of potential values that each field can have. And having to manually manage and enter all those fields, for numerous WAF rules, would be a guaranteed headache.

That’s why we introduced IP lists. Having a separate list of values that can be referenced, reused, and managed independently of the actual rule makes for a better WAF user experience. You can create a new list, such as $organization_ips, and then use it in a rule like “allow requests where source IP is in $organization_ips”. If you need to add or remove IPs, you do that in the list, without touching each of the rules that reference the list. You can even add a descriptive name to help track its content. It’s easy, clean, and organized.

Which led us, and our customers, to ask the next natural question: why stop at IPs?

Cloudflare’s WAF is highly configurable and allows you to write rules evaluating a set of hostnames, Autonomous System Numbers (ASNs), countries, header values, or values of JSON fields. But to do so, you’ve to input a list of items directly into the rule expression editor, with all the associated downsides: it’s slow (you need to modify each rule individually), prone to error, and sometimes impossible (given the 4 KB limit of a custom rule expression).

Well, no longer! Today we are expanding Custom Lists by enabling you to create lists of hostnames and ASNs. The new list types are included in all Enterprise plans, so you’re free to start creating expansive lists the moment you read this.

You can now create a list of hostnames by navigating to Configurations > Lists in your account. This is the same place where you can manage your IP lists and browse the available Managed IP Lists.

Once the list is created, you can use it in any WAF rule expression. Account WAF users will find this useful as they will be able to run a managed or custom ruleset only on traffic matching a set of hosts.

Use of hostname lists in the filter of a custom ruleset (available to Account-level WAF users).

SSL for SaaS users will find hostname lists useful as they can restrict specific rules or rulesets to run on a subset of hosts. The list can be updated programmatically (via the API) to add or remove hosts as they are onboarded to the account.

A few things to know about lists: you can add domains and subdomains to a list, and a domain doesn’t automatically match subdomains. For example, if you add example.com to your list and use it in a custom rule to block traffic, requests for api.example.com won’t match. Hostname lists accept the * wildcard to include subdomains. For example, adding *.example.com to a list will match on api.example.com but it won’t match on ‘example.com’. Finally, example.com/path/subfolder wouldn’t be a valid entry — we are building string lists for this use case (more on that below).

An autonomous system (AS) is a large network or group of networks that has a unified routing policy. Every device that connects to the Internet is connected to an AS. Imagine an AS, as being like a town's post office, while an IP is the address of a single home. ASNs do not rotate like IPs do, making ASNs a better option when managing a larger portion of the IP space. Typically, each AS is operated by a single large organization, such as an Internet service provider, a large enterprise technology company, or a government agency.

You can use an ASN list to manage traffic from ISPs and cloud providers where bots generating automated traffic might be hosted. Using IPs would be less practical as the range of addresses would be too broad and change very frequently. Note that while ASNs are useful in this scenario, they should be used with caution, because blocking the wrong ASN could cause a large range of IPs to be affected.

Every Enterprise account can create up to 10 Custom lists with a total of 10,000 items shared across all lists. An account is considered Enterprise when at least 1 Enterprise plan has been purchased. Quotas are shared across all data types (IP, ASN and Hostname) and they are defined at the account level, so you can use your lists across all your applications.

For example, an account with one (or more) Enterprise plan can have 8 IP lists with 1,000 items each, one Hostname list with 1,700 entries, and one list with 300 ASNs; no more lists or items can be added.

Enterprise customers can increase their quotas by reaching out to their account team.

At the moment of writing Free, Professional and Business accounts have access only to IP Lists.

The following table summarizes what Custom List types will be available and on what fields they can be used. ASN and Hostname Lists are accessible from any WAF product built on the Ruleset Engine, including Custom, Rate Limiting and Managed rules.

The next chapter for lists is a hyper-flexible string type. You’ll be able to use it against fields such as header, cookie, path, query, JSON body field, user agent, JA3, MIME type, and more. For each entry you will be able to specify different matching operators, such as ‘exact match’, ‘start with’, ’ends with’ or ‘contains’.

Common applications include listing all the user agents you want blocked, listing URLs you want to restrict access to, and more.

More complex use cases include creating a different rate limiting rule that applies to a group of users that can be identified using cookies, API keys, or session IDs. A string list will also allow you to collect JA3 fingerprints of known malicious bots.

We are still working on string lists, and it will be released in the coming months. In the meantime — if you’d like to start using our new and improved lists, you can jump right into your dash today.