Protecting your APIs from abuse and data exfiltration

API traffic is growing fast. Last year alone it grew 300% faster at our edge than web traffic. Because APIs power mobile and web applications, transmitting instructions as diverse as “order a pizza from my favourite restaurant using this credit card” or “place a cryptocurrency trade and these are my personal details”, they are ripe for data theft and abuse. Data exposure is listed as one of the top threats for API traffic by OWASP; this includes data leaks. The increase in API traffic and more frequent data attacks call for new security solutions.

Cloudflare’s security toolkit had always been designed to protect web and API traffic. However, after talking to hundreds of customers we realised that there is a need for easily deployed and configured security tools for API traffic in a single interface. To meet this demand, in October 2020 we launched API Shield^TM, a new product aimed at bringing together all security solutions designed for API traffic. We started by providing mTLS authentication to all Cloudflare users free of charge, gRPC support and Schema Validation in Beta. During the launch we laid a plan for future releases with more advanced security functionality. We are now thrilled to be able to expand our offering with new features designed to protect your APIs from exposing sensitive data.

Today we are launching four features to help reduce the impact of exfiltration attacks: Schema Validation for all Enterprise customers, a managed IP List allowing you to block traffic from Open Proxies, more control over the certificate lifecycle, and a Data Loss Prevention solution. Later this week, we’ll also announce capabilities to help you discover APIs running on your network that you may not be aware of, and ways to identify anomalous requests that deviate from intended uses.

During the API Shield launch we introduced Schema Validation and we released it to a few selected customers. Over the last few months, we have been working with our early adopters to add more capabilities and build an easy-to-use interface directly into our dashboard. You can now navigate to the ‘API Shield’ tab where you can deploy API security products directly from the UI. The deployment flow will be expanded soon to include additional capabilities found elsewhere in the dashboard, such as mTLS and Rate Limiting. We will also be integrating new features such as API anomaly detection for an easy feedback loop.

Schema Validation works by creating a positive security model based on an API “schema”, which is a contract that defines the consumption of an API and guides developers to integrate it in their systems. Conversely to a negative security model (where rules define what characteristics a request must have to trigger an action, such as block), Schema Validation is designed to allow requests that have been verified as compliant while taking actions on everything else. Schema Validation accepts schemas that adhere to the OpenAPI v3 Specification (also known as Swagger Specification), which is the standard for defining RESTful interfaces (learn more at this page).

Schema Validation evaluates each request against an API Schema logging or blocking requests that do not comply with it.

API Shield offers an easy to use UI where users can upload their schemas to the Firewall and automatically create rules that validate each request against the API definition. If the request is compliant then it is forwarded to the origin. Conversely, if the format or data content of the request does not match what is expected by API Shield, the call is either logged or dropped protecting the origin from an invalid request or a malicious payload. Requests with extraneous input may not have been anticipated by the API developer, and they may trigger unforeseen application behaviour, such as a data leak.

API Shield with Schema Validation being deployed using the Cloudflare Rulesets OpenAPI schema.‌‌

The UI guides the user through different steps, including defining the hostname and base path of the API where API Shield will be deployed and uploading the schema file. Once API Shield with Schema Validation has been deployed, it is possible to inspect what endpoints have been parsed by the Firewall and what level of protection is being provided. In the review page, there are two groups of endpoints: protected and unprotected. The former lists all endpoints and methods whose schema is supported, the latter indicates any endpoint whose definition was either not supported or ambiguous. To avoid breaking traffic directed to endpoints not listed in the schema file, we include a final rule that matches traffic not directed to any of the protected endpoints.

Every time a non-compliant request is identified by Schema Validation and an action such as block or log is taken, a new event tagged with the source ‘API Shield’ is created and added to the Firewall logs. Users can access analytics and logs by visiting the Overview page where they can then drill down data using the flexibility of our GraphQL-powered dashboard.

Schema validation performs checks on the path, path variables, query parameters, headers, and cookies, and allows logging non-compliant traffic. Schema validation Beta is now broadly available to Enterprise customers — fill this form to get access.

Data loss is one of the biggest security concerns that affect small and large organisations, but it also has an impact on individuals and their privacy. Loss of sensitive data can have a massive impact on companies in terms of financial impact, brand value erosion, and compliance with the latest laws on data protection. Finally, loss of sensitive data belonging to individuals can have a detrimental effect in terms of monetary loss and privacy concerns.

Earlier today we announced our Data Loss Prevention (DLP) product suite; we’re now extending this to identify sensitive data leaving your origin in the response phase of an HTTP or API request. The solution evaluates the egress traffic, checking payloads against common patterns of sensitive data. These include personally identifiable information such as Social Security numbers and financial information, including credit card numbers, bank details, etc. For the first release, users will be able to log any match triggered by DLP. We are planning to add other actions such as obfuscating and blocking sensitive data leaving origins behind Cloudflare. Next, we intend to let customers customise the rules to identify sensitive data that are specific to their application.

We developed DLP with simplicity in mind so that every customer can be protected without requiring complex and time-consuming set up periods. We are releasing DLP as a managed ruleset that can be turned on through the Firewall Managed Rules tab. DLP can be used as part of the WAF of a reverse proxy, but it can also be used as part of Cloudflare for Teams integrating data protection in a Zero Trust configuration. This tight integration enables better control on who can access sensitive information within your organization.

DLP is in beta and we are releasing it to selected early adopter customers. Please fill this form to join the waitlist.

Data Loss Protection can be turned on as a Managed Ruleset.

We are launching our first Managed IP List which will be available for use within Firewall Rules. In July 2020 we released IP Lists, which gave customers the ability to upload large lists of IPs that can be used when writing Firewall Rules. Today we are launching a list that is curated by Cloudflare and that customers can use in their rules exactly as they use custom uploaded Lists.

‘Cloudflare Open Proxies’ contains the IPs of Open SOCKS and HTTP Proxies determined by Cloudflare by analysing traffic at its edge. This is not just limited to API requests; rules can apply to all types of traffic being evaluated by the Firewall.

The list is the first feed we are making public based on Cloudflare threat intelligence that leverages the scale and reach of our network. How do we populate this list? We see requests from every publicly routable IP address on the Internet. Cloudflare combines open source lists with its large network to identify open proxies. After verifying the proxies, Cloudflare determines the exit IPs and creates a list. We then feed this reputation data back into our security systems and make it available to our customers in the form of a managed IP list.

The list is available to all Enterprise plans and it can be used by selecting ‘Cloudflare Open Proxies’ in the drop-down menu collecting all available IP Lists (see picture below).

Cloudflare Open Proxies managed list can be used directly in the Firewall rule builder.

We launched mTLS with the first release of API Shield with mobile apps and IoT devices in mind. Enforcing strong authentication with client side certificates is a very effective measure to protect traffic from data exfiltration and abuse in general. However, in the event of an IoT device or mobile phone being stolen, lost or having control taken over by a malicious actor, Cloudflare users need a way to revoke the certificate that is considered a potential security risk. Having the ability to permanently exclude traffic from compromised devices is an effective way to prevent data loss and malicious attacks.

Many of our customers who have started embedding API Shield certificates in their apps have implemented a revocation solution using Workers with Workers KV. Although this solution allows granular control on certificates, it does require significant development effort from our customers and does not scale easily.

For Security Week, we are releasing a fully managed solution to revoke (and restore) certificates without the need to write a single line of code. We built a straightforward interface to manage the entire lifecycle of your certificates at our edge, from issuance to revocation. We take care of this for you, so you can focus on building your application without having to worry about setting up a complex and costly Public Key Infrastructure (PKI) and managing the revocation of potentially risky devices. The customer touchpoints are a new ‘Revoke’ and ‘Restore’ button in the client certificate tab, its supporting API calls and a new field for Firewall Rules.

Each request presenting a certificate to the Cloudflare’s edge will have two Firewall fields set: cf.tls_client_auth.cert_verified and cf.tls_client_auth.cert_revoked. The request is processed by the Firewall where users can combine these fields with all other Firewall functionality. This allows customers to set different behaviours based on whether the certificate was verified or verified but revoked. It also allows you to implement the required security policy while providing a good experience for end users. A classic configuration is to allow only requests with a verified certificate, while forwarding requests from revoked certificates to a different page or endpoint to handle the exception as required by your users’ journey.

Users can revoke certificates by visiting the Client Certificates tab.

The Cloudflare team is working on releasing additional features under the API Shield umbrella. We are talking to hundreds of customers who are using Cloudflare for API traffic and three features come up as high priority: in-depth API Analytics, a more flexible Rate Limiting tool, and integration with API Anomaly Detection.

Schema Validation and Data Loss Prevention are released today with full integration in our logs and analytics engine. Going forward, we are planning to expand the ability to analyse traffic and provide customers with tools to identify and manage attacks specifically directed to API endpoints.

Cloudflare’s rate limiting is designed to work best for web traffic where you can write rules based on URLs and request methods. We are now working on integrating the power of Firewall Rules with the control provided by rate limiting. This will allow users to segment their traffic leveraging the powerful logic available in the Firewall. We are also extending the counting mechanism to include the ability to rate limit based on API key and User ID.

When targeting API traffic, attack patterns can vary greatly, making traditional Bot Management solutions not the ideal candidate to identify suspicious behaviour. On Friday we will announce two major features that focus on further protecting your applications: API Discovery and Anomaly Detection. Discovery allows customers to map their endpoints and get visibility on the surface area of their APIs. Anomaly Detection is Cloudflare’s solution to autonomously separate good API traffic from malicious activity reliably and at scale. Customers will be able to set this up along with mTLS, Schema Validation, and Rate Limiting to maximise level of protection. Check out our blog on Friday to learn more about these new products.