From Googlebot to GPTBot: who’s crawling your site in 2025
From May 2024 to May 2025, crawler traffic rose 18%, with GPTBot growing 305% and Googlebot 96%.
Carlos Azevedo
Lisbon
AI Everywhere with the WAF Rule Builder Assistant, Cloudflare Radar AI Insights, and updated AI bot protection
This year for Cloudflare’s birthday, we’ve extended our AI Assistant capabilities to help you build new WAF rules, added new AI bot & crawler traffic insights to Radar, and given customers new AI bot
Declare your AIndependence: block AI bots, scrapers and crawlers with a single click
To help preserve a safe Internet for content creators, we’ve just launched a brand new “easy button” to block all AI bots. It’s available for all customers, including those on our free tier
Gone offline: how Cloudflare Radar detects Internet outages
Cloudflare Radar will be publishing anomalous traffic events for countries and Autonomous Systems (ASes). These events are the same ones referenced above that have been triggering our internal workflow to validate and confirm disruptions
Sudan was cut off from the Internet for 25 days
The Internet started to come back to Sudan (with limitations) this past Thursday, November 18. This happens after 25 days (three weeks and four days) of an almost complete shutdown that affected the whole country.
