Good security depends on having a lot of information and being able to react to it quickly. One of the problems with traditional web security has been that it relies on installing an appliance or software.
Once buried deep in a network, it is difficult for these security layers to receive updates on new threats, and even more difficult for them to relay information about the emerging threats they may have seen. As such, even security systems with a large installed base had a hard time getting smarter and responding to emerging threats.
CloudFlare's approach to security from the beginning has been different. Instead of hiding our appliance deep in the network, we built a performance and security network in the cloud. Our goal was to get as many sites as possible behind our network and form a sort of "neighborhood watch" for the Internet. The founding idea was that whenever any site on CloudFlare was attacked, information about the attack would immediately be shared with the rest of the network so we could all be better protected together.
To make this happen, today CloudFlare analyzes hundreds of megabytes of log data every minute looking for anomalies that indicate a potential attack. For example, we watch for visitors that generate a large number of Page Not Found (404) errors across multiple sites since this is a tell-tale sign of an attacker scanning for a vulnerability. We measure the rate at which crawlers move from page to page in order to sort human from non-human traffic. We look for signatures of known attacks as they are POSTed to forms. We record all the connections from zombie botnets during denial of service attacks. And, even once when we have stopped a potential threat, we continue to monitor the attacker for new, previously unknown behaviors that are then incorporated back into CloudFlare's security layer.
Spotting new security threats, however, is only half the battle. Of equal importance is ensuring that legitimate users are allowed to get through unhindered. To ensure this, CloudFlare's systems not only watch for evidence of bad behavior, but also evidence of good behavior. A web surfer who crawls in a way that statistically resembles a human visitor -- downloading images and CSS, following logical paths through a website, executing javascript and storing cookies -- gets a positive score making their behavior less likely to be challenged. If someone is challenged and they successfully pass the CAPTCHA that helps their score as well. Adding to these automated systems, information from the thousands of CloudFlare users TRUSTing and BLOCKing visitors on their own sites gets fed back into our security engine helping us get smarter over time.
We watch our false negative (when an attacker gets through) and false positive (when a legitimate visitor is stopped) metrics carefully and are proud that both metrics already rival enterprise-class security systems. That is in no small part because of all the existing members of the CloudFlare community. Every site that joins CloudFlare, whether a small personal blog or a major enterprise site, feeds data back to the community. And, together, with each new site that joins CloudFlare, we will continue to get smarter and smarter together toward our goal of securing and accelerating the entire Internet.