Industry News

CloudFlare Suffers an Hour-Long Outage while Mitigating a DDoS Attack

What started as a small-scale DDoS attack on a computer server wiped out a chunk of the Internet managed by DDoS protection company CloudFlare.

“The outage affected all of CloudFlare’s services including DNS and any services that rely on our web proxy. During the outage, anyone accessing or any site on CloudFlare’s network would have received a DNS error,” writes CloudFlare co-founder and CEO Matthew Prince in a blogpost.

The outage occurred when CloudFlare edge routers failed to connect the 23 CloudFlare data centers to the rest of the Internet using routers. The CloudFlare routers could no longer announce the Internet path the data packets need to reach their destination. Some 785,000 websites, including Wikileaks, 4chan and suffered.

Image Credit: CloudFlare

It started with a DDoS attack against the servers of one of CloudFlare customers, something that CloudFlare is extremely good at detecting and fending off. As they profiled the attack and issued a traffic routing rule (drop attack data packets of a considerable large size, between 99,971 and 99,985 bytes), they started propagating it via Juniper’s Flowspec protocol, to Juniper edge routers.

It is unknown why the rule, instead of dropping the attack traffic, drained all routers out of RAM memory, leaving them half crashed, unable to route any kind of data and also unable to serve remote management requests for a soft reboot.

With many of the edge routers unable to reboot automatically, the remaining routers got hit with the traffic across the entire CloudFlare network and got overloaded. The operation team had to manually unplug edge routers in the CloudFlare data centers, a time-consuming physical reboot that caused the hour-long outage.

“We let our customer down this morning, but we will learn from the incident and put more controls in place to eliminate problems like this in the future,” Prince said the official company announcement.

About the author


A blend of teacher and technical journalist with a pinch of e-threat analysis, Loredana Botezatu writes mostly about malware and spam. She believes that most errors happen between the keyboard and the chair. Loredana has been writing about the IT world and e-security for well over five years and has made a personal goal out of educating computer users about the ins and outs of the cybercrime ecosystem.