How the Consumer Product Safety Commission is (Inadvertently) Behind the Internet’s Largest DDoS Attacks

by Matthew Prince.

The mission of the United State's Government's Consumer Product Safety Commission (CPSC) is to protect consumers from injury by products. It's ironic then that the CPSC is playing an unwitting role in most of the largest DDoS attacks seen on the Internet. To understand how, you need to understand a bit about how you launch a high volume DDoS.

Logo of the Consumer Product Safety Commission Logo of the Consumer Product Safety Commission

Amplification

DDoS attacks are inherently about an attacker sending more traffic to a victim than the victim can handle. The challenge for an attacker is to find a way to generate a large amount of traffic. Launching a DDoS attack is a criminal act, so an attacker can't simply go sign up for large transit contracts. Instead, attackers find ways to leverage other people's resources.

One of the most effective strategies is known as an amplification attack. In these attacks, an attacker can amplify their resources by reflecting them off other resources online that magnify the level of traffic. The most popular amplification vector is known as DNS reflection.

DNS Reflection

We've written about DNS reflection attacks in detail before. The basics are that an attacker generates DNS requests from a network that allows source IP address spoofing. The attacker forges the victim's IP as the source of the DNS requests. The attacker sends these requests to DNS resolvers that aren't locked down and respond to requests from any network. The DNS resolvers receive the requests and then send the responses back to the spoofed IP of the victim.

This technique "amplifies" an attack because a small DNS request will generate a larger response. How much larger depends on the size of the DNS record that's being queried. And here's where the Consumer Product Safety Commission comes in.

#1 Among DDoS Attackers

You see, the CPSC’s DNS server will respond with a very large DNS record when sent a very small DNS query. This DNS query is only 65 bytes:

dig ANY cpsc.gov +notcp +bufsize=65535 @X.X.X.X  

Substitute X.X.X.X for one of the 13 million currently open DNS resolvers and you'll get back a response that is 4,426 bytes. To generate 68Gbps of traffic to a victim requires only 1Gbps of requests with a spoofed source IP address to open resolvers for the CPSC.gov DNS record. That's an amplification factor of 68x.

Now, lots of people have large DNS records, but the CPSC.gov record is very large and particularly popular among attackers. Based on both data about the large volumetric attacks we see hitting CloudFlare, as well as data from various resolver honey pots we run, approximately 94% of the DNS reflection attack requests we see are for CPSC.gov. (The next most popular are httrack.com which is used by 2.9% of attacker requests and isc.org which is used by 0.5%.)

At one level there's nothing the CPSC can do to prevent attackers from using their DNS record to launch attacks. CPSC's resources aren't being queried directly by the attackers so they can't block the requests themselves. However, they could construct their DNS record more carefully to make it smaller and therefore less attractive to attackers.

To see how, we can walk through the CPSC DNS record entry by entry and make recommendations on how to reduce its size.

The CPSC's Ginormous Zone

Here’s the CPSC.gov's entire 4,426-byte DNS response which you get from running the ANY DNS query above against an open resolver (scroll to the left in the grey boxes below to see the full DNS record):

cpsc.gov.        18894   IN  SOA auth00.ns.uu.net. hostmaster.uu.net. 994622 1800 600 1728000 21600  
cpsc.gov.        18894   IN  A   63.74.109.2  
cpsc.gov.        18894   IN  AAAA    2600:803:240::2  
cpsc.gov.        18894   IN  NS  auth61.ns.uu.net.  
cpsc.gov.        18894   IN  NS  auth00.ns.uu.net.  
cpsc.gov.        18894   IN  MX  5 stagg.cpsc.gov.  
cpsc.gov.        18894   IN  MX  5 hormel.cpsc.gov.  
cpsc.gov.        18894   IN  TXT "v=spf1 ip4:63.74.109.6 ip4:63.74.109.10 ip4:63.74.109.20 mx a:list.cpsc.gov -all"  
cpsc.gov.        18894   IN  DNSKEY  256 3 7 AwEAAbpeSszphwwkOIJn1ha6DE/W3YRXFR2vsMi0RKhq5x9t487UJc0c eamz5TZj6KV5/tzL8/Qr2jnTaQmpWtJHbnF0kqpxeZIR+wzaNbMtEH30 UF5BDv9Bya0W9I+40dS48996kedhEvL6KwmMelB7FH6QPd0ixyhp0+ci 5vew9lzTESEsJ2X2uJrCqo3UacsHyYIzaTSXPpfwizQCql4VySq6+im1 74QaYw/FU4aADAv3R2KQvsR/uI0a7oOihxDDAvtYG7SZvotW3ASZFscd 4B6Yd84RMZC3yGdGtyrSD6tZsiJZoXhLQkkf0kOTWCjPvD8oPm+yYiGI Cm64eM3kflU=  
cpsc.gov.        18894   IN  DNSKEY  256 3 7 AwEAAXQUfP1/CdN5/YYXxWdePx3dWhhY7RmzxEfGXSZ0ea5BZoOXTLHd giWlm9ORnZ5hC+kaDRoYjgZXnkZOkhQhCDBwm2O2IOGBjBLMMtbm9hNK b2WGE8WC/E3j56YfepaMSzhxICu1xgY8JeYhmfpc3C5Z9Mm2oPm91cwU WZZY8i5f2F04tNBBymXTfu0mytCvp/dNxUjM45svY+SNRJltgcy07qBj T/GHDglEc6iJdBtvik3Nd4RfFfI+ftG8xSxfna3Nv4BVdYxPkE4us3ti 0dv/Ejwl9kuoXhT7/Ydpdze/boWmIuwjn3a66Afg7CHtmYyW6InLz57r tzUTGUdStgc=  
cpsc.gov.        18894   IN  DNSKEY  257 3 7 AwEAAZztz17cVspxUk8egfYEFLuyPXVETlPdT2PAuy+cZTk3afTS7cda Tnsk43AIqgnCkTvHE9m4gVuOhNmFjPIABPkfmaCtOzyqVmLjxb36JMxJ TnhBPBYjWY0HrBdEGCGG7eZzY4ll9kAMPIxE1OmMl9iM0dQSzamITEWN 89oPptHnlbjz8k7nQO3xyzXreamjhIW/2iIJhM+CdHe2CgMhPtf8b4QR 8CuIBMH07gvsTKljuvQLiS1ThQYYpmLgriiWjnQFum2FJe6J7x8joDAq YCzbQUdGSyJPp6FYibaG70Y62fIf9DNgHRMH/3c79DW9RmwzFggjfKLf y4hOgRbsVFc=  
cpsc.gov.        18894   IN  DNSKEY  257 3 7 AwEAAX5Tor9V7TnhfUMAL67reT+IFYd+4ciQv/UnvZbNgj7DgDuJPpcl Owh6ypAldCYgTXkF2Qt+an9WVp+Khsp2wRCCOhvGIUR9sOGdzxumDUCT Uru2dxHAqIn1QYSjuT8huMDDyBJmnoA4AY1Te86mcE1Jwpo+S9KoB23Z JgnMedU+6i8Qm9cdGLNM7nqEXhgKgmKc/387UFdh25jltsg0d2gOK//q k2HfLdDqv8XlrlacfMsSXniVwK7E6mtqcfbF518M2bl6UFJWXuxp+cU8 0WdmGiQfxmLvm62a2aVs9IzR6qGg0Ce5bxbx68v6gYTgIOUBm8ERytZ3 T2jzc0QOKQc=  
cpsc.gov.        18894   IN  NSEC3PARAM 1 0 12 AABBCCDD  
cpsc.gov.        18894   IN  RRSIG   NS 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. e4NrJJq0j7ZcHb/UAm0E34nuPfnbgCW6FARJM/5QKwgYDqZpCixaVQgL s9KGLR/eYQsp+BdPQibdLD3Ef2ICqQmBl0qRIyxh5Sbg7XiXnObJXgtR +vJuhVjzLtYfAGGicAm3MfYzcxk2/Usr8rwr/EakzhMLEr03Tshj4uwm WaRQ97M3R6dyTDvJ35X0m76KeWAyIS/Z2WmbGzNWUn8qpgerD8d1CIrP R7S5psgTMFIApD5I6mfJjuvZ5B+RRs7VhjY9tn6IKKoBLMqHxZAtwf1n td9Ho8anT3LJrrB1b/NZsRWhVdGLDlohDKzM9qqa6wQkoHg+/zrRrapZ 5NiUlQ==  
cpsc.gov.        18894   IN  RRSIG   MX 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. jAYmQYaSMOkU7VfwTqIb9CHO3nBBjkiYQ4zOoRIaD8u0ASMaweXhy/H2 75kxrhvwYs9VR0RXOlTal69uV9vN74cffu4lIfi9mIfSSVvo3JZz6dDM bpgmpdXRd6DEFVpmrHfTUK1sbKGB5qo3Yxiz38VzcZr7hclApmC/Bf0/ /luymk7BeCvtGjLVamRaaHKD76f/FA+0dkLvNOZITqrj36wZwOb1JTv8 XAl61dVxJl3R3z8TCtWm8+Za5nTzt3oNjad3Hf4g9r+88ugCvCcOibSU ujUH1sdc3/Uvr7ZEGT+51a6H6BkoctWwAimkN6CwbM0M8GhqwNiISXov 5QKO+Q==  
cpsc.gov.        18894   IN  RRSIG   TXT 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. CYoU8McQOKcy82SKfTXPfT8AnheHQmtdf4m0se/Ht7q64ll6iflSkh3O VGRmrQlgiMdAKaPIRMU7liGftMUntYKDu7RivVG18JokePDYHH9vYjss nQf92Reu08dOwm/Ica9cM1naGL1RDzvBbDkQkucyjJe2fvd00Npj+vBG H5kyLULmbkfD2PVL6c5F+eEhyGCF/IKPveag1ymkwBMGRM+Za+LYOeLY SkB1zwL5GodnHj/uaZfkGxT5ulNOnukMx536719pjf7ig/oMdaf8MySo hGQCjseS5ON6dRYuAzkOVDwdc5/Qw2jWqQlQUUH213AAYKz6eZXZtLAt +AflUQ==  
cpsc.gov.        18894   IN  RRSIG   A 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. DOLz8p1/pyr3LByKjHKZEcrgQwDqkB+Gg3nlcqSbFp8ChIenAjBGBvXy 24u7atI55ZxTqF6dtQEo3+D9a30piWeDm5U2tOg/VTHtwFFfi2h5DwQx WNxh6/71h7uzoidiBr4cKEkZSjB4F+bRviqJomxSdGPA9ZrQFZv4RcCb B8InJJZwyu9gfT767nt3yOq+Os/nlr19vyicmt5MiG9WQpxBn/ZhUG2u DYf6WkZVVo2i3nrD9PmpOjeDuj4g0DKAlhyPzPM5L8CKNDJSxGAdAHy+ auOJUFRPan3TGQqmgaqaEs/8HLiRr9QSgafuRJCO0HNW97p91zXX2krI AtDHgw==  
cpsc.gov.        18894   IN  RRSIG   AAAA 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. ANnB5Mw+iU6b2TD/3ULnt2LR138Vvd80BaYuOX765TNxpsoPosTxbls1 n6810qVvkHxR+uFafmrHA9PHOwJavEgmwSk4TyKbiTdu5fY0+wnR0tXr Fl8ARAALvmunyZ49aaEQUa0TUgKyNQIKoNktnfiLFhcFpsfQD678FGm3 IKtFnFpj3HvM3Y3vkfKORnqHEyT37O8EK5cIbK4YZ3UE/AJwqBEqUt7x xca51gRK3IW0/MBRoargRbg1zgVxhNCnnxWk59uMEER0wePsQK9XAMJ1 9QjpXlPFW+0426pFqbUTQyymwTjHFu2G8yzJas5g2r+oQShPTR80K6WG rmNbCA==  
cpsc.gov.        18894   IN  RRSIG   SOA 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. I7umZbpbUnh24bvphYP9K70imTyDFxc5QgOEDVVCSkmtVMhRaOU0eiSO 6b/O3ExcMhL0xn+D+/Xs1R7z9zPsKrzvGCfqotkxu8L1mRpAmjgr21WH mEzXftcx+IbSrOCwOa7YuPyBGqJW3f8kb/UHcI7ykqk08xvglD2I77PV TgI59oYsZyufRhfChcFxEyhBY2LZ4o/UO5Tw8Lg5HH/9OGtA0FHum1To v/CeLBOaATrZNygZINKmHlsP6Mm8j83jM4bRNxLGPs8VjpDbKVAHtROq aXR0Zy81zNs0eWfs4vThdOf56nCUnA/HiwlVNgmbO219CCMKQxoBBuY+ FYcWfA==  
cpsc.gov.        18894   IN  RRSIG   DNSKEY 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. HlTGZb9hSEwtxLayRdqxBkS+1jZIx/2FbsGaNJREfijFf7HTX/gpmXil g2DcVSvXwh6J473YKKpvdIe0LOfgIGA9NWaO5M3FVeKbwRk4OENdKTEG Vwm/uB8Av0aLsxD8YRvprPKxi7YxVDta7/cRMX1vP4ULQHrAbrJ5xlbu SzrlwK5b1BDoil3qfPNVbpHg2NMuqyp/hbFzUfWzvRNbLG3PJZfUEppl QTzInHnBONKmS0oRW4oUj8la2Zu6BGIOjB/0IktrLyxRKHJ2KNZDa1+o 0pKr68/gY2j7EfJwWrrohieG5LHfisCdTgaqs1zZbYgv+CAAl9hvq6vJ 5xrFxA==  
cpsc.gov.        18894   IN  RRSIG   DNSKEY 7 2 21600 20160824030507 20160817020507 58273 cpsc.gov. DlWHCC0re6XY3+MzRDYpJ2nIo2bWczGkJpxvA1yXhUivz8Qlv5mK1mtV 3YTislUzE9qWQTDOMoatcmQHVqZiHZG9d7w8/5XzBv3YuAqJNvUKs5QF 7YFBfF/acQCI40/pEyDhwNn5NbCJ3kpJBo7KSHBJpo9GNMnVQMmcK6bo UCeXEeI9Z1cnu/9XiD8jVSyVEMSs0KdjDmKEXNXgChVUWCr6+oWn8g/T 1SCUHF5mWn7r/ZgGPAyBwzoJMA/f9qdEBighiOz/DNe5dSKHAr/3Cf78 AJIFvEPe+Yn+KsRo14WLzzfuNT2NZp/nQ6TIHVLC04uDMXeimjQiHMJZ 8QDQzw==  
cpsc.gov.        18894   IN  RRSIG   NSEC3PARAM 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. ink6+sysDZdHK+1Ntfllouw2I7nAvqiV+9uPQI7O0MFdGMSIZJ6Dli7v z81S+yTp3diXZv+m4j0wTFSGtcN3Zte14jC+Kl+SJ2SRmfXedgcAXcZW QrPYgDps2C4jxxytxePKxHuDKLuSsxtUFmonOWKJuC2p7UqGDhsa5AHS XB8SoUX5ezgWjrSZgUmA7sWDE8szlvOtv1DtJKR62RIXQEOckd7z2HKh oQe5fZx1C1eoR+Ppgi1eNiC+Dqh3Q4FYcCzvSfY/rtZ88dyg8DKj0tV1 RMXYDR8+x18qSqAK9NlXLldumVTKBcC3rzTUKd0PLj5/sdbyaYo6X2mn RpHQXg==  
cpsc.gov.        893 IN  RRSIG   DS 8 2 3600 20160824041014 20160817041014 64415 gov. c78BIiQZ0L8C19OaNWlRtmr1/2cnL9yap0tcC4WPgKTtzluj4esJaN7u PZkKgaQME0BuImPs94s/IKy0JmWrobApCWPVjhtHq0E4m8UWsBbaORJX Ihg8JuS80rHFdPU6Y4u6Gu2ShTmE5eM+0wE1L5KKjt2co+Wa7AN4W2Kv Udo=  
cpsc.gov.        893 IN  DS  58273 7 1 75F81A67F1D76F017B02689E9586FCF3A68F655F  
cpsc.gov.        893 IN  DS  58273 7 2 A04919AA51B4395014E2753430A6DD6AF7585CE2C5C2E693E9ADD4DE C1365DC9  
cpsc.gov.        893 IN  DS  27793 7 2 C078BB7D7A46B0654A7D83755BA6FAB6AA56DAC4CB4D94B6A3562B67 BA7C285C  
cpsc.gov.        893 IN  DS  27793 7 1 E148C892A62ABF2B9157293DA4EE270D9880C238  

You may get the records back in a different order, but you should see something similar if you run the same ANY query.

Walking through the DNS zone file line-by-line you can see the mistakes that the CPSC has made that bloat its record and learn a bit about DNS and, in particular, DNSSEC.

The Good

The Good The Good, The Bad, and The Ugly artwork by Billy Perkins

Let's start with the good. Here are the first eight records, which are pretty standard and can't be optimized much.

cpsc.gov.        18894   IN  SOA auth00.ns.uu.net. hostmaster.uu.net. 994622 1800 600 1728000 21600  
cpsc.gov.        18894   IN  A   63.74.109.2  
cpsc.gov.        18894   IN  AAAA    2600:803:240::2  
cpsc.gov.        18894   IN  NS  auth61.ns.uu.net.  
cpsc.gov.        18894   IN  NS  auth00.ns.uu.net.  
cpsc.gov.        18894   IN  MX  5 stagg.cpsc.gov.  
cpsc.gov.        18894   IN  MX  5 hormel.cpsc.gov.  
cpsc.gov.        18894   IN  TXT "v=spf1 ip4:63.74.109.6 ip4:63.74.109.10 ip4:63.74.109.20 mx a:list.cpsc.gov -all"  

These first eight records are bread and butter DNS: SOA, A, AAAA, NS, MX, and TXT. If you want a quick refresher on these basic DNS records, read on. Otherwise, you can skip to the next section ("The Bad") to see where the DNSSEC mess begins.

The first record is the SOA record. SOA stands for Start of Authority. Walking through the record left-to-right, the number 18894 — which you'll see appears next to all records — is the TTL (Time to Live) at the moment for the particular resolver I queried. That number represents the seconds until the resolver needs to fetch another result from authoritative DNS server. If I queried the same resolver 10 seconds later, the number would be 18884 — 10 seconds less than it had been before. If you query a different resolver you'll likely get another number that is somewhere between 21600 (the max TTL the CPSC.gov has specified) and 0. When the number reaches 0, the resolver will query the authoritative name server, update its cache, and reset the TTL to 21600 — the maximum TTL specified for the record.

cpsc.gov.        18894   IN  SOA auth00.ns.uu.net. hostmaster.uu.net. 994622 1800 600 1728000 21600  

Continuing on the SOA record, next up is "IN" — which also appears on every DNS record in the CPSC zone. IN stands for "Internet Class." DNS predated the Internet, so there are other classes that are possible, but IN is the only class you'll see in common use today. After "IN" comes "SOA" which designates this record type.

The next entries are the contents of the SOA record. The first entry specifies the primary authoritative name server for the domain ("auth00.ns.uu.net."). The second is the email address of the name server's administrator ("hostmaster.uu.net." means [email protected] is the name server's administrator's email). "994622" is the zone serial number, which other name servers use to make sure they're in sync with the primary authoritative name server. "1800" is the refresh interval, specifying the number of seconds before non-primary name servers should check to see if the zone has changed. "600" is the retry interval, specifying the number of seconds to wait if a refresh failed. "1728000" is the number of seconds that non-primary name servers can serve a zone if they've failed to reach the primary name server. In practice with modern DNS setups, none of these numbers end up having much of an impact.

The last number in the SOA record, however, is important. "21600" is the length of time in seconds a negative result should be cached by recursive resolvers. For example, if you query "12345.cpsc.gov" — a record that doesn't exist — then the resolver you query will store the fact that the record doesn't exist for 21600 seconds.

SOA records look complicated but, in practice, only the last number — specifying the number of seconds that a negative result should be cached — has an impact on modern DNS implementations.

The other records in this group of 8 are less cryptic than SOA so we can quickly work through how they work.

cpsc.gov.        18894   IN  A   63.74.109.2  
cpsc.gov.        18894   IN  AAAA    2600:803:240::2  
cpsc.gov.        18894   IN  NS  auth61.ns.uu.net.  
cpsc.gov.        18894   IN  NS  auth00.ns.uu.net.  

The A and AAAA records are "anchor" records which specify the IP addresses responsible for the domain. A records are for IPv4 addresses — in this case 63.74.109.2 — and AAAA records are for IPv6 addresses — in this case 2600:803:240::2. The two NS entries define the name servers responsible for the domain ("auth00.ns.uu.net" which is primary as specified by the SOA record, and "auth61.ns.uu.net" which is secondary).

cpsc.gov.        18894   IN  MX  5 stagg.cpsc.gov.  
cpsc.gov.        18894   IN  MX  5 hormel.cpsc.gov.  

The two MX records specify the mail servers responsible for email sent to the domain ("hormel.cpsc.gov" and "stagg.cpsc.gov" — someone at the CPSC seems to have a sense of humor as Hormel is the maker of the pork product spam and "Stagg" is a brand of chili also made by Hormel). The number "5" in both MX records is the weight of that mail server. The lower the number the more it will be preferred. Since these are equally weighted, email will be load balanced equally between the two.

cpsc.gov.        18894   IN  TXT "v=spf1 ip4:63.74.109.6 ip4:63.74.109.10 ip4:63.74.109.20 mx a:list.cpsc.gov -all"  

Not much from the above could be optimized. The next record is special kind of TXT record known as a SPF record. It specifies what domains or IPs are allowed to send email on behalf of CPSC.gov. In this case, three IP addresses (63.74.109.6, 63.74.109.10 & 63.74.109.20), the MX records (Hormel & Stagg), and the A record for the subdomain (lists.cpsc.gov). The "-all" at the end means mail sent from other domains or IPs not specified here should fail. Alternatives could be "~all" which would soft-fail mail sent from other domains/IPs, or "+all" which would allow mail to be sent from any domain/IP — which would effectively render the SPF record useless.

There's a tiny optimization that CPSC.gov could make by listing one CIDR (63.74.109.0/24) rather than each of the three IPs. By doing so they'd save 15 bytes:

CPSC.gov.        18894   IN  TXT "v=spf1 ip4:63.74.109.0/24 mx a:list.cpsc.gov -all"  

However, as you'll see, that's hardly worth the effort compared with other optimizations they could make.

The Bad

The Bad The Good, The Bad, and The Ugly artwork by Billy Perkins

The misconfiguration of the CPSC.gov DNS largely involves how they've setup DNSSEC. DNSSEC is the mechanism to sign DNS records and help prevent cache poisoning. Each record in the DNS zone needs to be signed. To accomplish this you need RRSIG records and a simple change in their configuration could cut the size of the DDoS attacks the CPSC is indirectly responsible for nearly in half.

The signatures for each record are included as RRSIG records. Here's an example of the RRSIG record signing CPSC.gov's SOA record (again, scroll to the left in the box to see the whole record):

cpsc.gov.        18894   IN  RRSIG   SOA 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. I7umZbpbUnh24bvphYP9K70imTyDFxc5QgOEDVVCSkmtVMhRaOU0eiSO 6b/O3ExcMhL0xn+D+/Xs1R7z9zPsKrzvGCfqotkxu8L1mRpAmjgr21WH mEzXftcx+IbSrOCwOa7YuPyBGqJW3f8kb/UHcI7ykqk08xvglD2I77PV TgI59oYsZyufRhfChcFxEyhBY2LZ4o/UO5Tw8Lg5HH/9OGtA0FHum1To v/CeLBOaATrZNygZINKmHlsP6Mm8j83jM4bRNxLGPs8VjpDbKVAHtROq aXR0Zy81zNs0eWfs4vThdOf56nCUnA/HiwlVNgmbO219CCMKQxoBBuY+ FYcWfA==  

Let's walk through this RRSIG record. Just like with the previous records "18894" is the TTL in seconds, "IN" stands for "Internet Class", and RRSIG is the type of DNS record. "SOA" here specifies the type of record this signature applies to. That's easy. However, after that, things get cryptic (both literally and metaphorically).

The "7" here specifies the signature algorithm being used. By the DNSSEC specification, the 7th zone signing algorithm is: RSASHA1-NSEC3-SHA1.

Here's a list of the 10 current algorithms (represented by 14 numbers, just to be confusing) that can be used for signing public DNS records along with the number — or range of numbers if the signatures can be of variable lengths — of bytes each signature generates:

  1. RSA/MD5 / 128–512 bytes (deprecated)
  2. [Not used for RRSIG records]
  3. DSA/SHA1 / 41 bytes
  4. [Reserved]
  5. RSA/SHA-1 / 128–512 bytes
  6. DSA-NSEC3-SHA1 / 328 bits
  7. RSASHA1-NSEC3-SHA1 / 128–512 bytes
  8. RSA/SHA-256 / 128–512 bytes
  9. [Reserved]
  10. RSA/SHA-512 / 128–512 bytes
  11. [Reserved]
  12. GOST R 34.10-2001 / 64 bytes
  13. ECDSA Curve P-256 with SHA-256 / 64 bytes
  14. ECDSA Curve P-384 with SHA-384 / 96 bytes

Algorithm 7 allows the signer to specify a key length between 128 and 512 bytes. The CPSC specifies a key length of 256 bytes, which, today, for RSA is considered secure. Unfortunately, that means they have a 256-byte signature for every RRSIG record. Compare that with Algorithm 13 which, for a higher level of security, would produce only a 64-byte signature — one fourth the size per RRSIG record.

Returning to the RRSIG record, the "2" (after the "7") represents the number of "labels" in the domain being signed. In this case, the domain is cpsc.gov, which is 2 labels deep. If it were www.cpsc.gov then it would be 3 labels. a.b.cpsc.gov would be 4 labels.

"21600" represents the TTL that was used when the signature was originally calculated. Since the TTL could be different for each resolver's response, the TTL value needs to be set to a default to ensure that the signature is calculated consistently.

The next two numbers are timestamps. The first (20160824030507) represents the time this signature will expire (24 August 2016 at 03:05:07 UTC). The second (20160817020507) represents the time this signature became valid (17 August 2016 at 02:05:07 UTC).

"53799" is a checksum on the DNSKEY that was used to sign this record. This checksum is used to more quickly find the DNSKEY that was used to sign this particular record when more than one key is returned. (More on DNSKEYs in the next section.)

After the checksum is "cpsc.gov." which is the root domain. This root domain is included regardless of the number of subdomains (e.g., cpsc.gov, www.cpsc.gov, and a.b.cpsc.gov would all have "cpsc.gov." listed as the root domain). This specifies where to find the DNSKEY records that could have signed this record.

Next up is the signature itself. In this case, the signature is encoded in base64 and 256 bytes long. The base64 encoding means that the 256 bytes turns into 344 on the wire.

Stepping back, the easy optimization here for the RRSIG record would be to pick a better signature algorithm. By choosing Algorithm 13, rather than Algorithm 7, the CPSC could save 192 bytes per RRSIG record. We've written previously about why we use Algorithm 13 for CloudFlare's DNSSEC implementation.
Doing so here would add up because there are ten RRSIG records in the CPSC's zone in order to sign each of the other records (e.g., A, AAAA, NS, MX, etc...).

Just by switching to a better encryption algorithm the CPSC could increase the security of their DNS zone and save 1,920 bytes per DNS response. That small change alone would cut the maximum possible DNS reflection attack possible with the CPSC's zone by 44 percent.

While that's the biggest savings, choosing a less optimal outcome can't be considered an actual mistake. Unfortunately, the CPSC's zone has some ugly mistakes as well.

The Ugly

The Ugly The Good, The Bad, and The Ugly artwork by Billy Perkins

The RRSIG records discussed in the previous section need to be verified with a public key. These public keys are stored as DNSKEY records. For the CPSC, there are four DNSKEYs that each look like this:

cpsc.gov.        18894   IN  DNSKEY  256 3 7 AwEAAbpeSszphwwkOIJn1ha6DE/W3YRXFR2vsMi0RKhq5x9t487UJc0c eamz5TZj6KV5/tzL8/Qr2jnTaQmpWtJHbnF0kqpxeZIR+wzaNbMtEH30 UF5BDv9Bya0W9I+40dS48996kedhEvL6KwmMelB7FH6QPd0ixyhp0+ci 5vew9lzTESEsJ2X2uJrCqo3UacsHyYIzaTSXPpfwizQCql4VySq6+im1 74QaYw/FU4aADAv3R2KQvsR/uI0a7oOihxDDAvtYG7SZvotW3ASZFscd 4B6Yd84RMZC3yGdGtyrSD6tZsiJZoXhLQkkf0kOTWCjPvD8oPm+yYiGI Cm64eM3kflU=  

Walking through the record they're similarly formatted to RRSIG records. Once again, 18894 is the TTL on the record, the record type is DNSKEY and that's followed by three numbers: 256, 3 and 7. These are the flags field, protocol, and algorithm.

The flags field is 16-bits wide representing a number of potential boolean flags. Only two bits of this 16-bit field are currently assigned for use with the rest reserved for the future. A value of 256 (or 0000000010000000 or only the 9th flag set to true) indicates that this DNSKEY holds a key that can be used to verify RRSIGs. The protocol requires a 16-bit number so there's no optimization possible here.

The protocol field is 3. That is the only valid value for this field in a DNSKEY record. As the RFC says:

The Protocol Field MUST have value 3, and the DNSKEY RR MUST be  
treated as invalid during signature verification if it is found  
to be some value other than 3.  

Again, that seems a bit arbitrary but it's what the protocol requires so there's no optimization.

Finally, the algorithm field is 7 which indicates RSASHA1-NSEC3-SHA1. So, it should be an RSA public key formatted according to RFC3110.

Passing it through a base64 decoder and hexdump we can take a look inside:

00000000  03 01 00 01 ba 5e 4a cc  e9 87 0c 24 38 82 67 d6  |.....^J....$8.g.|  
00000010  16 ba 0c 4f d6 dd 84 57  15 1d af b0 c8 b4 44 a8  |...O...W......D.|  
00000020  6a e7 1f 6d e3 ce d4 25  cd 1c 79 a9 b3 e5 36 63  |j..m...%..y...6c|  
00000030  e8 a5 79 fe dc cb f3 f4  2b da 39 d3 69 09 a9 5a  |..y.....+.9.i..Z|  
00000040  d2 47 6e 71 74 92 aa 71  79 92 11 fb 0c da 35 b3  |.Gnqt..qy.....5.|  
00000050  2d 10 7d f4 50 5e 41 0e  ff 41 c9 ad 16 f4 8f b8  |-.}.P^A..A......|  
00000060  d1 d4 b8 f3 df 7a 91 e7  61 12 f2 fa 2b 09 8c 7a  |.....z..a...+..z|  
00000070  50 7b 14 7e 90 3d dd 22  c7 28 69 d3 e7 22 e6 f7  |P{.~.=.".(i.."..|  
00000080  b0 f6 5c d3 11 21 2c 27  65 f6 b8 9a c2 aa 8d d4  |..\..!,'e.......|  
00000090  69 cb 07 c9 82 33 69 34  97 3e 97 f0 8b 34 02 aa  |i....3i4.>...4..|  
000000a0  5e 15 c9 2a ba fa 29 b5  ef 84 1a 63 0f c5 53 86  |^..*..)....c..S.|  
000000b0  80 0c 0b f7 47 62 90 be  c4 7f b8 8d 1a ee 83 a2  |....Gb..........|  
000000c0  87 10 c3 02 fb 58 1b b4  99 be 8b 56 dc 04 99 16  |.....X.....V....|  
000000d0  c7 1d e0 1e 98 77 ce 11  31 90 b7 c8 67 46 b7 2a  |.....w..1...gF.*|  
000000e0  d2 0f ab 59 b2 22 59 a1  78 4b 42 49 1f d2 43 93  |...Y."Y.xKBI..C.|  
000000f0  58 28 cf bc 3f 28 3e 6f  b2 62 21 88 0a 6e b8 78  |X(..?(>o.b!..n.x|  
00000100  cd e4 7e 55                                       |..~U|  

Notice that the first four bytes are 03 01 00 01. That indicates an exponent of 65537 (03 means three bytes of exponent, 010001 in hex is 65537) which is very common for the RSA algorithm. The rest of the record consists of 256 bytes (2,048 bits). These 2,048 bits are the modulus part of the RSA key. Thus this is a 2,048 bit key.

Like with RRSIG, the CPSC could have better security and a shorter record if they specified a more efficient algorithm for their DNSKEY records. Using Algorithm 13 would save 192 bytes per DNSKEY, or 768 bytes across the 4 records.

But there's the rub: the CPSC doesn't need to have 4 DNSKEY records at all. Except during a key rollover, there should generally only be 2 DNSKEY records. In the CPSC's case, one of the DNSKEYs (id 59364) isn't signing anything. Another of the DNSKEYs (id 27793) is redundant. In other words, the CPSC includes 552 bytes of completely worthless data in every DNS record. Just eliminating these two useless records would cut the maximum size of a DNS Amplification DDoS attack using the CPSC's zone by almost 13 percent.

You can visualize the chain of how DNSSEC records are used to sign each other by using a tool provided by DNSVIZ. The diagram below shows how the two unused DNSKEYs don't actually sign any records.

CPSC's DNSSEC Diagram

A Better CPSC Zone

There are a handful of other minor optimizations. Add them all up and the CPSC Zone could be much smaller. At CloudFlare, we've worked to optimize the size and performance of our zone records. We used our automated zone generator on the CPSC and got the following zone file:

cpsc.gov.        281 IN  A   104.27.142.66  
cpsc.gov.        281 IN  A   104.27.143.66  
cpsc.gov.        281 IN  RRSIG   A 13 2 300 20160402121755 20160331101755 35273 cpsc.gov. xYJVr9iNMm50wgoGbMaG76QNyZk2aU2STmzxlUGe09G9LyEqnlDW6YLK pyAeMqHODKuxt83dKDUeB0bgvSPCyQ==  
cpsc.gov.        278 IN  AAAA    2400:cb00:2048:1::681b:8e42  
cpsc.gov.        278 IN  AAAA    2400:cb00:2048:1::681b:8f42  
cpsc.gov.        278 IN  RRSIG   AAAA 13 2 300 20160402121752 20160331101752 35273 cpsc.gov. LHmzdf6AqsrBVT5ejyoRt2oruXuaHkavhvoNHiqr2OXALnquouOfHsdq qRlHgQ1mkWRbeJ3nwIwDnWbInbIhkg==  
cpsc.gov.        272 IN  MX  5 stagg.cpsc.gov.  
cpsc.gov.        272 IN  MX  5 hormel.cpsc.gov.  
cpsc.gov.        272 IN  RRSIG   MX 13 2 300 20160402121746 20160331101746 35273 cpsc.gov. jmAvZk3bMxcqBECvvnhC7Xn2gtdvJnrRqj3xypzQ4Kona3tzs8H5+IBp a9X+TbcIGs1zgIny9UjybVUws+YTqw==  
cpsc.gov.        86391   IN  SOA abby.ns.cloudflare.com. dns.cloudflare.com. 2021076894 10000 2400 604800 3600  
cpsc.gov.        86391   IN  RRSIG   SOA 13 2 86400 20160402121805 20160331101805 35273 cpsc.gov. SrUvxJjlRvbAhFUXlEF6pbilXKLN/gzYcsyi4yYRR5o89k73nMv5mYZr vIi+uaPZe8Qht6nuNWRHhrYxQiUpLw==  
cpsc.gov.        86367   IN  NS  lars.ns.cloudflare.com.  
cpsc.gov.        86367   IN  NS  abby.ns.cloudflare.com.  
cpsc.gov.        86367   IN  RRSIG   NS 13 2 86400 20160402121741 20160331101741 35273 cpsc.gov. QO3FN7XVRMTewKuJGlbS1AglYnQrsbEfiWS65Mp+wT3D6n973Be9XzcR nvel4fPLvQTcIe/h5kpbXjut3nfhFQ==  
cpsc.gov.        3505    IN  DNSKEY  257 3 13 mdsswUyr3DPW132mOi8V9xESWE8jTo0dxCjjnopKl+GqJxpVXckHAeF+ KkxLbxILfDLUT0rAK9iUzy1L53eKGQ==  
cpsc.gov.        3505    IN  DNSKEY  256 3 13 koPbw9wmYZ7ggcjnQ6ayHyhHaDNMYELKTqT+qRGrZpWSccr/lBcrm10Z 1PuQHB3Azhii+sb0PYFkH1ruxLhe5g==  
cpsc.gov.        3505    IN  RRSIG   DNSKEY 13 2 3600 20160426183927 20160226183927 2371 cpsc.gov. o0c75Fa7KwogAAEpYIoWuTcsV1fPQ4sfuER+WOtxkOPb0Fe8PdF2yBWa 3gRLA/N2pmdCVrAAeIqdR0NDfX6PCw==  
cpsc.gov.        47  IN  TXT "v=spf1 ip4:63.74.109.6 ip4:63.74.109.10 ip4:63.74.109.20 mx a:list.cpsc.gov -all"  
cpsc.gov.        47  IN  RRSIG   TXT 13 2 300 20160402121801 20160331101801 35273 cpsc.gov. LXjZYeejRtkCFOVqqUpupc3I/4s7Ypua/b/xE3wIC+7a4NRjM7+UA90s gDdINQ8ZPF3SLcCqw2nfc3ex2II8Kg==  
cpsc.gov.        3324    IN  NSEC    \000.cpsc.gov. A NS SOA WKS HINFO MX TXT AAAA LOC SRV CERT SSHFP IPSECKEY RRSIG NSEC DNSKEY TLSA HIP CDS CDNSKEY OPENPGPKEY SPF  
cpsc.gov.        3324    IN  RRSIG   NSEC 13 2 3600 20160402121810 20160331101810 35273 cpsc.gov. 3/1I/SQrwvdffqawg6qG3zfetA1ou5SQioVv0kzMmHbYWJooQv9QEEFX L9RwdWxly0Hly/Aq+UMWzcQ8D2UXZg==  

That is 1,389 bytes and close to as optimized as you could manually create. The actual response may be a bit larger because resolvers can add a 320-byte DS record. Even so, the optimized CPSC zone would be about one third the size without compromising any functionality and actually increasing security.

But we can do better than that. Unlike the CPSC's current DNS provider (Verizon/UU.net), CloudFlare has anti-DNS reflection protections in place. Specifically, we automatically upgrade from UDP to TCP when a DNS response is particularly large (generally, over 512 bytes). Since TCP requires a handshake, it prevents source IP address spoofing which is necessary for a DNS amplification attack.

In addition, we rate limit unknown resolvers. Again, this helps ensure that our infrastructure can't be abused to amplify attacks.

Finally, across our DNS infrastructure we have deprecated ANY queries and have proposed to the IETF to restrict ANY queries to only authorized parties. By neutering ANY, we've significantly reduced the maximum size of responses even for zone files that need to be large due to a large number of records.

Denouement

DNSSEC is complicated and it's easy to get wrong. Unfortunately, getting your DNSSEC configuration wrong creates a real potential harm to the rest of the Internet by making your domain's zone file into a potential weapon to be abused by attackers. That's why, in CloudFlare's implementation, we spent a significant amount of time to optimize our automatic DNSSEC configuration to be as efficient as possible while still being easy and free for all our customers.

As Clint Eastwood says in the final scene of the movie "The Good, the Bad, and the Ugly": "You see in this world there's two kinds of people, my friend, those with loaded guns and those who dig. You dig." Pardon the puns but… we hope the CPSC will fix their zone and unload the giant gun attackers have used to launch some of the biggest DDoS attacks on the Internet. When they do, it'll be a relief to "dig" their domain and get back a reasonably sized answer.

If the instructions above aren't enough on their own, we'd be happy to work with the CPSC to help them get their DNS record under control and ensure they are no longer inadvertently facilitating the Internet's largest DDoS attacks. Of course, when the CPSC does get its DNS issues fixed, undoubtedly attackers will find another misconfigured and bloated zone they can abuse to launch DNS amplification DDoS attacks. And, when that happens, we'll be happy to work with them too — until the whole Internet has the well-configured DNS foundation it needs to be safe and secure.

comments powered by Disqus