Last month we shared statistics on some popular reflection attacks. Back then the average SSDP attack size was ~12 Gbps and largest SSDP reflection we recorded was:
30 Mpps (millions of packets per second)
80 Gbps (billions of bits per second)
using 940k reflector IPs
This changed a couple of days ago when we noticed an unusually large SSDP amplification. It's worth deeper investigation since it crossed the symbolic threshold of 100 Gbps.
The packets per second chart during the attack looked like this:
The bandwidth usage:
This packet flood lasted 38 minutes. According to our sampled netflow data it utilized 930k reflector servers. We estimate that the during 38 minutes of the attack each reflector sent 112k packets to Cloudflare.
The reflector servers are across the globe, with a large presence in Argentina, Russia and China. Here are the unique IPs per country:
$ cat ips-nf-ct.txt|uniq|cut -f 2|sort|uniq -c|sort -nr|head
439126 CN
135783 RU
74825 AR
51222 US
41353 TW
32850 CA
19558 MY
18962 CO
14234 BR
10824 KR
10334 UA
9103 IT
...The reflector IP distribution across ASNs is typical. It pretty much follows the world’s largest residential ISPs:
$ cat ips-nf-asn.txt |uniq|cut -f 2|sort|uniq -c|sort -nr|head
318405 4837 # CN China Unicom
84781 4134 # CN China Telecom
72301 22927 # AR Telefonica de Argentina
23823 3462 # TW Chunghwa Telecom
19518 6327 # CA Shaw Communications Inc.
19464 4788 # MY TM Net
18809 3816 # CO Colombia Telecomunicaciones
11328 28573 # BR Claro SA
7070 10796 # US Time Warner Cable Internet
6840 8402 # RU OJSC "Vimpelcom"
6604 3269 # IT Telecom Italia
6377 12768 # RU JSC "ER-Telecom Holding"
...What's SSDP anyway?
The attack was composed of UDP packets with source port 1900. This port is used by the SSDP and is used by the UPnP protocols. UPnP is one of the zero-configuration networking protocols. Most likely your home devices support it, allowing them to be easily discovered by your computer or phone. When a new device (like your laptop) joins the network, it can query the local network for specific devices, like internet gateways, audio systems, TVs, or printers. Read more on how UPnP compares to Bonjour.
UPnP is poorly standardised, but here's a snippet from the spec about the M-SEARCH frame - the main method for discovery:
When a control point is added to the network, the UPnP discovery protocol allows that control point to search for devices of interest on the network. It does this by multicasting on the reserved address and port (239.255.255.250:1900) a search message with a pattern, or target, equal to a type or identifier for a device or service.
Responses to M-SEARCH frame:
To be found by a network search, a device shall send a unicast UDP response to the source IP address and port that sent the request to the multicast address. Devices respond if the ST header field of the M-SEARCH request is “ssdp:all”, “upnp:rootdevice”, “uuid:” followed by a UUID that exactly matches the one advertised by the device, or if the M-SEARCH request matches a device type or service type supported by the device.
This works in practice. For example, my Chrome browser regularly asks for a Smart TV I guess:
$ sudo tcpdump -ni eth0 udp and port 1900 -A
IP 192.168.1.124.53044 > 239.255.255.250.1900: UDP, length 175
M-SEARCH * HTTP/1.1
HOST: 239.255.255.250:1900
MAN: "ssdp:discover"
MX: 1
ST: urn:dial-multiscreen-org:service:dial:1
USER-AGENT: Google Chrome/58.0.3029.110 WindowsThis frame is sent to a multicast IP address. Other devices listening on that address and supporting this specific ST (search-target) multiscreen type are supposed to answer.
Apart from queries for specific device types, there are two "generic" ST query types:
upnp:rootdevice: search for root devicesssdp:all: search for all UPnP devices and services
To emulate these queries you can run this python script (based on this work):
#!/usr/bin/env python2
import socket
import sys
dst = "239.255.255.250"
if len(sys.argv) > 1:
dst = sys.argv[1]
st = "upnp:rootdevice"
if len(sys.argv) > 2:
st = sys.argv[2]
msg = [
'M-SEARCH * HTTP/1.1',
'Host:239.255.255.250:1900',
'ST:%s' % (st,),
'Man:"ssdp:discover"',
'MX:1',
'']
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
s.settimeout(10)
s.sendto('\r\n'.join(msg), (dst, 1900) )
while True:
try:
data, addr = s.recvfrom(32*1024)
except socket.timeout:
break
print "[+] %s\n%s" % (addr, data)On my home network two devices show up:
$ python ssdp-query.py
[+] ('192.168.1.71', 1026)
HTTP/1.1 200 OK
CACHE-CONTROL: max-age = 60
EXT:
LOCATION: http://192.168.1.71:5200/Printer.xml
SERVER: Network Printer Server UPnP/1.0 OS 1.29.00.44 06-17-2009
ST: upnp:rootdevice
USN: uuid:Samsung-Printer-1_0-mrgutenberg::upnp:rootdevice
[+] ('192.168.1.70', 36319)
HTTP/1.1 200 OK
Location: http://192.168.1.70:49154/MediaRenderer/desc.xml
Cache-Control: max-age=1800
Content-Length: 0
Server: Linux/3.2 UPnP/1.0 Network_Module/1.0 (RX-S601D)
EXT:
ST: upnp:rootdevice
USN: uuid:9ab0c000-f668-11de-9976-000adedd7411::upnp:rootdeviceThe firewall
Now that we understand the basics of SSDP, understanding the reflection attack should be easy. You see, there are in fact two ways of delivering the M-SEARCH frame:
what we presented, over the multicast address
directly to a UPnP/SSDP enabled host on a normal unicast address
The latter method works. We can specifically target my printer IP address:
$ python ssdp-query.py 192.168.1.71
[+] ('192.168.1.71', 1026)
HTTP/1.1 200 OK
CACHE-CONTROL: max-age = 60
EXT:
LOCATION: http://192.168.1.71:5200/Printer.xml
SERVER: Network Printer Server UPnP/1.0 OS 1.29.00.44 06-17-2009
ST: upnp:rootdevice
USN: uuid:Samsung-Printer-1_0-mrgutenberg::upnp:rootdeviceNow the problem is easily seen: the SSDP protocol does not check whether the querying party is in the same network as the device. It will happily respond to an M-SEARCH delivered over the public Internet. All it takes is a tiny misconfiguration in a firewall - port 1900 UDP open to the world - and a perfect target for UDP amplification will be available.
Given a misconfigured target our script will happily work over the internet:
$ python ssdp-query.py 100.42.x.x
[+] ('100.42.x.x', 1900)
HTTP/1.1 200 OK
CACHE-CONTROL: max-age=120
ST: upnp:rootdevice
USN: uuid:3e55ade9-c344-4baa-841b-826bda77dcb2::upnp:rootdevice
EXT:
SERVER: TBS/R2 UPnP/1.0 MiniUPnPd/1.2
LOCATION: http://192.168.2.1:40464/rootDesc.xmlThe amplification
The real damage is done by the ssdp:all ST type though. These responses are much larger:
$ python ssdp-query.py 100.42.x.x ssdp:all
[+] ('100.42.x.x', 1900)
HTTP/1.1 200 OK
CACHE-CONTROL: max-age=120
ST: upnp:rootdevice
USN: uuid:3e55ade9-c344-4baa-841b-826bda77dcb2::upnp:rootdevice
EXT:
SERVER: TBS/R2 UPnP/1.0 MiniUPnPd/1.2
LOCATION: http://192.168.2.1:40464/rootDesc.xml
[+] ('100.42.x.x', 1900)
HTTP/1.1 200 OK
CACHE-CONTROL: max-age=120
ST: urn:schemas-upnp-org:device:InternetGatewayDevice:1
USN: uuid:3e55ade9-c344-4baa-841b-826bda77dcb2::urn:schemas-upnp-org:device:InternetGatewayDevice:1
EXT:
SERVER: TBS/R2 UPnP/1.0 MiniUPnPd/1.2
LOCATION: http://192.168.2.1:40464/rootDesc.xml
... 6 more response packets....In this particular case, a single SSDP M-SEARCH packet triggered 8 response packets. tcpdump view:
$ sudo tcpdump -ni en7 host 100.42.x.x -ttttt
00:00:00.000000 IP 192.168.1.200.61794 > 100.42.x.x.1900: UDP, length 88
00:00:00.197481 IP 100.42.x.x.1900 > 192.168.1.200.61794: UDP, length 227
00:00:00.199634 IP 100.42.x.x.1900 > 192.168.1.200.61794: UDP, length 299
00:00:00.202938 IP 100.42.x.x.1900 > 192.168.1.200.61794: UDP, length 295
00:00:00.208425 IP 100.42.x.x.1900 > 192.168.1.200.61794: UDP, length 275
00:00:00.209496 IP 100.42.x.x.1900 > 192.168.1.200.61794: UDP, length 307
00:00:00.212795 IP 100.42.x.x.1900 > 192.168.1.200.61794: UDP, length 289
00:00:00.215522 IP 100.42.x.x.1900 > 192.168.1.200.61794: UDP, length 291
00:00:00.219190 IP 100.42.x.x.1900 > 192.168.1.200.61794: UDP, length 291That target exposes 8x packet count amplification and 26x bandwidth amplification. Sadly, this is typical for SSDP.
IP Spoofing
The final step for the attack is to fool the vulnerable servers to flood the target IP - not the attacker. For that the attacker needs to spoof the source IP address on their queries.
We probed the reflector IPs used in the shown 100 Gbps+ attack. We found that out of the 920k reflector IPs, only 350k (38%) still respond to SSDP probes.
Out of the reflectors that responded, each sent on average 7 packets:
$ cat results-first-run.txt|cut -f 1|sort|uniq -c|sed -s 's#^ \+##g'|cut -d " " -f 1| ~/mmhistogram -t "Response packets per IP" -p
Response packets per IP min:1.00 avg:6.99 med=8.00 max:186.00 dev:4.44 count:350337
Response packets per IP:
value |-------------------------------------------------- count
0 | ****************************** 23.29%
1 | **** 3.30%
2 | ** 2.29%
4 |************************************************** 38.73%
8 | ************************************** 29.51%
16 | *** 2.88%
32 | 0.01%
64 | 0.00%
128 | 0.00%The response packets had 321 bytes (+/- 29 bytes) on average. Our request packets had 110 bytes.
According to our measurements with the ssdp:all M-SEARCH attacker would be able to achieve:
7x packet number amplification
20x bandwidth amplification
We can estimate the 43 Mpps/112 Gbps attack was generated with roughly:
6.1 Mpps of spoofing capacity
5.6 Gbps of spoofed bandwidth
In other words: a single well connected 10 Gbps server able to perform IP spoofing can deliver a significant SSDP attack.
More on the SSDP servers
Since we probed the vulnerable SSDP servers, here are the most common Server header values we received:
104833 Linux/2.4.22-1.2115.nptl UPnP/1.0 miniupnpd/1.0
77329 System/1.0 UPnP/1.0 IGD/1.0
66639 TBS/R2 UPnP/1.0 MiniUPnPd/1.2
12863 Ubuntu/7.10 UPnP/1.0 miniupnpd/1.0
11544 ASUSTeK UPnP/1.0 MiniUPnPd/1.4
10827 miniupnpd/1.0 UPnP/1.0
8070 Linux UPnP/1.0 Huawei-ATP-IGD
7941 TBS/R2 UPnP/1.0 MiniUPnPd/1.4
7546 Net-OS 5.xx UPnP/1.0
6043 LINUX-2.6 UPnP/1.0 MiniUPnPd/1.5
5482 Ubuntu/lucid UPnP/1.0 MiniUPnPd/1.4
4720 AirTies/ASP 1.0 UPnP/1.0 miniupnpd/1.0
4667 Linux/2.6.30.9, UPnP/1.0, Portable SDK for UPnP devices/1.6.6
3334 Fedora/10 UPnP/1.0 MiniUPnPd/1.4
2814 1.0
2044 miniupnpd/1.5 UPnP/1.0
1330 1
1325 Linux/2.6.21.5, UPnP/1.0, Portable SDK for UPnP devices/1.6.6
843 Allegro-Software-RomUpnp/4.07 UPnP/1.0 IGD/1.00
776 Upnp/1.0 UPnP/1.0 IGD/1.00
675 Unspecified, UPnP/1.0, Unspecified
648 WNR2000v5 UPnP/1.0 miniupnpd/1.0
562 MIPS LINUX/2.4 UPnP/1.0 miniupnpd/1.0
518 Fedora/8 UPnP/1.0 miniupnpd/1.0
372 Tenda UPnP/1.0 miniupnpd/1.0
346 Ubuntu/10.10 UPnP/1.0 miniupnpd/1.0
330 MF60/1.0 UPnP/1.0 miniupnpd/1.0
...The most common ST header values we saw:
298497 upnp:rootdevice
158442 urn:schemas-upnp-org:device:InternetGatewayDevice:1
151642 urn:schemas-upnp-org:device:WANDevice:1
148593 urn:schemas-upnp-org:device:WANConnectionDevice:1
147461 urn:schemas-upnp-org:service:WANCommonInterfaceConfig:1
146970 urn:schemas-upnp-org:service:WANIPConnection:1
145602 urn:schemas-upnp-org:service:Layer3Forwarding:1
113453 urn:schemas-upnp-org:service:WANPPPConnection:1
100961 urn:schemas-upnp-org:device:InternetGatewayDevice:
100180 urn:schemas-upnp-org:device:WANDevice:
99017 urn:schemas-upnp-org:service:WANCommonInterfaceConfig:
98112 urn:schemas-upnp-org:device:WANConnectionDevice:
97246 urn:schemas-upnp-org:service:WANPPPConnection:
96259 urn:schemas-upnp-org:service:WANIPConnection:
93987 urn:schemas-upnp-org:service:Layer3Forwarding:
91108 urn:schemas-wifialliance-org:device:WFADevice:
90818 urn:schemas-wifialliance-org:service:WFAWLANConfig:
35511 uuid:IGD{8c80f73f-4ba0-45fa-835d-042505d052be}000000000000
9822 urn:schemas-upnp-org:service:WANEthernetLinkConfig:1
7737 uuid:WAN{84807575-251b-4c02-954b-e8e2ba7216a9}000000000000
6063 urn:schemas-microsoft-com:service:OSInfo:1
...The vulnerable IPs are seem to be mostly unprotected home routers.
Open SSDP is a vulnerability
It's not a novelty that allowing UDP port 1900 traffic from the Internet to your home printer or such is not a good idea. This problem has been known since at least January 2013:
Authors of SSDP clearly didn't give any thought to UDP amplification potential. There are a number of obvious recommendations about future use of SSDP protocol:
The authors of SSDP should answer if there is any real world use of unicast
M-SEARCHqueries. From what I understandM-SEARCHonly makes practical sense as a multicast query in local area network.Unicast
M-SEARCHsupport should be either deprecated or at least rate limited, in similar way to DNS Response Rate Limit techniques.M-SEARCHresponses should be only delivered to local network. Responses routed over the network make little sense and open described vulnerability.
In the meantime we recommend:
Network administrators should ensure inbound UDP port 1900 is blocked on firewall.
Internet service providers should never allow IP spoofing to be performed on their network. IP spoofing is the true root cause of the issue. See the infamous BCP38.
Internet service providers should allow their customers to use BGP flowspec to rate limit inbound UDP source port 1900 traffic, to relieve congestion during large SSDP attacks.
Internet providers should internally collect netflow protocol samples. The netflow is needed to identify the true source of the attack. With netflow it's trivial to answer questions like: "Which of my customers sent 6.4Mpps of traffic to port 1900?". Due to privacy concerns we recommend collecting netflow samples with largest possible sampling value: 1 in 64k packets. This will be sufficient to track DDoS attacks while preserving decent privacy of single customer connections.
Developers should not roll out their own UDP protocols without careful consideration of UDP amplification problems. UPnP should be properly standardized and scrutinized.
End users are encouraged to use the script scan their network for UPnP enabled devices. Consider if these devices should be allowed to access to the internet.
Furthermore, we prepared on online checking website. Click if you want to know if your public IP address has a vulnerable SSDP service:
Sadly, the most unprotected routers we saw in the described attack were from China, Russia and Argentina, places not historically known for the most agile internet service providers.
Summary
Cloudflare customers are fully protected from SSDP and other L3 amplification attacks. These attacks are nicely deflected by Cloudflare anycast infrastructure and require no special action. Unfortunately the raising of SSDP attack sizes might be a tough problem for other Internet citizens. We should encourage our ISPs to stop IP spoofing within their network, support BGP flowspec and configure in netflow collection.
This article is a joint work of Marek Majkowski and Ben Cartwright-Cox.
Dealing with large attacks sounds like fun? Join our world famous DDoS team in London, Austin, San Francisco and our elite office in Warsaw, Poland.