QUIC Version 1 is live on Cloudflare

On May 27 2021, the Internet Engineering Task Force published RFC 9000 - the standardarized version of the QUIC transport protocol. The QUIC Working Group declared themselves done by issuing a Last Call 7 months ago. The i's have been dotted and the t's crossed, RFC 8999 - RFC 9002 are a suite of documents that capture years of engineering design and testing of QUIC. This marks a big occasion.

And today, one day later, we’ve made the standardized version of QUIC available to Cloudflare customers.

Transport protocols have a history of being hard to deploy on the Internet. QUIC overcomes this challenge by basing itself on top of UDP. Compared to TCP, QUIC has security by default, protecting almost all bytes from prying eyes or "helpful" middleboxes that can end up making things worse. It has designed-in features that speed up connection handshakes and mitigate the performance perils that can strike on networks that suffer loss or delays. It is pluggable, providing clear standardised extensions point that will allow smooth, iterative development and deployment of new features or performance enhancements for years to come.

The killer feature of QUIC, however, is that it is deployable in reality. We are excited to announce that QUIC version 1, RFC 9000, is available to all Cloudflare customers.  We started with a limited beta in 2018, we made it general availability in 2019, and we've been tracking new document revisions every step of the way. In that time we've seen User-Agents like browsers join us in this merry march and prove that this thing works on the Internet.

QUIC is just a transport protocol. To make it do anything you need an application protocol to be mapped onto it. In parallel to the QUIC specification, the Working Group has defined an HTTP mapping called HTTP/3. The design is all done, but we're waiting for a few more i's to be crossed before it too is published as an RFC. That doesn't prevent people from testing it though, and for the 3+ years that we've supported QUIC, we have supported HTTP on the top of it.

According to Cloudflare Radar, we're seeing around 12% of Internet traffic using QUIC with HTTP/3 already. We look forward to this increasing now that RFC 9000 is out and raising awareness of the stability of things.

HTTP/3 and QUIC are controlled from the "Network" tab of your dashboard. Turn it on and start testing.

Cloudflare servers sit and listen for QUIC traffic on UDP port 443. Clients send an Initial QUIC packet in a UDP datagram which kicks off the handshake process. The Initial packet contains a version identifier, which the server checks and selects from. The client also provides, via the TLS Application Layer Negotiation Protocol extension, a list of application protocols it speaks. Today Cloudflare supports clients that directly connect to us and attempt to speak QUIC version 1 using the ALPN identifier "h3".

Over the years, as the draft wire format of the protocol has changed, new version and ALPN identifiers have been coined, helping to ensure the client and server pick something they can agree on. RFC 9000 coins the version 0x00000001. Since it's so new, we expect clients to continue sending some of the old ones as it takes time to support. These look like 0xff00001d, 0xff00001c, and 0xff00001b, which mark draft 29, 28, and 27 respectively. Version identifiers are 32-bits, which is conspicuous because a lot of other fields use QUICs variable-length integer encoding (see here).

Before a client can even send an Initial QUIC packet however, it needs to know we're sat here listening for them! In the old days, HTTP relied on the URL to determine which TCP port to speak to. By default, it picked 80 for an http scheme, 443 for an https scheme, or used the value supplied in the authority component e.g. https://example.com:1234. Nobody wanted to change the URL schemes to support QUIC; that would have added tremendous friction to deployment.

While developing QUIC and HTTP/3, the Working Group generally relied on prior knowledge that a server would talk QUIC. And if something went wrong, we'd just ping each other directly on Slack. This kind of model obviously doesn't scale. For widespread real-world deployment we instead rely on HTTP Alternative Services (RFC 7838) to tell TCP-based clients that HTTP/3 is available. This is the method that web browsers will use to determine what protocols to use.When a client makes HTTP requests to a zone that has Cloudflare QUIC and HTTP/3 support enabled, we return an Alt-Svc header that tells it about all the QUIC versions we support. Here's an example:

$ curl -sI https://www.cloudflare.com/ | grep alt-svc
alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400

The entry "h3-29" tells clients that we support HTTP/3 over QUIC draft version 29 on UDP port 443. If they support that they might just send us a QUIC Initial with the single version 0xff00001 and identifier "h3-29". Or to hedge their bets, they might send an initial with all versions that they support. Whichever type of Initial Cloudflare receives, we'll pick the highest version. They also might choose _not_ to use QUIC, for whatever reason they like, in which case they can just carry on as normal.

Previously, you needed to enable experimental support if you wanted to test it out in browsers. But now many of them have QUIC enabled by default and we expect them to start enabling QUIC v1 support soon. So today we've begun rolling out changes to our Alt-Svc advertisements to also include the "h3" identifier and we'll have complete world-wide support early next week. All of these protocol upgrade behaviours are done behind the scenes, hopefully your browsing experiences just appear to get magically faster. If you want to check what's happening, you can, for example, use a browser's network tools - just be sure to enable the Protocol column. Here's how it looks in Firefox:

Cloudflare's QUIC and HTTP/3 support is powered by quiche, our own open-source implementation written in Rust. You can find it on GitHub at github.com/cloudflare/quiche.

Quiche is a Rust library that exposes a C API. We've designed it from day one to be easily integratable into many types of projects. Our edge servers use it, curl uses it, Mozilla uses our qlog sub-crate in neqo (which powers Firefox's QUIC), netty uses it, the list is quite long. We're excited to support this project and grow support for QUIC and HTTP/3 wherever we can. And it won't surprise you to hear that we have built some of our own tools to help us during development. quiche-client is a tool we use to get detailed information on all the nitty gritty details of QUIC connections. It also integrates into the interoperability testing matrix that lets us continually assess interoperability and performance.

You can find quiche-client in the /tools folder of the quiche repository. Here's an example of running it with all trace information turned on, I've highlighted the Initial version and selected ALPN. A corresponding client-{connection ID}.qlog file will be written out.

RUST_LOG=trace QLOG_DIR=$PWD cargo run --manifest-path tools/apps/Cargo.toml --bin quiche-client -- --wire-version 00000001  https://www.cloudflare.com

[2021-05-28T18:59:30.506616991Z INFO  quiche_apps::client] connecting to 104.16.123.96:443 from 192.168.0.50:41238 with scid 5875ecd13154429e5c618eee35b6bbd9ecfe8c6b
[2021-05-28T18:59:30.506842369Z TRACE quiche::tls] 5875ecd13154429e5c618eee35b6bbd9ecfe8c6b write message lvl=Initial len=310
[2021-05-28T18:59:30.506891348Z TRACE quiche] 5875ecd13154429e5c618eee35b6bbd9ecfe8c6b tx pkt Initial version=1 dcid=ed8de2d33a2e830279dfeaae8a7ad674 scid=5875ecd13154429e5c618eee35b6bbd9ecfe8c6b len=330 pn=0
[2021-05-28T18:59:30.506982912Z TRACE quiche] 5875ecd13154429e5c618eee35b6bbd9ecfe8c6b tx frm CRYPTO off=0 len=310
[2021-05-28T18:59:30.507044916Z TRACE quiche::recovery] 5875ecd13154429e5c618eee35b6bbd9ecfe8c6b timer=998.815785ms latest_rtt=0ns srtt=None min_rtt=0ns rttvar=166.5ms loss_time=[None, None, None] loss_probes=[0, 0, 0] cwnd=13500 ssthresh=18446744073709551615 bytes_in_flight=377 app_limited=true congestion_recovery_start_time=None delivered=0 delivered_time=204.765µs recent_delivered_packet_sent_time=206.283µs app_limited_at_pkt=0  pacing_rate=0 last_packet_scheduled_time=Some(Instant { tv_sec: 620877, tv_nsec: 428024928 }) hystart=window_end=None last_round_min_rtt=None current_round_min_rtt=None rtt_sample_count=0 lss_start_time=None  
[2021-05-28T18:59:30.507160963Z TRACE quiche_apps::client] written 1200
[2021-05-28T18:59:30.537123997Z TRACE quiche_apps::client] got 1200 bytes
[2021-05-28T18:59:30.537194566Z TRACE quiche] 5875ecd13154429e5c618eee35b6bbd9ecfe8c6b rx pkt Initial version=1 dcid=5875ecd13154429e5c618eee35b6bbd9ecfe8c6b scid=017022e8618952fd8c7177e863894eb0447b85f4 token= len=117 pn=0
<snip>
[2021-05-28T18:59:30.542581460Z TRACE quiche] 5875ecd13154429e5c618eee35b6bbd9ecfe8c6b connection established: proto=Ok("h3") cipher=Some(AES128_GCM) curve=Some("X25519") sigalg=Some("ecdsa_secp256r1_sha256") resumed=false TransportParams { original_destination_connection_id: Some(ed8de2d33a2e830279dfeaae8a7ad674), max_idle_timeout: 180000, stateless_reset_token: None, max_udp_payload_size: 65527, initial_max_data: 10485760, initial_max_stream_data_bidi_local: 0, initial_max_stream_data_bidi_remote: 1048576, initial_max_stream_data_uni: 1048576, initial_max_streams_bidi: 256, initial_max_streams_uni: 3, ack_delay_exponent: 3, max_ack_delay: 25, disable_active_migration: false, active_conn_id_limit: 2, initial_source_connection_id: Some(017022e8618952fd8c7177e863894eb0447b85f4), retry_source_connection_id: None, max_datagram_frame_size: None }

The road has been long, and we should celebrate the success of the community's efforts over many years to dream big and deliver something.  But as far as we're concerned, we're far from done. We've learned a lot from our early QUIC deployments - a big thank you to everyone in the wider Cloudflare team that supported the Protocols team getting here today. We'll continue to invest that back into our implementation and standardisation activities. Alessandro, Junho, Lohith and I will continue to participate in our respective areas of expertise in the IETF. Speaking for myself, I'll be continuing to co-chair the QUIC Working Group and help guide it through a new chapter focused on maintenance, operations, extensibility and… QUIC version 2. And I'll be moonlighting in other places like the HTTP WG to push Prioritization over the line, and the MASQUE WG to help define how we can use unreliable DATAGRAMS to tunnel almost anything over QUIC and HTTP/3.

If you've made it this far, you are obviously very interested in QUIC. My colleague, Chris Wood, is like that too. He was very excited about the RFCs being shipped. So excited that he sent me this cryptic message:

"QUIC is finally here -- RFC 8999, 9000, 9001, and 9002. Are we and the rest of the Internet ready to turn it on? 0404d3f63f040214574904010a5735!

I have no clue what this means, can you help me out?