We make no secret about how passionate we are about building a world-class global network to deliver the best possible experience for our customers. This means an unwavering and continual dedication to always improving the breadth (number of cities) and depth (number of interconnects) of our network.

This is why we are pleased to announce that Cloudflare is now connected to over 12,000 Internet networks in over 300 cities around the world!

The Cloudflare global network runs every service in every data center so your users have a consistent experience everywhere—whether you are in Reykjavík, Guam or in the vicinity of any of the 300 cities where Cloudflare lives. This means all customer traffic is processed at the data center closest to its source, with no backhauling or performance tradeoffs.

Having Cloudflare’s network present in hundreds of cities globally is critical to providing new and more convenient ways to serve our customers and their customers. However, the breadth of our infrastructure network provides other critical purposes. Let’s take a closer look at the reasons we build and the real world impact we’ve seen to customer experience:

Our network allows us to sit approximately 50 ms from 95% of the Internet-connected population globally. Nevertheless, we are constantly reviewing network performance metrics and working with local regional Internet service providers to ensure we focus on growing underserved markets where we can add value and improve performance. So far, in 2023 we’ve already added 12 new cities to bring our network to over 300 cities spanning 122 unique countries!

In May, we activated a new data center in Campos dos Goytacazes, Brazil, where we interconnected with a regional network provider, serving 100+ local ISPs. While it's not too far from Rio de Janeiro (270km) it still cut our 50th and 75th percentile latency measured from the TCP handshake between Cloudflare's servers and the user's device in half and provided a noticeable performance improvement!

A larger number of local interconnections facilitates direct connections between network providers, content delivery networks, and regional Internet Service Providers. These interconnections enable faster and more efficient data exchange, content delivery, and collaboration between networks.

Currently there are approximately 74,000¹ AS numbers routed globally. An Autonomous System (AS) number is a unique number allocated per ISP, enterprise, cloud, or similar network that maintains Internet routing capabilities using BGP. Of these approximate 74,000 ASNs, 43,000² of them are stub ASNs, or only connected to one other network. These are often enterprise, or internal use ASNs, that only connect to their own ISP or internal network, but not with other networks.

It’s mind blowing to consider that Cloudflare is directly connected to 12,372 unique Internet networks, or approximately 1/3rd of the possible networks to connect globally! This direct connectivity builds resilience and enables performance, making sure there are multiple places to connect between networks, ISPs, and enterprises, but also making sure those connections are as fast as possible.

A previous example of this was shown as we started connecting more locally. As seen in this blog post the local connections even increased how much our network was being used: better performance drives further usage!

At Cloudflare we ensure that infrastructure expansion strategically aligns to building in markets where we can interconnect deeper, because increasing our network breadth is only as valuable as the number of local interconnections that it enables. For example, we recently connected to a local ISP (representing a new ASN connection) in Pakistan, where the 50th percentile improved from ~90ms to 5ms!

Network expansion may be driven by reducing latency and improving interconnections, but it’s equally valuable to our existing network infrastructure. Increasing our geographic reach strengthens our redundancy, localizes failover and helps further distribute compute workload resulting in more effective capacity management. This improved resilience reduces the risk of service disruptions and ensures network availability even in the event of hardware failures, natural disasters, or other unforeseen circumstances. It enhances reliability and prevents single points of failure in the network architecture.

Ultimately, our commitment to strategically expanding the breadth and depth of our network delivers improved latency, stronger interconnections and a more resilient architecture - all critical components of a better Internet! If you’re a network operator, and are interested in how, together, we can deliver an improved user experience, we’re here to help! Please check out our Edge Partner Program and let’s get connected.

........

¹CIDR Report

²Origin ASs announced via a single AS path

As home to over 200 million Internet users and the fourth-largest population in the world, Indonesians depend on fast and reliable Internet, but this has always been a challenging part of the world for Internet infrastructure. This has real world implications on performance and reliability (IP transit is on average 6x more expensive than our major South East Asian interconnection markets). That said, first we wanted to share what makes things challenging in Indonesia; geography, infrastructure, and market dynamics.

Geography: The Internet backbone for many countries is almost entirely delivered by terrestrial fiber optic cables, where connectivity is more affordable and easier to build when the land mass is contiguous and there is a concentrated population distribution. However, Indonesia is a collection of over 18,000 islands, spanning three time zones, and approximately 3,200 miles (5,100 km) east to west. By comparison, the United States is 2,800 miles (4,500 km) east to west. While parts of Indonesia are geographically close to Singapore (the regional Internet hub with over 60% of the region's data centers) given how large Indonesia is, much of it is far away.

Infrastructure: Indonesia is a large country and to connect it to the rest of the Internet it currently relies on submarine fiber optic cables. There are a total of 22 separate submarine cables connecting Indonesia to Singapore, Malaysia, Australia and onward. Many of the cable systems cross the Strait of Malacca, a narrow stretch of water, between the Malay Peninsula (Peninsular Malaysia) and the Indonesian island of Sumatra to the southwest, connecting the Indian and Pacific Oceans. This makes reliability challenging as a result of human activities, such as ships dropping their anchors, fishing trawlers, and dredging as it is one of the world's top five busiest shipping lanes. Additionally, Indonesia is geographically located in a very active seismic zone and is very earthquake prone.

There are a number of new submarine cable systems that have come online and four significant builds planned (Apricot, ACC-1, Echo and Nui) that will improve both available capacity and cost economics in the market. Right now the cost is still significantly higher than comparable distances. For example Jakarta to Singapore is approximately 60 times more expensive than a service the same distance would be in the continental US or Europe for a 100Gbps wavelength service. Staying in Asia, a similar distance from Hong Kong to Taiwan costs around 1/6th that of Jakarta to Singapore.

Cyber 1 and Cyber 3 (NTT NexCenter) Data Center Buildings in Jakarta, 2019 (Photo Credit: Tom Paseka).

Picture of Cyber 1 Lobby Directory

While areas like Batam are becoming increasingly popular for data center builds due to its proximity to Singapore, Jakarta is still the most developed and mature market. It has the largest and best interconnected data centers in the country, including the two pictured.

Cloudflare is deployed in the facility on the right (NTT NexCenter), however most ISPs are inside the building on the left (Cyber 1). The two buildings are approximately 30-50 meters apart, yet it's surprisingly difficult to be able to connect between them. One of the reasons why is market fragmentation and how many options are available. In the adjacent picture of the Cyber 1 building lobby directory many of the listings are unique data centers each with different policies and access conditions.

In the past, we’ve talked about the Cost of Bandwidth around the world (and updated here), but we’ve never talked about Indonesia specifically. Using the same methodology as we’ve used in the past, Indonesia's cost is 43x times more expensive than North America or Europe, or even multiples more expensive than other countries in Asia.

Market dynamics: While Indonesia has good and functioning Internet Exchanges, there are a few ISPs who dominate the market. The three largest ISPs in the country (Telkom Indonesia, Indosat Ooredoo Hutchison and XL Axiata) collectively control 80% of the market, while Telkom Indonesia alone has a market share of around 60% by revenue.

This results in Telkom Indonesia having a heavily dominant market share position to leverage resulting in refusal to peer, or exchange Internet traffic in Indonesia without expensive payments, or instead, preferring to connect to other networks outside of Indonesia, introducing latency and diminished performance.

Despite all of these challenges, our network has come a long way since our initial deployment to Jakarta in 2019.

We’ve established:

• A carrier neutral local point of presence at NTT Indonesia Nexcenter Data Center, one of the major interconnection hubs in Jakarta

• An edge partnership point of presence in Yogyakarta with CitranetIX

• Direct interconnections in country with two of the top three networks.

• Peering across three of the larger local internet exchanges, Indonesia Internet Exchange, Jakarta Internet Exchange and Biznet Internet Exchange

• Dedicated 100G wavelength transport back to Singapore

All of this results in a more performant and reliable network for our local customers.

We wanted to see how our network is performing since deployment. We mentioned during Speed Week in 2021 how we benchmark against different networks, and sharing some of those benchmarks here.

At the end of December 2021, Cloudflare was only faster in a few networks, as compared to other providers in Indonesia.

Fast forward twelve months to December 2022, Cloudflare is significantly faster in even more networks.

The TCP protocol is a connection-oriented protocol, which means that a connection is established and maintained until the application programs at each end have finished exchanging messages. The Connect Time summarizes how fast a session can be set up between a client and a server over a network. TTLB (or time to last byte) is the time taken to send the entire response to the web browser. It’s a good measure of how long a complete download takes. Check out our recent blog on Benchmarking Edge Network Performance for more information on how we measure the performance of our network and benchmark ourselves against industry players.

On closer inspection against the three major ISPs specifically, we’re the top provider for two out of the three networks. Cloudflare’s performance has improved year-on-year (16% reduction) and continues to lead (comparative to the other networks) meaning faster and more responsive services for our customers.

Helping build a better Internet for Indonesia doesn’t stop here and there is always more work to be done! We want to be the number one network everywhere and won’t rest until we are. We are continuing to connect to more networks locally, invest in direct submarine cable capacity, as well as further deployments into new data center buildings, Internet Exchanges and new cities too!

Are you operating a network and not yet peering with Cloudflare? Log-in to our Peering Portal or find out more information here for ways to set up peering, or request we deploy nodes into your network directly.