Announcing Cloudflare’s Database Partners

Cloudflare Workers is the easiest way for developers to deploy their application’s code with performance, scale and security baked in. No configuration necessary. Worker code scales to serve billions of requests close to your users across Cloudflare’s 200+ data centers.

But that’s not the only interesting problem we need to solve. Every application has two parts: code and state.

State isn’t always the easiest to work in a massive distributed system. When an application runs in 200+ data centers simultaneously, there’s an inherent tradeoff between distributing the data for better performance, availability, scale, and guaranteeing that all data centers see the same data at a given point in time.

Our goal is to make state at the edge seamless. We started that journey with Workers KV, which provides low-latency access to globally distributed data. We’re since added Durable Objects, with strong consistency and the ability to design coordination patterns on top of Workers. We’re continuing to invest in and build out these products.

However, some use cases aren’t easily implemented with Workers KV or Durable Objects. Think querying complex datasets, or communicating with an existing system-of-record. Even if we built this functionality ourselves, there will always be customers who want to use a different tool for data storage and access.

Today, we’re making our data strategy clear so that developers can build on Workers with confidence. First, we’re announcing a partnership with two distributed data companies, Macrometa and Fauna, to help developers choose an edge-first database when they build a new application on Workers. Next, we’ve introduced tutorials and guides for databases that already support HTTP connections to make connecting to existing databases even easier.

With this, the Workers runtime is the easiest way to handle both parts of your application that matter: code and state.

Cloudflare is excited to announce a partnership for global data with Macrometa. Macrometa provides a serverless data platform for developers to build rich, geo-distributed, stateful applications on the edge with Cloudflare Workers. The Macrometa Global Data Network (GDN) is an ultra-fast, globally replicated noSQL database with integrated features for search, pub/sub, streams, and stream processing.

Macrometa complements Cloudflare Workers by enabling stateful functions to run, store, query, mutate, and serve data globally with local read/write latencies. Tables replicate across Macrometa’s global data centers with configurable consistency and without additional code, cost, or complexity. Data calls between Workers and Macrometa are automatically routed and served from the closest Macrometa region relative to the Cloudflare Worker invocation. We have seen that end-to-end latency for a database request from a client to the edge and back is <75ms for 99th percentile.

Macrometa combines typically disparate data services and APIs (each with their own data model and copy of data) into a single, integrated, and intuitive data API (with a single, unified common data set and model). At a high level, one can think of Macrometa as combining a noSQL database together with pub/sub, streaming data and event processing along with search into a seamless, globally replicated, low latency data platform.

The following multimodal interfaces are available to the developer via a SQL-like query language:

• Key/value database with high write performance, low latencies, and Time to Live (TTL) support

• JSON document database with a SQL-like query language

• Graph database for storing and retrieving relationships between entities

• DynamoDB API compatible database

• Low latency edge cache for database and API results with global cache invalidation and Time to Live (TTL)

• Search

• Pub/Sub & queues with Kafka, Pulsar and MQTT message compatibility

• Streaming data and complex event processing - evaluate events, aggregate, enrich, filter, summarize, pattern match on stream data in real time.

Apps written using Cloudflare Workers consume Macrometa via a JavaScript SDK or REST API. Developers have a choice of using one of two SDKs tightly integrated with Cloudflare Workers:

• A Cloudflare Worker specific JavaScript SDK which provides the full set of capabilities mentioned above

• The Dynamo client SDK that enables developers to use Macrometa as a drop-in replacement AWS DynamoDB (or as an edge cache).

With Cloudflare and Macrometa, developers can now build and deliver powerful and compelling data-driven experiences in a way that the centralized clouds will never be able to. The world deserves a better cloud - an edge cloud.- Chetan Venkatesh, Co-founder/CEO, Macrometa

If you are interested in learning more about using Macrometa and Cloudflare Workers together, please register here.

To showcase the power and performance of Cloudflare and Macrometa together, we re-built AWS’s demo application — a complete E-Commerce backend that serves an online bookstore. eCommerce backends for large global sites are complex and require many different types of databases. These complex architectures are necessary to store, perform multi-table or multi-data store queries, and serve product catalogs, pricing, search, recommendations, order management, cart operations, fulfillment, etc.

The bookstore is a serverless app built as a Cloudflare Worker to serve a React single page application and uses Macrometa for storing and serving product/catalog data, orders, cart operations, etc. It uses streams and complex event processing to provide a real-time leaderboard of the top-selling books and uses the Macrometa search service to deliver near-instant search results. You can test drive the end-user experience here. Our tests show orders of magnitude latency improvement for a user vs. the same use case built on a legacy cloud provider.

To check out the source code, head over to GitHub.

Cloudflare is excited to announce a new partnership for transactional workloads with Fauna, a data API for modern applications. Together, Cloudflare and Fauna enable developers to build more powerful stateful applications that scale dynamically and maintain strong consistency without any additional configuration.

Fauna is a flexible, developer-friendly, global transactional database delivered as a secure, global, cloud API — no database operations required. With Fauna, developers can simplify code and ship faster by replacing their database infrastructure with a data API that combines the flexibility of relational and document-centric querying with the power of custom business logic, and the ease of GraphQL at the edge.

Fauna is a simple yet scalable backend for applications built on Cloudflare’s services such as Workers and Pages. By leveraging a web-native (HTTP-based) invocation model with support for modern security protocols, Fauna eliminates the connection limits introduced by traditional databases and can be integrated directly with serverless functions and applications running at the edge.

Fauna complements Cloudflare KV and Durable Objects by providing a global, queryable, strongly consistent persistence layer for the mission-critical data required to build modern web and mobile applications. It’s underlying globally distributed storage engine is fast, consistent, and reliable. Developers can rely on Fauna’s unique global transactional capabilities to ensure that mission-critical business data is always strongly consistent with minimal latency.

Fauna, a serverless offering, is easy to get started with for free, and lets developers experience freedom from database operations at any scale. Fauna is available through GraphQL or drivers that work with Workers in several popular programming languages.

If you’re interested in learning how to use Fauna with Cloudflare Workers, please visit this step-by-step tutorial for a hands-on experience! If you are interested in learning more about using Cloudflare Workers and Fauna together, please register here.

The integration of Cloudflare workers with Fauna allows our joint customers to simply and cost effectively bring both data and logic to the edge. Fauna was developed from the ground up as a globally distributed, serverless database and when combined with Cloudflare Workers serverless compute fabric provides developers with a simple, yet powerful abstraction over the complexities of distributed infrastructure.- Evan Weaver, Co-founder/CTO, Fauna

Let’s take a look at a current customer of Fauna and Cloudflare who benefits from Fauna’s ability to offer globally-distributed transactional data access at low latency.

MeetKai is an AI-powered voice assistant that makes a consumer’s life easier through conversation, personalization & curation. Deployed by enterprises with Monthly Active Users in the hundred million plus range, MeetKai uses Fauna and Cloudflare together to deliver a responsive, global experience.

Cloudflare Workers act as the proxy to all downstream requests. Doing so enables MeetKai to perform much smarter load balancing at the edge, using the Worker rather than some other competing solutions. When combined with Fauna, this serverless-first approach delivers scale and performance at predictable costs, while reducing time to market. By pushing data to the edge with Fauna, MeetKai are able to inject personalization information before a request ever hits the backend! Learn more about how MeetKai uses Cloudflare Workers and Fauna in this technical case study.

Combining Cloudflare Workers with Fauna allows us to bring both data and logic to the edge, enabling us to deliver a differentiated search engine while still meeting high operations requirements in latency and uptime -- all at a fraction of the cost and complexity of non-serverless solutions.- James Kaplan, Founder/CEO MeetKai

In addition to our partners, Cloudflare Workers can work with any database that accepts connections over HTTP.

In particular, we’ve documented connections to both DynamoDB and AWS Aurora, supporting either Postgres or MySQL. These connections let you access already existing databases directly from Cloudflare’s edge, making it easy to build new applications on top of existing data.

Connect a Worker to DynamoDB

Connect a Worker to AWS Aurora

In addition, we’ll be adding more tutorials to document how to connect to other databases that work with Workers, like Google’s Cloud Firestore.

The Workers platform has built a new paradigm for developers for serverless. Across our own Durable Objects and Workers KV, our partner databases, and connections to existing providers, we’re bringing that same paradigm shift to state at the edge. As we continue to build out our database partnerships and data platform, we want to hear from you! Send your ideas and feedback to the Workers Product team to let us know what databases and features you’d most like to see us build.