Subscribe to receive notifications of new posts:

Expanding the Cloudflare Workers Observability Ecosystem

2021-04-13

6 min read

One of the themes of Developer Week is “it takes a village”, and observability is one area where that is especially true. Cloudflare Workers lets you quickly write code that is infinitely scalable — no availability regions, no scaling policies. Your code runs in every one of our data centers by default: region Earth, as we like to say. While fast time to market and effortless scale are amazing benefits, seasoned developers know that as soon as your code is in the wild… stuff happens, and you need the tools in place to investigate, diagnose, fix and monitor those issues.

Today we’re delighted to add to our existing analytics partners. We’re announcing new partnerships with six observability-focused companies that are deeply integrated into the Cloudflare Workers ecosystem. We’re confident these partnerships will provide immediate value in building the operational muscle to maintain and make your next generation of applications fast, secure and bullet-proof in production.

console.log(`Got request. Extracted name=${name}. Saving…`);

Cloudflare wrangler gives you the ability to generate, configure, build, preview and publish your projects, from the comfort of your dev environment. Writing code in your favorite IDE with a fully-fledged CLI tool that also allows you to simulate how your code will run on the edge is a delightful developer experience and one I personally love.

If you’re like me, you’ll start out your app with console.log statements. wrangler dev and wrangler tail both make it incredibly easy to get visibility into your code during dev and test, but for robust applications, you need more — much more. Things like correlating client and server side event data, seeing context around issues, version awareness and data visualization are what allows DevOps teams to create truly robust applications and make customers happy. The great news is — it’s easy to go from console.log to a code or systems monitoring solution with our partners Sentry and New Relic.

Sentry enables monitoring application code health. From error tracking to performance monitoring, developers can see issues that really matter, solve them more quickly, and learn continuously about their applications — from frontend to backend.

Toucan-js, courtesy of Cloudflare’s very own Robert Cepa, is a reliable Sentry client for Cloudflare Workers and it’s an open-source npm module. It makes it easy to convert basic logging into full application monitoring. A simple npm install toucan-js and a couple of lines of boilerplate setup allow you convert those console.logs into a streaming source of client-side events that will be rendered for analysis in Sentry. Additionally, the distributed nature of serverless means developers need to think about where and how they can manage state. Toucan-js abstracts that away and allows simple log statements like:

sentry.addBreadcrumb({
    message: "Got request. Extracted name=${name}. Saving…",
    category: "log"
});

to be visualized in Sentry as a user journey with filters, times, versioning and more, allowing you to understand what events led to the errors.

New Relic is a popular observability platform, particularly with Enterprises offering a Telemetry Data Platform, Full-Stack observability and Applied Intelligence. While there isn't (yet!) a specific npm package for NewRelic and Cloudflare Workers, the combination of NewRelic’s https log endpoint and Cloudflare Workers event.waitUntil() means you can very easily instrument your application with NewRelic, without blocking the request and thus not impacting performance.

let url = "https://log-api.newrelic.com/log/v1";
let init = {
	method: "POST", 
	headers: {"content-type":"application/json"}, 
	body: JSON.stringify(payload)
};
event.waitUntil(fetch(url, init));

Like Sentry, those logs and events are then available for analysis in the NewRelicOne platform. Cloudflare uses both Sentry and New Relic for exactly the reasons outlined above, and I’m delighted to welcome them to our Developer Ecosystem as Observability Partners.

New Relic One | New Relic

Monitoring your Cloudflare Workers serverless environment with New Relic lets you deliver serverless apps with confidence by rapidly identifying when something goes wrong and quickly pinpointing the problem—without wading through millions of invocation logs. Monitor, visualize, troubleshoot, and alert on all your Workers apps in a single experience.-- Raj Ramanujam, Vice President, Alliances and Channels, New Relic.

With Cloudflare Workers and Sentry, software teams immediately have the tools and information to solve issues and learn continuously about their code health instead of worrying about systems and resources. We’re thrilled to partner with Cloudflare on building technologies that make it easy for developers to deploy with confidence.-- Elain Szu, Vice President of Marketing, Sentry.

Developers are not the only part of an organization that need to observe all aspects of their applications in production. As organizations grow and the sophistication of their infrastructure monitoring and security systems grow, they typically implement observability platforms, which provide overall visibility into the entire infrastructure and the ability to alert on anomalies — not just individual applications, appliances, hardware or network. To achieve that goal, observability platforms must ingest as much data as possible. Cloudflare already partners with Datadog, Sumo Logic and Splunk — this allows security and operations teams to ingest HTTP logs from the network edge along with origin logs and many other sources of data.

Since that announcement, specific Cloudflare Workers fields such as WorkerCPUTime, WorkerStatus, WorkerSubrequest, and WorkerSubrequestCount have been added to offer out-of-the-box visibility to Cloudflare Workers execution. Of course, since the value of observability platforms is about whole-of-infrastructure visibility, ideally we want not just execution logs, but the application logs from our systems, similar to the examples in the section above.

Fortunately, our partners all offer simple HTTP interfaces into their ingestion engines. Check out Datadog’s HTTP API, Splunk’s REST API and SumoLogic’s HTTP Logs and Metric Source for step-by-step instructions on how to easily ingest your Cloudflare Workers logs. Besides getting on your CISO’s good side, if your organization has a Detection and Response team, they’ll be able to help you ensure your Cloudflare Workers application is integrated and monitored as a first-class citizen in your organization's security apparatus. For example, the screenshot below shows Datadog surfacing a security signal detecting malicious activity in Cloudflare HTTP logs based on threat intel feeds.

Maintaining a strong security posture means ensuring every part of your toolchain is being monitored - from the datacenter/VPC, to your edge network, all the way to your end users. With Datadog’s partnership with Cloudflare, you get edge computing logs alongside the rest of your application stack’s telemetry - giving you an end to end view of your application’s health, performance and security.- Michael Gerstenhaber, Sr. Director, Datadog.

Teams using Cloudflare Workers with Splunk Observability get full-stack visibility and contextualized insights from metrics, traces and logs across all of their infrastructure, applications and user transactions in real-time and at any scale. With Splunk Observability, IT and DevOps teams now have a seamless and analytics-powered workflow across monitoring, troubleshooting, incident response and optimization. We're excited to partner with Cloudflare to help developers and operations teams slice through the complexity of modern applications and ship code more quickly and reliably.- Jeff Lo, Director of Product Marketing, Splunk

Reduce downtime and solve customer-impacting issues faster with an integrated observability platform for all of your Cloudflare data, including its Workers serverless platform. By using Cloudflare Workers with Sumo Logic, customers can seamlessly correlate system issues measured by performance monitoring, gain deep visibility provided by logging, and monitor user experience provided by tracing and transaction analytics.- Abelardo Gonzalez, Director of Product Marketing at Sumo Logic

Honeycomb.io is an observability platform that gives you high level data regarding how your services are performing, combined with the ability to drill down all the way to the individual user level to troubleshoot issues without having to hop across different data types to piece the data together. Traditionally, when debugging production incidents with dashboards and metrics, it is difficult to drill down beyond aggregate measures. For example, a graph with error rates can’t tell you which exact customers are experiencing the most errors. Logs show you the raw error data, but it's hard to see the big picture unless you know exactly where to look.

Honeycomb’s event-based model for application telemetry and powerful query engine make it possible to slice your data across billions of rows and thousands of fields to find hidden patterns. The BubbleUp feature also helps you automatically detect the differences between “good” sets and “bad” sets of events. The ability to quickly get results means teams can resolve incidents faster and figure out where to make system optimizations.

The Honeycomb beta npm module for Cloudflare Workers observability is unique, in that it has first-class knowledge of the concept of sub-requests that are a core part of many Worker applications, and this is surfaced directly in the platform. We can’t wait to see the GA version and more innovation around observability for Cloudflare Workers.

Honeycomb is excited to partner with Cloudflare as they build an ecosystem of tools that support the full lifecycle of delivering successful apps. Writing and deploying code is only part of the equation. Understanding how that code performs and behaves when it is in the hands of users also determines success. Cloudflare and Honeycomb together are shining the light of observability all the way to the edge, which helps technical teams build and operate better apps.- Charity Majors, Honeycomb CTO & cofounder.

Summary

Developers love writing code on Cloudflare Workers. The speed, scale, and developer tooling all combine to make it a delightful experience. Our observability partner announcements today extend that experience from development to operations. Getting real-time, contextual insights into what your code is doing, how it’s performing and any errors it’s generating is at the core of shipping the next generation of transformative apps. Our serverless platform takes care of getting your code right next to your users, and our observability partners make sure that that code does exactly what you designed it to do.

Cloudflare's connectivity cloud protects entire corporate networks, helps customers build Internet-scale applications efficiently, accelerates any website or Internet application, wards off DDoS attacks, keeps hackers at bay, and can help you on your journey to Zero Trust.

Visit 1.1.1.1 from any device to get started with our free app that makes your Internet faster and safer.

To learn more about our mission to help build a better Internet, start here. If you're looking for a new career direction, check out our open positions.
Developer WeekDevelopersCloudflare WorkersPartnersObservabilityServerlessProduct News

Follow on X

Steven Pack|@steven_pack
Erwin van der Koogh|@evanderkoogh
Cloudflare|@cloudflare

Related posts

October 31, 2024 1:00 PM

Moving Baselime from AWS to Cloudflare: simpler architecture, improved performance, over 80% lower cloud costs

Post-acquisition, we migrated Baselime from AWS to the Cloudflare Developer Platform and in the process, we improved query times, simplified data ingestion, and now handle far more events, all while cutting costs. Here’s how we built a modern, high-performing observability platform on Cloudflare’s network....

October 24, 2024 1:05 PM

Build durable applications on Cloudflare Workers: you write the Workflows, we take care of the rest

Cloudflare Workflows is now in open beta! Workflows allows you to build reliable, repeatable, long-lived multi-step applications that can automatically retry, persist state, and scale out. Read on to learn how Workflows works, how we built it on top of Durable Objects, and how you can deploy your first Workflows application....