Subscribe to receive notifications of new posts:

Automatically generating types for Cloudflare Workers

2021-11-16

2 min read
This post is also available in 简体中文 and 日本語.

Historically, keeping our Rust and TypeScript type repos up to date has been hard. They were manually generated, which means they ran the risk of being inaccurate or out of date. Until recently, the workers-types repository needed to be manually updated whenever the types changed. We also used to add type information for mostly complete browser APIs. This led to confusion when people would try to use browser APIs that aren’t supported by the Workers runtime they would compile but throw errors.

That all changed this summer when Brendan Coll, whilst he was interning with us, built an automated pipeline for generating them. It runs every time we build the Workers runtime, generating types for our TypeScript and Rust repositories. Now everything is up-to-date and accurate.

A quick overview

Every time the Workers runtime code is built, a script runs over the public APIs and generates the Rust and TypeScript types as well as a JSON file containing an intermediate representation of the static types. The types are sent to the appropriate repositories and the JSON file is uploaded as well in case people want to create their own types packages. More on that later.

This means the static types will always be accurate and up to date. It also allows projects running Workers in other, statically-typed languages to generate their own types from our intermediate representation. Here is an example PR from our Cloudflare bot. It’s detected a change in the runtime types and is updating the TypeScript files as well as the intermediate representation.

Using the auto-generated types

To get started, use wrangler to generate a new TypeScript project:

$ wrangler generate my-typescript-worker https://github.com/cloudflare/worker-typescript-template

If you already have a TypeScript project, you can install the latest version of workers-types with:

$ npm install --save-dev @cloudflare/workers-types

And then add @cloudflare/workers-types to your project's tsconfig.json file.

{
  "compilerOptions": {
    "target": "ES2020",
    "module": "CommonJS",
    "lib": ["ES2020"],
    "types": ["@cloudflare/workers-types"]
  }
}

After that, you should get automatic type completion in your IDE of choice.

How it works

Here is some example code from the Workers runtime codebase.

class Blob: public js::Object {
public:
  typedef kj::Array<kj::OneOf<kj::Array<const byte>, kj::String, js::Ref<Blob>>> Bits;
  struct Options {
    js::Optional<kj::String> type;
    JS_STRUCT(type);
  };

  static js::Ref<Blob> constructor(js::Optional<Bits> bits, js::Optional<Options> options);
  
  int getSize();
  js::Ref<Blob> slice(js::Optional<int> start, js::Optional<int> end);

  JS_RESOURCE_TYPE(Blob) {
    JS_READONLY_PROPERTY(size, getSize);
    JS_METHOD(slice);
  }
};

A Python script runs over this code during each build and generates an Abstract Syntax Tree containing information about the function including an identifier, any argument types and any return types.

{
  "name": "Blob",
  "kind": "class",
  "members": [
    {
      "name": "size",
      "type": {
        "name": "integer"
      },
      "readonly": true
    },
    {
      "name": "slice",
      "type": {
        "params": [
          {
            "name": "start",
            "type": {
              "name": "integer",
              "optional": true
            }
          },
          {
            "name": "end",
            "type": {
              "name": "integer",
              "optional": true
            }
          }
        ],
        "returns": {
          "name": "Blob"
        }
      }
    }
  ]
}

Finally, the TypeScript types repositories are automatically sent PRs with the updated types.

declare type BlobBits = (ArrayBuffer | string | Blob)[];

interface BlobOptions {
  type?: string;
}

declare class Blob {
  constructor(bits?: BlobBits, options?: BlobOptions);
  readonly size: number;
  slice(start?: number, end?: number, type?: string): Blob;
}

Overrides

In some cases, TypeScript supports concepts that our C++ runtime does not. Namely, generics and function overloads. In these cases, we override the generated types with partial declarations. For example, DurableObjectStorage makes heavy use of generics for its getter and setter functions.

declare abstract class DurableObjectStorage {
	 get<T = unknown>(key: string, options?: DurableObjectStorageOperationsGetOptions): Promise<T | undefined>;
	 get<T = unknown>(keys: string[], options?: DurableObjectStorageOperationsGetOptions): Promise<Map<string, T>>;
	 
	 list<T = unknown>(options?: DurableObjectStorageOperationsListOptions): Promise<Map<string, T>>;
	 
	 put<T>(key: string, value: T, options?: DurableObjectStorageOperationsPutOptions): Promise<void>;
	 put<T>(entries: Record<string, T>, options?: DurableObjectStorageOperationsPutOptions): Promise<void>;
	 
	 delete(key: string, options?: DurableObjectStorageOperationsPutOptions): Promise<boolean>;
	 delete(keys: string[], options?: DurableObjectStorageOperationsPutOptions): Promise<number>;
	 
	 transaction<T>(closure: (txn: DurableObjectTransaction) => Promise<T>): Promise<T>;
	}

You can also write type overrides using Markdown. Here is an example of overriding types of KVNamespace.

Creating your own types

The JSON IR (intermediate representation) has been open sourced alongside the TypeScript types and can be found in this GitHub repository. We’ve also open sourced the type schema itself, which describes the format of the IR. If you’re interested in generating Workers types for your own language, you can take the IR, which describes the declaration in a "normalized" data structure, and generate types from it.

The declarations inside workers.json contain the elements to derive function signatures and other elements needed for code generation such as identifiers, argument types, return types and error management. A concrete use-case would be to generate external function declarations for a language that compiles to WebAssembly, to import precisely the set of available function calls available from the Workers runtime.

Conclusion

Cloudflare cares deeply about supporting the TypeScript and Rust ecosystems. Brendan created a tool which will ensure the type information for both languages is always up-to-date and accurate. We also are open-sourcing the type information itself in JSON format, so that anyone interested can create type data for any language they’d like!

Watch on Cloudflare TV

Cloudflare's connectivity cloud protects entire corporate networks, helps customers build Internet-scale applications efficiently, accelerates any website or Internet application, wards off DDoS attacks, keeps hackers at bay, and can help you on your journey to Zero Trust.

Visit 1.1.1.1 from any device to get started with our free app that makes your Internet faster and safer.

To learn more about our mission to help build a better Internet, start here. If you're looking for a new career direction, check out our open positions.
Full Stack WeekCloudflare WorkersDevelopersServerlessDeveloper Platform

Follow on X

Brendan Coll|@_mrbbot
Jonathan Kuperman|@jkup
Cloudflare|@cloudflare

Related posts

September 27, 2024 1:00 PM

Our container platform is in production. It has GPUs. Here’s an early look

We’ve been working on something new — a platform for running containers across Cloudflare’s network. We already use it in production, for AI inference and more. Today we want to share an early look at how it’s built, why we built it, and how we use it ourselves. ...

September 27, 2024 1:00 PM

AI Everywhere with the WAF Rule Builder Assistant, Cloudflare Radar AI Insights, and updated AI bot protection

This year for Cloudflare’s birthday, we’ve extended our AI Assistant capabilities to help you build new WAF rules, added new AI bot & crawler traffic insights to Radar, and given customers new AI bot blocking capabilities...