The Cloudflare Blog

Edge-Side-Includes with Cloudflare Workers

Stephane Nouvellon — Mon, 20 Aug 2018 08:00:00 GMT

At Cloudflare we’re accelerating web assets in a number of different ways. Part of this is caching, by storing the response given by the origin server directly within our 151+ global data centers. This will dramatically improve the delivery of the resources as the visitor will directly get them from the data center closest to them, instead of waiting for us to fetch the request from the origin web server.

The issue with dynamic (but not a lot) pages

The subject we’re gonna cover today is the concept of Edge-Side-Includes. And what’s better than a real use-case to introduce what it is used for? Let’s take a website where all pages are including advertisements at the head and bottom. Could we consider these pages static? We couldn’t as at least part of this page is dynamic. Could we consider caching it? That’s a no again as it would mean the first dynamic part rendered will be cached and served for the other visitors trying to get the page. It would be a catastrophe if the advertisements are user-specific.

So the issue here is that we can’t cache the page. That’s quite a shame as it means that we’ll fetch the page again, and over again for every new request just to get this 1% portion of dynamic content.

Counter-measure with delta-compression?

Back in time, we’ve released Railgun, which consist in doing delta-compression of the requests received by a web server so Railgun listener could just send the delta bytes since the last request. We’re also working on the inclusion of this delta-compression in our Argo Tunnel listener, which is a small agent opening tunnels to us so you don’t even have to have your applications public on the internet, a simple outbound HTTPS access is enough to publish, secure and accelerate applications on Internet.

In both cases, we’ll need to fetch the complete webpage in order to calculate the difference from the last request, right? This is where Edge-Side-Includes takes place.

The Edge-Side-Includes standard has been submitted to the W3C (https://www.w3.org/TR/esi-lang) in August 2001 and defines an XML markup language that can be inserted in HTML or other text based contents which defines how interstitial proxies/CDNs need to combine static and dynamic portion and were to get them. The result is that it’s possible to keep those 99% in cache and for the remaining 1%, the interstitial cache proxy will fetch directly from the destination defined in the Edge-Side-Include block for finally combining both static and dynamic parts and sending the final webpage to the visitor.

ESI block example:

Implementing it in a Worker

We released months ago Cloudflare Workers, our serverless framework which helps to implement custom logics directly within the Edge. The Workers are triggering in the path of the requests and the responses and can manipulate almost everything and spin subrequests on-the-fly.This could dramatically improve the time to action for the implementation of new logics on your applications since you won’t have to modify them anymore as this can directly be done in the Edge, even if your applications are hosted on a bunch of different locations (Cloud and on-premise).

This scenario sounds then quite compatible with what we can achieve with Cloudflare Workers. For memories, here are the actions the EDGE needs to do for implementing the ESIs:

Fetching the static content
Searching in the payload for any blocks
Fetching separately every ESI blocks found
Merging the static and dynamic contents
Sending the whole new payload to visitors

For this purpose, I created a small page on my test web server, with the following content:





ESI Demonstration




Cloudflare Edge-Side-Include demonstration
How to ESI with Cloudflare
ESI (or Edge-Side-Include consist in fetching a globally static page with some dynamic fragment, this can be done directly from our EDGE with the workers, where the HTML page would be like that one:
Dynamic portion coming from https://httpbin.org/headers:

So in this page we can see that most of the content will be static and we can find just before the end of the an ESI pointing to https://httpbin.org/headers that the Edge will need to combine with the static content, in place of the ESI block.

The worker explained

addEventListener("fetch", event => {
  event.respondWith(fetchAndStream(event.request))
  event.passThroughOnException()
})

async function fetchAndStream(request) {
  let response = await fetch(request)
  let contentType = response.headers.get('content-type')

  if (!contentType || !contentType.startsWith("text/")) {
    return response
  }
  let { readable, writable } = new TransformStream()
  let newResponse = new Response(readable, response)
  newResponse.headers.set('cache-control', 'max-age=0')
  streamTransformBody(response.body, writable)
  return newResponse
}

This is the main portion of the script, testing first if the response is text-based and return the content directly to the visitor if not. Then instanciating a stream pipeline to send to the visitor using a specific function called streamTransformBody() in charge of the payload chunking.

You'll notice that I'm setting a max-age=0 cache-control header in the response as I don't want browsers to cache this response in case of a bad configuration on the Origin side.

I’m also declaring this script as fail-safe so that the request will go through normally if raising an Exception. More debugging information can be found on our developer HUB. Feel free to adapt for your use-cases and for example sending the Exception to a specific header, or even fancier sending the event to a logging tool.

async function handleTemplate(encoder, templateKey) {
  const linkRegex = /(esi:include.*src="(.*?)".*\/)/gm
  let result = linkRegex.exec(templateKey);
  let esi
  if (!result) {
    return encoder.encode(`<${templateKey}>`);
  }
  if (result[2]) {
    esi = await subRequests(result[2]);
  }
  return encoder.encode(
    `${esi}`
  );
}

In this portion, we’re receiving the chunks and searching for ESI blocks and if found, getting the dynamic parts. You can see that when the regex is matching, we’re calling a subRequest() function. This function will fetch the content of the ESI destination and return + encode the received payload.

async function subRequests(target){
  const init = {
            method: 'GET',
            headers: {
                'user-agent': 'cloudflare'
            }
        }
  let response = await fetch(target, init)
  let text = await response.text()
  
  return text
}

This portion is quite simple as it defines the function that does the subrequest to the ESI destination, pay attention to what the Origin server is serving as cache-control response header for those dynamic parts as we want to keep them dynamic and not cached by Cloudflare. It’s also possible to override the TTL via Cloudflare directly if you don’t want to modify your application. The documentation about how to manipulate Cloudflare features is available on the developers HUB

async function streamTransformBody(readable, writable) {
  const startTag = "<".charCodeAt(0);
  const endTag = ">".charCodeAt(0);
  let reader = readable.getReader();
  let writer = writable.getWriter();

  let templateChunks = null;
  while (true) {
    let { done, value } = await reader.read();
    if (done) break;
    while (value.byteLength > 0) {
      if (templateChunks) {
        let end = value.indexOf(endTag);
        if (end === -1) {
          templateChunks.push(value);
          break;
        } else {
          templateChunks.push(value.subarray(0, end));
          await writer.write(await translate(templateChunks));
          templateChunks = null;
          value = value.subarray(end + 1);
        }
      }
      let start = value.indexOf(startTag);
      if (start === -1) {
        await writer.write(value);
        break;
      } else {
        await writer.write(value.subarray(0, start));
        value = value.subarray(start + 1);
        templateChunks = [];
      }
    }
  }
  await writer.close();
}

In the streamTransformBody() function, I’m chunking the payload received in the readable object with specific boundaries in order to avoid chunks to terminate in the middle of a line or worse, in the middle of an ESI block.

async function translate(chunks) {
  const decoder = new TextDecoder();

  let templateKey = chunks.reduce(
    (accumulator, chunk) =>
      accumulator + decoder.decode(chunk, { stream: true }),
    ""
  );
  templateKey += decoder.decode();

  return handleTemplate(new TextEncoder(), templateKey);
}

The translate() function decode the chunks and send the stringified payload to the handleTemplate() function, which is for memories intended to properly replace the ESI by the dynamic pieces.

The complete worker script

addEventListener("fetch", event => {
  event.respondWith(fetchAndStream(event.request))
  event.passThroughOnException()
})

async function fetchAndStream(request) {
  let response = await fetch(request)
  let contentType = response.headers.get('content-type')

  if (!contentType || !contentType.startsWith("text/")) {
    return response
  }
  let { readable, writable } = new TransformStream()
  let newResponse = new Response(readable, response)
  newResponse.headers.set('cache-control', 'max-age=0')
  streamTransformBody(response.body, writable)
  return newResponse
}

async function handleTemplate(encoder, templateKey) {
  const linkRegex = /(esi:include.*src="(.*?)".*\/)/gm
  let result = linkRegex.exec(templateKey);
  let esi
  if (!result) {
    return encoder.encode(`<${templateKey}>`);
  }
  if (result[2]) {
    esi = await subRequests(result[2]);
  }
  return encoder.encode(
    `${esi}`
  );
}

async function subRequests(target){
  const init = {
            method: 'GET',
            headers: {
                'user-agent': 'cloudflare'
            }
        }
  let response = await fetch(target, init)
  let text = await response.text()
  
  return text
}

async function streamTransformBody(readable, writable) {
  const startTag = "<".charCodeAt(0);
  const endTag = ">".charCodeAt(0);
  let reader = readable.getReader();
  let writer = writable.getWriter();

  let templateChunks = null;
  while (true) {
    let { done, value } = await reader.read();
    if (done) break;
    while (value.byteLength > 0) {
      if (templateChunks) {
        let end = value.indexOf(endTag);
        if (end === -1) {
          templateChunks.push(value);
          break;
        } else {
          templateChunks.push(value.subarray(0, end));
          await writer.write(await translate(templateChunks));
          templateChunks = null;
          value = value.subarray(end + 1);
        }
      }
      let start = value.indexOf(startTag);
      if (start === -1) {
        await writer.write(value);
        break;
      } else {
        await writer.write(value.subarray(0, start));
        value = value.subarray(start + 1);
        templateChunks = [];
      }
    }
  }
  await writer.close();
}

async function translate(chunks) {
  const decoder = new TextDecoder();

  let templateKey = chunks.reduce(
    (accumulator, chunk) =>
      accumulator + decoder.decode(chunk, { stream: true }),
    ""
  );
  templateKey += decoder.decode();

  return handleTemplate(new TextEncoder(), templateKey);
}

Testing the script

Testing is easy, we’re going to cURL over the URL having my small HTML code presented earlier in the article, and see what looks like the answer.

cURL command:

curl https://www.justalittlebyte.ovh/esi.html -sv 2>&1 | grep -E "Cf-Ray|cf-cache-status|cf-ray"

The grep is trying to catch few things:

First CF-Ray is for the request itself
CF-Cache-Status is the header Cloudflare is using indicating if the requests has been fetched from the cache
Second CF-Ray is the CF-Ray coming from the call made to https://httpbin.org/headers, this will change as the CF-Ray is unique for any new requests made through our Edge

The answer is that one

< cf-cache-status: HIT 
< cf-ray: 447b3a379ce8bc3e-LHR
    "Cf-Ray": "447b3a37d217bc3e-IAD",

Doing it again

< cf-cache-status: HIT
< cf-ray: 447b3a3c4b5fbc44-LHR
    "Cf-Ray": "447b3a3c61f5bc44-IAD",

and the page looks like this

The most interesting comes from the cf-cache-status where we can see that the page is actually coming from our cache but the worker modified the payload as he detected an ESI block.

Share your worker recipes

You can find additional worker recipes and examples in our official documentation

Have you written a worker that you'd like to share? Send it to us and you might get featured on our blog or added to our Cloudflare worker recipe collection with a credit.

1 year and 3 months working at Cloudflare: How is it going so far?

Stephane Nouvellon — Thu, 19 Apr 2018 17:21:04 GMT

This post is inspired by a very good blog post from one of my colleagues in the US, which I really appreciated as I was a newcomer to the company. It was great to see what it is like working for Cloudflare after one year and to learn from the lessons she had learnt.

I'll try to do the same in three parts. Beginning with how my on-boarding went, my first customer experiences and finally what is my day-to-day life at Cloudflare. These writings only reflect my personal feelings and thoughts. The experience is different for each and every newcomer to Cloudflare.

Chapter 1 - On-boarding, being impressed and filling the (big) knowledge gaps.

Before I joined Cloudflare, I was working as a Security Consultant in Paris, France. I never had the opportunity to move abroad to speak English (me.englishLevel = 0), I never had any reason to live outside of France and was at the same time looking for another Job. Perfect then!

When I saw the job posting, I immediately applied as I knew the company well, the mindset and the products Cloudflare provided. It took me 6 months to get the offer probably because I was abroad and the French-speaking team was still under construction, to be honest, I would have given it a year if it was needed.At Cloudflare, every new Solutions Engineer is sent to San Francisco for about a month to get a proper onboarding.

This has primarily served three purposes:

Meet people,
Understand the Sales pitch
Be technically prepared to face the customers!

I was optimistic in meeting the criteria with four weeks of training in SFO. However, I quickly changed my mind after the first hour! Thinking "This is really tough." I literally had to learn two languages, English and Cloudflarian.

Post on linkedin I shared as I was so excited to start, that shows how hyped I was

For learning English, I decided to postpone as I had so many things to learn, I was meeting with (impressive) people talking about a product as if I never heard about it before. For me, Cloudflare was a plug-and-play product implementing complex things in a way that kids could setup and understand. Digging deeper I discovered a monster. Not just a simple, well-crafted feature set that can be toggled on/off. but at the same time, a very well furnished product where being a master of a specific piece can take an age to learn!

Things I learnt:

People at Cloudflare are impressive. They are at the same time smart, humble, knowledgeable and happy to share/help!
Break, reverse-engineer, test, re-break the product with your test zone, that's the best way to not presume but to understand how it works.
Ask questions as many as you can. If you're thinking about asking the question it means that's not clear to you so ASK!
Cloudflare is a transparent company, use this to your advantage to learn by yourself! We've got access to every single line of code of the product, if you're asking yourself how something works, just dig into the code or ask someone to point you to the correct portion of the codebase.
The internal WIKI is your new god!

Chapter 2 - Come back to London and First customer experience

After the 4 weeks, I was almost dead and my head felt like it had gained kilos not from the SF food but with knowledge, I've gone back to my new home, the UK! Remembering that my girlfriend and I had left the apartment before unpacking the boxes, the joy!

After the weekend, the big day had arrived! My first day at Cloudflare London, I met the team which I was already quite familiar with given the number of interviews I had during my hiring process. They hadn't changed, they were always so friendly and I felt at home very quickly.

English level at the time: 0 + 4 weeks in SFO

My first customer meeting

When I came back, I was literally thrown on to a call with a customer. Shadowed by a fellow colleague. I was excited and terrified at the same time. I discovered that they were Irish with a very difficult accent. I was not able to understand any words they were saying. Looking at my SE shadow with the MAYDAY eyes (please help me), who let me go it alone for this one, so I kept asking to repeat again and over again for the whole meeting. The customer had been quite nice and was repeating and we finally ended the call with what every customer wants: answers and solutions.

The calls after, I was gaining more and more confident and able to do them on my own, I was still ending the calls with things to catch up on as I wasn't able to answer directly on the phone. The more calls I had about different subjects, the more I felt capable. I was also noticing my English level improving. Today I'm able to follow a complete call with those Irish folks and it makes me proud to know how far I have come learning a new language.

Things I learnt:

A customer has NO reason to challenge you, make fun of you, or judge in any way what you say. The vision you have of your own presentation is ALWAYS perceived worse than what the customer/audience/colleague is observing. Keep this in mind when you need to do a presentation, or in general talk in public.
Cloudflare trusts you and when the company hires you. Don't doubt your capabilities, you ARE capable.
No need to postpone the opportunities. Put yourself in a challenging situation, make mistakes, that's the way we all learn.
Never assume. Ask or verify with someone if you are unsure. You'll never be expected to know everything about everything but just to be able to produce the work needed to get a valuable answer.

Chapter 3 - Mess around and enjoy to be part of the Rocketship

It took me about 4 months before feeling confident in myself and autonomous, I mean autonomous in a sense that I was not discovering a new subject for any new customer or researching general questions, I knew the global subject and that there was one, to be able to dig for myself and to get to the solution I was looking for.

I started to be by myself, taking the lead on things, being confident (what a sentiment). I started to do things not especially related to my work with customers or prospects like taking time to improve my LUA skills, HTTP knowledge, Python, I wrote my first technical blog article and I even took part of the project of building a sound-level monitoring system based on a Raspberry PI and a decibel meter sending alerts to our internal chat system when the level was too high! Talking with engineers is also so great, discovering what's going on behind the scene, how the product is built and designed you're supposed to be the guru of the product in front of the customers.

I discovered that no matter the subject you pick-up at Cloudflare, mastering it will take a LONG time, which is quite exciting as I hate to be bored, really. As a Solutions Engineer, you're not asked to master every subject, that's why we're a team and we've our own preferences / natural abilities in terms of technical subjects. We then kind of provide consulting to each other when it's needed, and that's what's great.

I really enjoy my life at Cloudflare because I see me as my own boss, with deadlines, pipelines, objectives and no matter the path I take, the importance is to reach the target. Personal development is part of it, I was never asked to stop doing non-directly related to customer things, you're even advised to do so. It will give you the satisfaction of doing something that makes sense and challenges you.

Sweets delivery when we started to roll-out the DNS F-root (Yeah!)

The company itself now, and being part of a Rocketship has its advantages. It reassures me of the fact I took the correct decision 1 year and 3 months ago. I'm not saying that because I am career-obsessed but because I see the technical choices we make, how much we're growing, the fact we have such a smart team and we are able to keep it and that gives me the evidence that the fun at Cloudflare isn't going to stop.

Things I learnt:

Take time to fill your gaps, you will never be reproached for it.
Spend time on what you like and share with the team, don't keep it a secret!
Don't carry the whole load on your shoulders because 1) you couldn't afford it for the long term and 2) we're a team and need coverage on subjects.

Conclusion:

Working at Cloudflare took a lot of energy at the beginning to keep up the pace with the team. A team which is knowledgeable and keen to share the information is priceless and gives you the mission of reproducing the same with the colleagues asking you for something.

During the 1 year and 3 months I have been:

part of the 70th class,
we only had at the time 100 data centers,
I have visited 3 countries I've never been
I've seen the release of

Furthermore, I'm still excited delivering my best, day after day, to create a better internet.

If you’re willing to join an impressive team and work for a very dynamic company to help creating a better internet, we’re looking for many different profiles in our different offices over the planet! Let's have a look!