The following was originally written as a guest post by Ashcon Partovi, while a computer science and business undergraduate at the University of British Columbia in Vancouver, Canada. He's the founder of a popular Minecraft multiplayer server, stratus.network, that provides competitive, team-based gameplay to thousands of players every week. He also now works at Cloudflare.
If you've ever played a video game in the past couple of years, chances are you know about Minecraft. You might be familiar with the game or even planted a tree or two, but what you might not know about is the vast number of Minecraft online communities. In this post, I'm going to describe how I used Cloudflare Workers to deploy and scale a production-grade API that solves a big problem for these Minecraft websites.
Introducing the Problem
Here is an example of my Minecraft player profile from one of the many multiplayer websites. It shows some identity information such as my username, a bitmap of my avatar, and a preview of my friends. Although rendering this page with 49 bitmap avatars may seem like an easy task, it's far from trivial. In fact, it's unnecessarily complicated.
Here is the current workflow to render a player profile on a website given their username:
- Find the UUID from the player's username.
curl api.mojang.com/users/profiles/minecraft/ElectroidFilms
{
"id": "dad8b95ccf6a44df982e8c8dd70201e0",
"name": "ElectroidFilms"
}
- Use that UUID to fetch the latest player information from the session server.
curl sessionserver.mojang.com/session/minecraft/profile/dad8b95cc...
{
"id": "dad8b95ccf6a44df982e8c8dd70201e0",
"name": "ElectroidFilms",
"properties": [{
"name": "textures",
"value": "eyJ0aW1lc3RhbXAiOjE1MzI1MDI..." // <base64>
}]
}
- Decode the textures string which is encoded as base64.
echo "eyJ0aW1lc3RhbXAiOjE1MzI1MDIwNDY5NjIsIn..." | base64 --decode
{
"timestamp": 1532502046962,
"profileId": "dad8b95ccf6a44df982e8c8dd70201e0",
"profileName": "ElectroidFilms",
"textures": {
"SKIN": {"url": "textures.minecraft.net/texture/741df6aa0..."},
"CAPE": {"url": "textures.minecraft.net/texture/e7dfea16d..."}
}
}
- Fetch the texture from the URL in the decoded JSON payload.
curl textures.minecraft.net/texture/741df6aa027... > skin.png
- Cache the texture in a database to avoid the 60-second rate limit.
mongo
> db.users.findOneAndUpdate(
{ _id: "dad8b95ccf6a44df982e8c8dd70201e0" },
{ skin_png: new BinData(0, "GWA3u4F42GIH318sAlN2wfDAWTQ...") })
Yikes, that's 5 complex operations required to render a single avatar! But that's not all, in my example profile, there are 49 avatars, which would require a total of 5 * 49 = 245 operations.
And that's just fetching the data, we haven't even started to serve it to players! Then you have to setup a host to serve the web traffic, ensure that the service scales with demand, handle cache expiration of assets, and deploy across multiple regions. Then you have to deploy There has to be a better way!
Prototyping with Workers
I'm a strong believer in the future of serverless computing. So naturally, when I learned how Cloudflare Workers allow you to run JavaScript code in 150+ points of presence, I started to tinker with the possibilities of solving this problem. After looking at the documentation and using the Workers playground, I quickly put together some JavaScript code that aggregated all that profile complexity into a single request.
addEventListener('fetch', event => {
event.respondWith(renderPlayerBitmap(event.request))
})
async function renderPlayerBitmap(request) {
var username = request.url.split("/").pop()
console.log("Starting request for... " + username)
// Step 1: Username -> UUID
var uuid = await fetch("https://api.mojang.com/users/profiles/minecraft/" + username)
if(uuid.ok) {
uuid = (await uuid.json()).id
console.log("Found uuid... " + uuid)
// Step 2: UUID -> Profile
var session = await fetch("https://sessionserver.mojang.com/session/minecraft/profile/" + uuid)
if(session.ok) {
session = await session.json()
console.log("Found session... " + JSON.stringify(session))
// Step 3: Profile -> Texture URL
var texture = atob(session.properties[0].value)
console.log("Found texture... " + texture)
// Step 4 + 5: Texture URL -> Texture PNG + Caching
texture = JSON.parse(texture)
return fetch(texture.textures.SKIN.url, cf: {cacheTtl: 60})
}
}
return new Response(undefined, {status: 500})
}
Within a couple minutes I had my first Workers implementation! I gave it my username and it was able to make all the necessary sub-requests to return my player's bitmap texture.
After realizing the potential of Workers, I started to wonder if I could use it for more than just a single script. What if I could design and deploy a production-ready API for Minecraft that runs exclusively on Workers?
Designing an API
I wanted to address an essential problem for Minecraft developers: too many APIs with too many restrictions. The hassle of parsing multiple requests and handling errors prevents developers from focusing on creating great experiences for players. There needs to be a solution that requires only 1 HTTP request with no rate limiting and no client-side caching. After looking at the various use-cases for the existing APIs, I created a JSON schema that encompassed all the essential data into a single response:
GET: api.ashcon.app/mojang/v1/user/<username|uuid>
{
"uuid": "<uuid>",
"username": "<username>",
"username_history": [
{
"username": "<username>",
"changed_at": "<date|null>"
}
],
"textures": {
"slim": "<boolean>",
"custom": "<boolean>",
"skin": {
"url": "<url>",
"data": "<base64>"
},
"cape": {
"url": "<url|null>",
"data": "<base64|null>"
}
},
"cached_at": "<date>"
}
One of the primary goals I had in mind was to minimize sub-requests by clients. For example, instead of giving developers a URL to a image/png static asset, why not fetch it for them and embed it as a base64 string? Now that's simplicity!
Getting Started
For this project, I decided to use CoffeeScript, which transcompiles to JavaScript and has a simple syntax. We'll also need to use Webpack to bundle all of our code into a single JavaScript file to upload to Cloudflare.
# Welcome to CoffeeScript!
str = "heyo! #{40+2}" # 'heyo! 42'
num = 12 if str? # 12
arr = [1, null, "apple"] # [1, null, 'apple']
val = arr[1]?.length() # null
hash = # {key: 'value'}
key: "value"
add = (a, b, {c, d} = {}) ->
c ?= 3
d ?= 4
a + b + c + d
add(1, 2, d: 5) # 1 + 2 + 3 + 5 = 11
First, let's make sure we have the proper dependencies installed for the project! These commands will create a package.json file and a node_modules/ folder in our workspace.
mkdir -p workspace/src
cd workspace
npm init --yes
npm install --save-dev webpack webpack-cli coffeescript coffee-loader workers-preview
Now, we're going to edit our package.json to add two helper scripts for later. You can delete the default "test" script as well.
"scripts": {
"build": "webpack",
"build:watch": "webpack --watch",
"preview": "workers-preview < dist/bundle.js"
}
We also need to initialize a webpack.config.js file with a CoffeeScript compiler.
const path = require('path')
module.exports = {
entry: {
bundle: path.join(__dirname, './src/index.coffee'),
},
output: {
filename: 'bundle.js',
path: path.join(__dirname, 'dist'),
},
mode: 'production',
watchOptions: {
ignored: /node_modules|dist|\.js/g,
},
resolve: {
extensions: ['.coffee', '.js', '.json'],
plugins: [],
},
module: {
rules: [
{
test: /\.coffee?$/,
loader: 'coffee-loader',
}
]
}
}
Before we start coding, we'll create a src/index.coffee file and make sure everything is working so far.
addEventListener('fetch', (event) ->
event.respondWith(route(event.request)))
# We will populate this with our own logic after we test it!
route = (request) ->
fetch('https://api.ashcon.app/mojang/v1/user/ElectroidFilms')
Open your terminal in the workspace/ directory and run the following commands:
npm run build
npm run preview
Your computer's default internet browser will open up a new window and preview the result of our Worker. If you see a JSON response, then everything is working properly and we're ready to go!
Writing Production Code for Workers
Now that we're setup with a working example, we can design our source code file structure. It's important that we break up our code into easily testable chunks, so I've gone ahead and outlined the approach that I took with this project:
src/
index.coffee # routing and serving requests
api.coffee # logic layer to mutate and package requests
mojang.coffee # non-logic layer to send upstream requests
http.coffee # HTTP requesting, parsing, and responding
util.coffee # util methods and extensions
If you've feeling adventurous, I've included a simplified version of my API code that you can browse through below. If you look at each file, you'll have a fully working implementation by the end! Otherwise, you can continue reading to learn about my deployment and analysis of the APIs impact.
http.coffee
Since our API will be making several HTTP requests, it's a good idea to code some common request and respond methods that can be reused among multiple requests. At the very least, we need to support parsing JSON or base64 responses and sending JSON or string data back to the client.
# Send a Http request and get a response.
#
# @param {string} url - Url to send the request.
# @param {string} method - Http method (get, post, etc).
# @param {integer} ttl - Time in seconds for Cloudflare to cache the request.
# @param {boolean} json - Whether to parse the response as json.
# @param {boolean} base64 - Whether to parse the response as a base64 string.
# @returns {promise<
# json -> [err, json]
# base64 -> string|null
# else -> response
# >} - A different response based on the method parameters above.
export request = (url, {method, ttl, json, base64} = {}) ->
method ?= "GET"
response = await fetch(url, method: method, cf: {cacheTtl: ttl} if ttl)
if json # Return a tuple of [err, json].
if err = coerce(response.status)
[err, null]
else
[null, await response.json()]
else if base64 # Return base64 string or null.
if response.ok
Buffer.from(await response.arrayBuffer(), "binary").toString("base64")
else # If no parser is specified, just return the raw response.
response
export get = (url, options = {}) ->
request(url, Object.assign(options, {method: "GET"}))
# Respond to a client with a http response.
#
# @param {object} data - Data to send back in the response.
# @param {integer} code - Http status code.
# @param {string} type - Http content type.
# @param {boolean} json - Whether to respond in json.
# @param {boolean} text - Whether to respond in plain text.
# @returns {response} - Raw response object.
export respond = (data, {code, type, json, text} = {}) ->
code ?= 200
if json
type = "application/json"
# "Pretty-print" our JSON response with 2 spaces.
data = JSON.stringify(data, undefined, 2)
else if text
type = "text/plain"
data = String(data)
else
type ?= "application/octet-stream"
new Response(data, {status: code, headers: {"Content-Type": type}})
export error = (reason = null, {code, type} = {}) ->
code ?= 500
type ?= "Internal Error"
# An example would be: "Internal Error - 500 (this is the reason)"
respond("#{code} - #{type}" + (if reason then " (#{reason})" else ""), code: code, text: true)
export badRequest = (reason = null) ->
error(reason, code: 400, type: "Bad Request")
export notFound = (reason = null) ->
error(reason, code: 404, type: "Not Found")
export tooManyRequests = (reason = null) ->
error(reason, code: 429, type: "Too Many Requests")
# Convert common http error codes into error responses.
#
# @param {integer} code - Http status code.
# @returns {response|null} - An error response or null if a 200 code.
export coerce = (code) ->
switch code
when 200 then null
# Some Minecraft APIs use 204 as a stand-in for a 404.
when 204 then notFound()
when 400 then invalidRequest()
# Theoretically this should never happen, but sometimes does.
when 429 then tooManyRequests()
else error("Unknown Response", code: code)
The cf key can be used to control various Cloudflare features, including how sub-requests are cached. See the Workers documentation for a more in-depth explanation.
cf:
cacheTtl: 120 # Cache for 2 mins.
# Pro+ only.
polish: "lossless" # Compress image data.
# Enterprise only.
cacheTtlByStatus:
"200-299": 60 # Cache for 60 secs.
"300-399": 0 # Cache but expire instantly.
"400-404": 10 # Cache for 10 secs.
"405-599": -1 # Do not cache at all.
cacheKey: url # Cache lookup key, defaults to the request URL.
mojang.coffee
Now that we have code to send and parse requests, we can create an interface to retrieve data from the upstream APIs. It's good to note that there should be no mutation logic in this file, it's purpose is just to get the old APIs, not change them.
import { get } from "./http"
# Get the UUID of a username at the current time.
#
# @param {string} name - Minecraft username.
# @throws {204} - When no user exists with that name.
# @returns {[err, json]} - An error or username and UUID response.
export usernameToUuid = (name) ->
get("https://api.mojang.com/users/profiles/minecraft/#{name}", json: true)
# Get the history of usernames for the given UUID.
#
# @param {string} id - The UUID to check the username history.
# @returns {[err, json]} - An error or the username history.
export uuidToUsernameHistory = (id) ->
get("https://api.mojang.com/user/profiles/#{id}/names", json: true)
# Get the session profile of the UUID.
#
# @param {string} id - UUID to get the session profile.
# @returns {[err, json]} - An error or the session profile.
export uuidToProfile = (id) ->
get("https://sessionserver.mojang.com/session/minecraft/profile/#{id}", json: true)
api.coffee
This is where the bulk of our API logic will reside. I've broken up the process into 3 interdependent tasks that are executed in order:
- Given a username, fetch its UUID.
- Given a UUID, fetch the user's profile.
- Given a user's profile, decode and fetch the textures.
import { get, respond, error, notFound, badRequest } from "./http"
import { usernameToUuid, uuidToProfile, uuidToUsernameHistory } from "./mojang"
# Get the uuid of a user given their username.
#
# @param {string} name - Minecraft username, must be alphanumeric 16 characters.
# @returns {[err, response]} - An error or the dashed uuid of the user.
export uuid = (name) ->
if name.asUsername() # Fits regex of a Minecraft username.
[err, res] = await usernameToUuid(name)
if id = res?.id?.asUuid(dashed: true)
[null, respond(id, text: true)]
else # Response was received, but contains no UUID.
[err || notFound(), null]
else
[badRequest("malformed username '#{name}'"), null]
# Get the full profile of a user given their uuid or username.
#
# @param {string} id - Minecraft username or uuid.
# @returns {[err, json]} - An error or user profile.
export user = (id) ->
if id.asUsername()
[err, res] = await uuid(id)
if err # Could not find a player with that username.
[err, null]
else # Recurse with the new UUID.
await user(id = await res.text())
else if id.asUuid()
# Fetch the profile and usernames in parallel.
[[err0, profile], [err1, history]] = await Promise.all([
uuidToProfile(id = id.asUuid())
uuidToUsernameHistory(id)])
# Extract the textures from the profile.
# Since this operation is complex, off-load
# the logic into its own method.
[err2, texture] = await textures(profile)
if err = err0 || err1 || err2
[err, null] # One of the last three operations failed.
else
# Everything is good, now just put the data together.
[null, respond(
uuid: profile.id.asUuid(dashed: true)
username: profile.name
username_history: history.map((item) ->
username: item.name
changed_at: item.changedToAt?.asDate())
textures: texture
cached_at: new Date(),
json: true)]
else
[badRequest("malformed uuid '#{id}'"), null]
# Parse and decode base64 textures from the user profile.
#
# @param {json} profile - User profile from #uuidToProfile(id).
# @returns {json} - Enhanced user profile with more convient texture fields.
textures = (profile) ->
unless profile # Will occur if the profile api failed.
return [error("no user profile found"), null]
properties = profile.properties
if properties.length == 1
texture = properties[0]
else
texture = properties.filter((pair) -> pair.name == "textures" && pair.value?)[0]
# If a embedded texture does not exist or is empty,
# that user does not have a custom skin.
if !texture || (texture = JSON.parse(atob(texture.value)).textures).isEmpty()
skinUrl = "http://assets.mojang.com/SkinTemplates/steve.png"
# Fetch the skin and cape data in parallel, and cache for a day.
[skin, cape] = await Promise.all([
get(skinUrl ?= texture.SKIN?.url, base64: true, ttl: 86400)
get(capeUrl = texture.CAPE?.url, base64: true, ttl: 86400)])
unless skin
[error("unable to fetch skin '#{skinUrl}'"), null]
else
texture =
slim: texture.SKIN?.metadata?.model == "slim"
skin: {url: skinUrl, data: skin}
cape: {url: capeUrl, data: cape} if capeUrl
[null, texture]
index.coffee
Now, we parse the request's route and respond with the corresponding API.
import "./util"
import { notFound } from "./http"
import { uuid, user } from "./api"
addEventListener("fetch", (event) ->
event.respondWith(route(event.request)))
route = (request) ->
[base, version, method, id] = request.url.split("/")[3..6]
if base == "mojang" && id?
if version == "v1"
v1(method, id)
else
notFound("unknown api version '#{version}'")
else
notFound("unknown route")
v1 = (method, id) ->
if method == "uuid"
[err, res] = await uuid(id)
else if method == "user"
[err, res] = await user(id)
err || res || notFound("unknown v1 route '#{method}'")
util.coffee
Finally, we'll add some prototype extensions that we used along the way.
# Insert a string at a given index.
#
# @param {integer} i - Index to insert the string at.
# @param {string} str - String to insert.
String::insert = (i, str) ->
this.slice(0, i) + str + this.slice(i)
# Ensure that the string is a valid Uuid.
#
# If dashed is enabled, it is possible the input
# string is not the same as the output string.
#
# @param {boolean} dashed - Whether to return a dashed uuid.
# @returns {string|null} - A uuid or null.
String::asUuid = ({dashed} = {}) ->
if match = uuidPattern.exec(this)
uuid = match[1..].join("")
if dashed
uuid.insert(8, "-")
.insert(12+1, "-")
.insert(16+2, "-")
.insert(20+3, "-")
else
uuid
uuidPattern = /^([0-9a-f]{8})(?:-|)([0-9a-f]{4})(?:-|)(4[0-9a-f]{3})(?:-|)([0-9a-f]{4})(?:-|)([0-9a-f]{12})$/i
# Ensure that the string is a valid Minecraft username.
#
# @returns {string|null} - Minecraft username or null.
String::asUsername = ->
if usernamePattern.test(this) then this else false
usernamePattern = /^[0-9A-Za-z_]{1,16}$/i
# Ensure that the unix number is a Date.
#
# @returns {date} - The number as a floored date.
Number::asDate = ->
new Date(Math.floor(this))
# Determine if the object is empty.
#
# @returns {boolean} - Whether the object is empty.
Object::isEmpty = ->
Object.keys(this).length == 0
Analyzing a Workers Deployment
I've had this code deployed and tested by real Minecraft users for the past few weeks. As a developer that has global web traffic, it's pivotal that players can quickly get access to my services. The essential advantage of Workers is that I don't need to deploy several replicas of my code to different cloud regions, it's everywhere! That means players from any part of the world get the same great web experience with minimal latency.
As of today, the API is processing over 400k requests per day from users all over the world! Cloudflare caches responses in the closest point of presence to the client, so I don't need to setup a database and developers don't need to worry about rate-limiting.
Since each request to the API generates 4 to 5 additional sub-requests, it handles approximately 1.8 million fetches per day with a 88% cache hit rate.
Wrapping Up
Cloudflare Workers have enabled me to solve complex technical problems without worrying about host infrastructure or cloud regions. It's simple, easy to deploy, and works blazing fast all around the world. And for 50 cents for every 1 million requests, it's incomparable to the other serverless solutions on the market.
If you're not already convinced to start using Workers, here's the deployment history of my API. I went from 0 to 5 million requests with no scaling, no resizing, no servers, no clusters, and no containers. Just code.
If you're interested in looking at all of the code used in the post, you can find it here:
https://github.com/Electroid/mojang-api
And if you're a Minecraft developer, my API is open for you to use for free:
curl https://api.ashcon.app/mojang/v1/uuid/ElectroidFilms
curl https://api.ashcon.app/mojang/v1/user/ElectroidFilms
You can also use this extra goodie that will crop just the face from a player texture:
curl https://api.ashcon.app/mojang/v1/avatar/ElectroidFilms > avatar.png
open avatar.png