The Anatomy of an Isolate Cloud

If you’ve ever tried Deno Deploy, our multi-tenant distributed JavaScript cloud, you may understand some of these glowing reviews:

(And if you haven’t, deploy a Fresh site today!)

Achieving this deployment speed was no mistake. Designing Deno Deploy was an opportunity to re-imagine how we want the developer experience of deploying an app to be, and we wanted to focus on speed and security. Instead of deploying VMs or containers, we decided on V8 isolates, which allows us to securely run untrusted code with significantly less overhead.

This blog post explains what V8 isolates are and outlines the major architectural pieces of the Deno Deploy isolate cloud.

Deno Deploy Projects and Deployments

Within Deno Deploy, users and organizations manage their applications as projects. Deploy projects can be linked to GitHub repositories, enabling code to be redeployed on each git push. Projects also allow applications to manage environment variables, provision TLS certificates, and associate domain names. The Deploy dashboard also provides views of logs and analytics out of the box for increased visibility into an application’s behavior.

As previously mentioned, Deploy projects can be linked to GitHub repositories so that running applications are updated each time a change is pushed to the backing repository. In Deploy, each of these updates is known as a deployment. Deployments are immutable snapshots of an application. Each deployment has a unique URL that can be used to test or otherwise interact with the application over the Internet. A project can have many deployments, but only one of the deployments can be tagged as the production environment at any given time.

Figure 1 shows the Deployments dashboard for deno.land. Each deployment shows its unique deno.dev URL, as well as the git commit information that was used to create the deployment. The “Prod” label indicates the deployment that is currently running in production on deno.land.

Routing Requests to Deployments

But how do users’ HTTP requests reach a running deployment in the Deno Deploy cloud?

Each deployment is exposed to the Internet via one or more public URLs. During DNS resolution, all of these URLs are mapped to Deno Deploy’s public IPv4 or IPv6 address. This ensures that all deployment traffic is routed to the Deploy servers.

Depending on where the user is in the world, sending all traffic to the same destination can be very inefficient. For example, if a user is located in Australia, but all of the Deploy servers are located on the East coast of the United States, each HTTP request, including all necessary TCP handshaking, would need to travel thousands of miles around the globe.

Because Deno Deploy is intended to run JavaScript at the edge, it is critical that requests are instead intelligently routed to the edge location nearest the user. Deploy leverages anycast routing to share the same IPv4 and IPv6 addresses among the 30+ edge locations around the world, shown in Figure 2.

Once an incoming request is routed to the appropriate edge location, it’s handed off to a runner process. As its name implies, a runner is responsible for running applications, including receiving traffic and routing it to the correct deployment.

Deploy maintains a mapping table between domain names and deployments to serve this purpose. Depending on the protocol in use, the HTTP/1.1 Host header or HTTP/2 :authority pseudo-header is used to determine the domain that the request is intended for. Then, the TLS Server Name Indication (SNI) is used to determine which TLS certificate to use for the connection. Finally, the runner checks the mapping table to determine the correct deployment to send the request to. (As an aside, Deploy HTTP traffic is redirected to HTTPS, so the SNI can be used reliably.)

The Runner as an Isolate Hypervisor

In a traditional cloud, the hypervisor is responsible for creating, running, monitoring, and destroying virtual machines. In Deploy, the runner acts as an isolate hypervisor, serving the same purpose, but working with processes running V8 isolates instead of virtual machines.

Before going any further, we need to understand what an isolate is. V8, the JavaScript engine that powers Deno, Google Chrome, and several other popular runtimes documents an isolate as:

Isolate represents an isolated instance of the V8 engine. V8 isolates have completely separate states. Objects from one isolate must not be used in other isolates.

This means that an isolate is a distinct execution context, including code, variables, and everything needed to run a JavaScript application.

To help understand this, imagine a browser window with multiple tabs. Each tab executes JavaScript for a single webpage, but the JavaScript code from each tab generally cannot interact with the code from other tabs. In this example, each browser tab executes JavaScript code in its own V8 isolate.

The Deno Deploy cloud takes a similar approach. Each deployment executes its own JavaScript code in its own V8 isolate in its own process. After an incoming request has been mapped to a deployment, the runner is responsible for passing the request off to the deployment process. If the requested deployment is not already running, the runner starts a new instance of the deployment first. In order to conserve resources, the runner also terminates deployments that have been running for a while without receiving traffic.

A high level architectural view of a runner process and deployment processes is shown in Figure 3.

Security Considerations

Allowing many arbitrary users to upload and execute untrusted code is an extremely difficult technical challenge to solve effectively. Layered security is generally considered to be a good practice, but in this case it is a necessity because there is no single security mechanism that solves the Deno Deploy use case.

This section discusses some of the ways in which Deploy improves its security posture.

Using a more restrictive Deno runtime. Deno takes security very seriously. The open source Deno CLI uses its built-in permission system to lock down what a JavaScript application can do. Deno Deploy takes things one step farther by using a custom build of Deno that is even more restrictive. For example, deployments can read their own static assets, but cannot write to the file system.

Relying on V8 sandboxing. V8 was designed to run untrusted JavaScript code in a hostile environment — the browser. While V8 isolates alone do not provide a perfect sandbox, they do a good enough job for many use cases. V8 is also a very high profile public project with corporate backing. This leads to constant auditing, fuzzing, and testing of the codebase to ensure that it is secure. Occasional security exploits are reported, which Google takes very seriously. The Deno team is committed to staying up to date with V8 releases, as well as following the V8 team’s recommendations for running untrusted code.

Running deployments in separate processes. V8 isolates are already… isolated… from one another. Deno Deploy goes a step further and enlists the operating system’s help to improve the isolation of each deployment.

Hypervisor monitoring of resource utilization. The runner continuously tracks the metrics of all running deployments. This is necessary for billing purposes, but also allows the runner to enforce resource utilization quotas. Deployments that consume too many resources are terminated to prevent service degradation.

Restricting network access. Cloud providers and the operating system allow network access to be customized and locked down. Deploy also employs separate networks for internal control plane traffic and end user data plane traffic.

Restricting allowed system calls. As an added layer of security, Deploy uses seccomp filtering to limit the system calls that user code is allowed to execute.

Using Rust as the implementation language. When it comes to Deploy, JavaScript is the language in the cloud, but Rust is the language of the cloud. Rust enforces memory safety, eliminating an entire class of bugs. Conveniently, Rust also provides performance on par with C/C++.

What’s Next?

Everyday, as we’re iterating and improving on Deno Deploy, more developers and businesses choose to run production-ready applications and infrastructure with us. We’re confident that our architectural decisions enable them to focus on building for their users, without having to worry about security or performance.

Check out Deno Deploy and let us know what you think!

Do you find these challenges interesting to solve? We’re hiring!