How security and tenant isolation allows Deno Subhosting to run untrusted code securely
Deno Subhosting offers the easiest and most secure way to extend your platform’s functionality by allowing third-party developers to write and run custom code. For instance, Deno Subhosting powers Netlify’s edge functions so their users can do more with their sites.
But anytime third-party code is executed in the cloud, security becomes an understandable and critical concern — how can we prevent our developers from writing malicious code that attempts to access other users’ code or even our internal systems?
Running arbitrary code securely is a top priority for the serverless v8 isolate cloud that powers Deno Deploy and Deno Subhosting, and maximizing tenant isolation was considered at every step of designing our platform. To understand how security considerations were made from the ground up in our serverless platform, let’s first dive into the threat model.
The threat model
Our serverless compute platform allows anyone to deploy and run code on the internet. In order to make sure a single bad actor doesn’t bring down the rest of the system, we’ve developed these 5 directives:
Each deployment (a single tenant) must:
- Have access only to its own runtime (JavaScript objects, methods)
- Have access only to its own data (file, code, databases, environment variables)
- Have no access to Deno Deploy internals, such as the underlying operating system and internal services
- Not deny resources (e.g. CPU, memory) to other isolates
- Not use Deno Deploy for unintended use cases (e.g. mining bitcoin)
Given this threat model, there are three main themes in which we can isolate a tenant:
Let’s go into each below.
API limitations
To prevent the code from malicious behavior, there are many safeguards that limit what is even capable at the application level. This includes:
- Using V8 isolates: Google defines a V8 isolate as:
…an isolated instance of the V8 engine. V8 isolates have completely separate states. Objects from one isolate must not be used in other isolates.
Each V8 isolate is a sandbox, built in a way that makes it impossible for a program to do anything to another isolate. For instance, there are no JavaScript methods or objects that can be shared between isolates
Using a virtualized filesystem: When the code in the deployment uses filesystem methods, they don’t actually touch the filesystem since they are virtualized and scoped to that isolate.
Having a separate virtual private cloud (VPC) network: Our serverless platform uses two VPCs — one that is used to route incoming requests from the internet to our runner, and a separate one that the isolates use to access the internet (e.g. if your deployment calls
fetch(), it uses this network).
These three work together to provide isolation around the application inside each deployment.
Defense in depth
Beyond simply isolating the application code in each tenant, one of our threat models is to make sure the code cannot also access underlying systems, such as the operating system. There are a few things that our infrastructure uses to guard against that:
Running its own process on a more restrictive Deno runtime: While the Deno runtime already offers an opt-in permission system to lock down what a JavaScript program can do, our serverless infrastructure extends this further by using a more restrictive version of Deno. For example, deployments can read their own static assets, but cannot write to the file system.
Using Linux namespaces: Namespaces partition Linux kernel resources so that one set of processes sees one set of resources while another set of processes sees a different set of resources. This feature was developed to further isolate services and minimize “blast radius” for changes. This is a common isolation technique used by Docker and Kubernetes.
Using seccomp filters: So far, every isolate runs in its own process and namespace, and for it to interact with another process, it would need to use syscalls that it would not use under normal operations. If a process is looking for unusual syscalls, we can detect attempts to break out of the sandbox. Seccomp (“SECure COMPuting”) filters allow us to check if processes are trying to use unusual syscalls and therefore attempt malicious behavior.
These safeguard the process from accessing lower level commands and systems that may allow it to interact with other tenants.
Resource fairness
Finally, another area of security is making sure that each tenant gets to use the resources it’s allotted for. This yields another benefit of making sure that every user of Deno Deploy and Deno Subhosting gets their fair share of resources, and that not one singular user is hogging all the CPU (for instance). The tools below also make sure that people are using our V8 isolate cloud within the terms and conditions (aka not mining bitcoin).
Using cgroups: Control groups (“cgroups”) allow us to define boundaries around resource usage (CPU, memory, disk I/O, network ingress/egress, etc.) of one or more processes. When a defined resource limit is exceeded, cgroups automatically throttle a process. The cgroup defense sets an upper limit, which prevents one single deployment or tenant from exceeding CPU usage that could negatively impact the performance of surrounding tenants.
Monitoring with our WatchDog: We’ve built our own internal service, WatchDog, that’s watches and monitors CPU utilization and memory across isolates on any given runner. For instance, the WatchDog terminates isolates that exceed their memory limit for some amount of time or block the event loop for a long time (i.e. they have become responsive).
These tools ensure that resources are shared fairly across all tenants and that no singular deployment is exploiting our platform.
Revisiting the threat model
So how do the above tools fit into our threat model? Let’s take a look:
Each deployment (a single tenant) must:
Have access only to its own runtime (JavaScript objects, methods)
This is achieved by running each deployment in its own V8 isolate, having a virtualized filesystem, and having a separate VPC network.
Have access only to its own data (file, code, databases, environment variables)
This is achieved from using its own process, namespaces, and seccomp filters.
Have no access to Deno Deploy internals, such as the underlying operating system and internal services
This is achieved through API limitations and a separate VPC network, as well as defense-in-depth measures like putting each process in a separate namespace and configure seccomp to detect syscalls that the process should not be making.
Not deny resources (e.g. CPU, memory) to other isolates
This is achieved by cgroups and the Deno WatchDog. The cgroup defense sets an upper limit, while the WatchDog terminates isolates that exceed their memory limit for some amount of time, or block the event loop for a long time.
Not use Deno Deploy for unintended use cases (e.g. mining bitcoin)
The Deno WatchDog is programmed to detect unusual behavior that goes against our terms and conditions.
Building a secure organization
Security is an important concern when building for the web. Not only is an opt-in permissions system built right into the runtime, it’s also a top focus for our globally distributed serverless hosting platform. Every decision we made while designing our hosting infrastructure was made with security in mind.
Finally, to drive home our commitment to security beyond our runtime and hosting platform, the Deno company was independently audited and achieved SOC 2 compliance.
Run your users’ untrusted code with confidence. Sign up for Deno Subhosting today.