Roll your own JavaScript runtime

In this post we’ll walk through creating a custom JavaScript runtime. Let’s call it runjs. Think of it as building a (much) simplified version of deno itself. A goal of this post is to create a CLI that can execute local JavaScript files, read a file, write a file, remove a file and has simplified console API.

Let’s get started.

Update 2022-12-04: updated the code samples to the latest version of deno_core

Update 2023-02-16: we posted a second part of this tutorial, where we implement fetch-like API and add TypeScript transpilation.

Update 2023-05-04: we posted a third part of this tutorial, where we create snapshots to speed up startup time.

Update 2024-09-26: updated the code samples to the latest version of deno_core

Pre-requisites

This tutorial assumes that the reader has:

a basic knowledge of Rust
a basic knowledge of JavaScript event loops

Make sure you have Rust installed on your machine (along with cargo) and it should be at least 1.80.0. Visit rust-lang.org to install Rust compiler and cargo.

Make sure we’re ready to go:

$ cargo --version
cargo 1.80.1 (3f5fd8dd4 2024-08-06)

Hello, Rust!

First off, let’s create a new Rust project, which will be a binary crate called runjs:

$ cargo init --bin runjs
     Created binary (application) package

Change your working directory to runjs and open it in your editor. Make sure that everything is set up properly:

$ cd runjs
$ cargo run
   Compiling runjs v0.1.0 (/Users/ib/dev/runjs)
    Finished dev [unoptimized + debuginfo] target(s) in 1.76s
     Running `target/debug/runjs`
Hello, world!

Great! Now let’s begin creating our own JavaScript runtime.

Dependencies

Next, let’s add the deno_core and tokio dependencies to our project:

$ cargo add deno_core
    Updating crates.io index
      Adding deno_core v0.311.0 to dependencies.
$ cargo add tokio --features=full
    Updating crates.io index
      Adding tokio v1.40.0 to dependencies.

Our updated Cargo.toml file should look like this:

[package]
name = "runjs"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
deno_core = "0.311"
tokio = { version = "1.40", features = ["full"] }

deno_core is a crate by the Deno team that abstracts away interactions with the V8 JavaScript engine. V8 is a complex project with thousands of APIs, so to make it simpler to use them, deno_core provides a JsRuntime struct that encapsulates a V8 engine instance (called an Isolate) and allows integration with an event loop.

tokio is an asynchronous Rust runtime that we will use as an event loop. Tokio is responsible for interacting with OS abstractions like net sockets or file system. deno_core together with tokio allow JavaScript’s Promises to be easily mapped onto Rust’s Futures.

Having both a JavaScript engine and an event loop allows us to create a JavaScript runtime.

Hello, runjs!

Let’s start by writing an asynchronous Rust function that will create an instance of JsRuntime, which is responsible for JavaScript execution.

// main.rs
use deno_core::error::AnyError;
use std::rc::Rc;

async fn run_js(file_path: &str) -> Result<(), AnyError> {
  let main_module =
    deno_core::resolve_path(file_path, &std::env::current_dir()?)?;
  let mut js_runtime = deno_core::JsRuntime::new(deno_core::RuntimeOptions {
    module_loader: Some(Rc::new(deno_core::FsModuleLoader)),
    ..Default::default()
  });

  let mod_id = js_runtime.load_main_es_module(&main_module).await?;
  let result = js_runtime.mod_evaluate(mod_id);
  js_runtime.run_event_loop(Default::default()).await?;
  result.await
}

fn main() {
  println!("Hello, world!");
}

There’s a lot to unpack here. The asynchronous run_js function creates a new instance of JsRuntime, which uses a file-system based module loader. After that, we load a module into js_runtime runtime, evaluate it, and run an event loop to completion.

This run_js function encapsulates the whole life-cycle that our JavaScript code will go through. But before we can do that, we need to create a single-threaded tokio runtime to be able to execute our run_js function:

// main.rs
fn main() {
  let runtime = tokio::runtime::Builder::new_current_thread()
    .enable_all()
    .build()
    .unwrap();
  if let Err(error) = runtime.block_on(run_js("./example.js")) {
    eprintln!("error: {}", error);
  }
}

Let’s try to execute some JavaScript code! Create an example.js file that will print “Hello runjs!”:

// example.js
Deno.core.print("Hello runjs!");

Notice that we are using the print function from Deno.core - this is a globally available built-in object that is provided by the deno_core Rust crate.

Now run it:

cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
     Running `target/debug/runjs`
Hello runjs!⏎

Success! In just 33 lines of Rust code we created a simple JavaScript runtime, that can execute local files. Of course this runtime can’t do much at this point (for example, console.log doesn’t work yet - try it!), but we have integrated a V8 JavaScript engine and tokio into our Rust project.

Adding the `console` API

Let’s work on the console API. First, create the src/runtime.js file that will instantiate and make the console object globally available:

// runtime.js
const { core } = Deno;

function argsToMessage(...args) {
  return args.map((arg) => JSON.stringify(arg)).join(" ");
}

globalThis.console = {
  log: (...args) => {
    core.print(`[out]: ${argsToMessage(...args)}\n`, false);
  },
  error: (...args) => {
    core.print(`[err]: ${argsToMessage(...args)}\n`, true);
  },
};

The functions console.log and console.error will accept multiple parameters, stringify them as JSON (so we can inspect non-primitive JS objects) and prefix each message with log or error. This is a “plain old” JavaScript file, like we were writing JavaScript in browsers before ES modules.

Now let’s include this code in our binary and execute on every run:

let mut js_runtime = deno_core::JsRuntime::new(deno_core::RuntimeOptions {
  module_loader: Some(Rc::new(deno_core::FsModuleLoader)),
  ..Default::default()
});
+ let internal_mod_id = js_runtime
+   .load_side_es_module_from_code(
+     &deno_core::ModuleSpecifier::parse("runjs:runtime.js")?,
+     include_str!("./runtime.js"),
+   )
+   .await?;
+ let internal_mod_result = js_runtime.mod_evaluate(internal_mod_id);
let mod_id = js_runtime.load_main_es_module(&main_module).await?;
let result = js_runtime.mod_evaluate(mod_id);
js_runtime.run_event_loop(Default::default()).await?;
+ internal_mod_result.await?;
result.await

Finally, let’s update example.js with our new console API:

- Deno.core.print("Hello runjs!");
+ console.log("Hello", "runjs!");
+ console.error("Boom!");

And run it again:

cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
     Running `target/debug/runjs`
[out]: "Hello" "runjs!"
[err]: "Boom!"

It works! Now let’s add an API that will allow us to interact with the file system.

Adding a basic filesystem API

Let’s start by updating our runtime.js file:

};

+ globalThis.runjs = {
+   readFile: (path) => {
+     return core.ops.op_read_file(path);
+   },
+   writeFile: (path, contents) => {
+     return core.ops.op_write_file(path, contents);
+   },
+   removeFile: (path) => {
+     return core.ops.op_remove_file(path);
+   },
+ };

We just added a new global object, called runjs, which has three methods on it: readFile, writeFile and removeFile. The first two methods are asynchronous, while the third is synchronous.

You might be wondering what these core.ops.[op name] calls are - they’re mechanisms in deno_core crate for binding JavaScript and Rust functions. When you call either of these, deno_core will look for a Rust function that has an #[op2] attribute and a matching name.

Let’s see this in action by updating main.rs:

+ use deno_core::extension;
+ use deno_core::op2;
use deno_core::PollEventLoopOptions;
use std::rc::Rc;

+ #[op2(async)]
+ #[string]
+ async fn op_read_file(#[string] path: String) -> Result<String, AnyError> {
+   let contents = tokio::fs::read_to_string(path).await?;
+   Ok(contents)
+ }
+ 
+ #[op2(async)]
+ async fn op_write_file(#[string] path: String, #[string] contents: String) -> Result<(), AnyError> {
+   tokio::fs::write(path, contents).await?;
+   Ok(())
+ }
+ 
+ #[op2(fast)]
+ fn op_remove_file(#[string] path: String) -> Result<(), AnyError> {
+   std::fs::remove_file(path)?;
+   Ok(())
+ }

We just added three ops that could be called from JavaScript. But before these ops will be available to our JavaScript code, we need to tell deno_core about them by registering an “extension”:

+ extension!(
+   runjs,
+   ops = [
+     op_read_file,
+     op_write_file,
+     op_remove_file,
+   ]
+ );

async fn run_js(file_path: &str) -> Result<(), AnyError> {
  let main_module = deno_core::resolve_path(file_path)?;
  let mut js_runtime = deno_core::JsRuntime::new(deno_core::RuntimeOptions {
    module_loader: Some(Rc::new(deno_core::FsModuleLoader)),
+    extensions: vec![runjs::init_ops()],
    ..Default::default()
  });

Extensions allow you to configure your instance of JsRuntime and expose different Rust functions to JavaScript, as well as perform more advanced things like loading additional JavaScript code.

Due to that, we can now also clean up our code a bit by changing our declaration of out runtime.js:

extension!(
  runjs,
  ops = [
    op_read_file,
    op_write_file,
    op_remove_file,
  ],
+  esm_entry_point = "ext:runjs/runtime.js",
+  esm = [dir "src", "runtime.js"],
);

let mut js_runtime = deno_core::JsRuntime::new(deno_core::RuntimeOptions {
  module_loader: Some(Rc::new(deno_core::FsModuleLoader)),
-  extensions: vec![runjs::init_ops()],
+  extensions: vec![runjs::init_ops_and_esm()],
  ..Default::default()
});
- let internal_mod_id = js_runtime
-   .load_side_es_module_from_code(
-     &deno_core::ModuleSpecifier::parse("runjs:runtime.js")?,
-     include_str!("./runtime.js"),
-   )
-   .await?;
- let internal_mod_result = js_runtime.mod_evaluate(internal_mod_id);
let mod_id = js_runtime.load_main_es_module(&main_module).await?;
let result = js_runtime.mod_evaluate(mod_id);
js_runtime.run_event_loop(Default::default()).await?;
- internal_mod_result.await?;
result.await

The esm directive in the macro defines the JavaScript files we want to include into the extension. the dir "src" part specifies that our JavaScript files for this extension are located in the src directory. Then we add runtime.js to it as a file we want to include.

The esm_entry_point directive declares the file we want to use as an entrypoint. Let’s break down the string we have specified for it:

"ext:": a special schema used by deno_core to refer to extensions
runjs: the name of the extension we are trying to access the files of
runtime.js: the JavaScript file we want to use as the entrypoint

Now, let’s update our example.js again:

console.log("Hello", "runjs!");
console.error("Boom!");
+
+ const path = "./log.txt";
+ try {
+   const contents = await runjs.readFile(path);
+   console.log("Read from a file", contents);
+ } catch (err) {
+   console.error("Unable to read file", path, err);
+ }
+
+ await runjs.writeFile(path, "I can write to a file.");
+ const contents = await runjs.readFile(path);
+ console.log("Read from a file", path, "contents:", contents);
+ console.log("Removing file", path);
+ runjs.removeFile(path);
+ console.log("File removed");
+

And run it:

$ cargo run
   Compiling runjs v0.1.0 (/Users/ib/dev/runjs)
    Finished dev [unoptimized + debuginfo] target(s) in 0.97s
     Running `target/debug/runjs`
[out]: "Hello" "runjs!"
[err]: "Boom!"
[err]: "Unable to read file" "./log.txt" {"code":"ENOENT"}
[out]: "Read from a file" "./log.txt" "contents:" "I can write to a file."
[out]: "Removing file" "./log.txt"
[out]: "File removed"

Congratulations, our runjs runtime now works with the file system! Notice how little code was required to call from JavaScript to Rust - deno_core takes care of marshalling data between JavaScript and Rust so we didn’t need to do any of the conversions ourselves.

Summary

In this short example, we have started a Rust project that integrates a powerful JavaScript engine (V8) with an efficient implementation of an event loop (tokio).

A full working example can be found on denoland’s GitHub.