Publishing WebAssembly modules with AssemblyScript

Introduction

WebAssembly (WASM) is a compilation target for other programming languages. It runs in many computing environments, and it runs fast. Typically a WebAssembly module begins life as C/C++, Rust, or AssemblyScript source code. A toolchain compiles the source code to a .wasm binary, and that binary is loaded into environments like web browsers and Node.js.

AssemblyScript — heavily inspired by TypeScript — will be more approachable to web developers than C/C++ or Rust, and we'll focus on AssemblyScript here.

This post explains how to set up a new AssemblyScript project, compile that project as a WebAssembly module, write unit tests, and publish the package to npm. We'll spend some time on compatibility, so that the same npm package works well in web pages, Node.js, and bundlers.

If you're new to AssemblyScript, reading an introduction to the language may be worthwhile. While I've claimed above that WebAssembly is fast, new code still needs to be optimized, and this requires thinking about issues that web developers typically don't. Read Surma's excellent post, Is WebAssembly magic performance pixie dust?, for an introduction to optimizing AssemblyScript code.

Setting up a new project

NOTE: This section mirrors AssemblyScript's own tutorials. If my post falls out of date, prefer the official documentation.

With a recent version of Node.js installed, run the following steps in a new directory:

# Initialize an empty npm package.
npm init

# Install the AssemblyScript compiler.
npm install --save-dev assemblyscript

# Generate AssemblyScript project structure
npx asinit .

The last step creates several folders and files in the project directory. These are worth mentioning:

The files below can be deleted. We'll set up more helpful unit tests later.

Then, create a src/ folder and two empty source files, which we'll use to load our WASM module and create a clean TypeScript API around it. If you'd prefer to use JavaScript instead of TypeScript, feel free.

Compile AssemblyScript to WebAssembly

The default assembly/index.ts defines a simple function adding two integers:

// The entry file of your WebAssembly module.

export function add(a: i32, b: i32): i32 {
    return a + b;
}

Let's leave that alone and compile it. An asbuild script is already defined in our package.json file, which we can execute:

npm run asbuild

The compiler builds our WebAssembly binaries and JavaScript bindings, and puts release and debug builds in the build/ folder. Open the release bindings in build/release.js, and you'll see something like this toward the end of the file:

export const { ... } = await (async url => instantiate(
    await (async () => {
        try { return await globalThis.WebAssembly.compileStreaming(globalThis.fetch(url)); }
        catch { return globalThis.WebAssembly.compile(await (await import("node:fs/promises")).readFile(url)); }
    })(), {
    }
))(new URL("release.wasm", import.meta.url));

If we only wanted drop these files into a web or Node.js application, this might be fine. But when publishing a package for npm, there are a few problems above. While an application might happily ignore the import("node:fs/promises") code unless it hits the catch block, our users' bundlers might not. Moreover, older bundlers don't support import.meta.url, and cannot manage static binary resources.

We'll dive deeper into both of those compatibility issues in a later section, but for now let's just strip out the problematic bindings by changing the AssemblyScript configuration. In asconfig.json, set "bindings": "raw" under the compiler options:

{
    ...
    "options": {
        "bindings": "raw"
    }
}

Now run "npm run asbuild" again, and the problematic parts of our build/release.js bindings should be gone.

Writing the wrapper

Wrapping AssemblyScript's output with hand-written code isn't strictly necessary, but I'd encourage it when publishing the package to npm. Wrapping allows us to:

  1. Make the package easier to import for our users
  2. Display documentation with JSDoc
  3. Provide more specific TypeScript hints
  4. Perform input validation before data is passed into the WASM module[1]
  5. Implement features that would be difficult or slow in WASM[2]

WebAssembly and its various toolchains will improve, and I expect this list to be shorter in a few years.

Let's start by writing a simple wrapper that supports only Node.js, and in the next section we'll adapt it for other environments. Without modifying ./assembly/tsconfig.json (which affects our AssemblyScript compilation), we'll add a new file, ./tsconfig.json, for our TypeScript wrapper. Replace "my-package" with whatever package name you prefer.

{
    "compilerOptions": {
        "paths": {
            "my-package": ["./"]
        },
        "typeRoots": ["node_modules/@types"],
        "moduleResolution": "node",
        "module": "ESNext",
        "lib": ["ESNext", "DOM"],
        "target": "ESNext",
        "declaration": true,
        "strict": true
    },
}

In src/wasm.ts, write:

// src/wasm.ts — Placeholder WASM loader.
import { readFile } from 'node:fs/promises';
const wasmURL = /* #__PURE__ */ new URL('./release.wasm', import.meta.url);
const wasm = /* #__PURE__ */ readFile(wasmURL);
export default wasm;

Here, we're loading the WASM binary from a TypeScript module[3] we define, which we'll override for different environments later. The #__PURE__ */ annotations allow bundlers to strip away (or "tree-shake") our WASM binary if the application never uses it. While optional, this will be important if our package has multiple WASM modules, or becomes an optional dependency of other downstream packages.

In src/index.ts (not assembly/index.ts), add the wrapper:

// src/index.ts
import wasm from './wasm';
import { instantiate, __AdaptedExports } from '../build/release';

// Initialization.

let exports: typeof __AdaptedExports;

/* Promise resolving when WebAssembly is ready. */
export const ready = /* #__PURE__ */ new Promise<void>(async (resolve, reject) => {
    try {
        const module = await WebAssembly.compile(await wasm);
        exports = await instantiate(module as BufferSource, {});
        resolve();
    } catch (e) {
        reject(e);
    }
});

// Wrapper API.

/**
 * Returns the sum of two 32-bit signed integers.
 * param a Integer in the range -2,147,483,648 to +2,147,483,647.
 * param b Integer in the range -2,147,483,648 to +2,147,483,647.
 */
export function add(a: number, b: number): number {
    if (!exports) {
        throw new Error('WebAssembly not yet initialized: await "ready" export.');
    }
    return exports.add(a, b);
}

The instantiate function provided by the AssemblyScript compiler takes the WASM binary and returns an instantiated, ready-to-use WASM module. Our wrapper includes any documentation, type definitions, and input validation we'll want for the public API.

Loading and instantiating the WASM module can take a while, and we don't want to block the entire application during that time. We'll export a ready promise, and users of our package should await that promise:

import { ready, add } from 'my-package';

await ready;
add(2, 2); // → 4

Bundling the package

With our TypeScript wrapper written, we're ready to bundle and test the package. We'll use a few more dependencies. The typescript and ts-node dependencies can be skipped if you're writing plain JavaScript. The test framework, ava, is optional. Use another test framework that supports TypeScript and ES Modules[4] if you prefer, or none at all.

npm install --save-dev microbundle ava typescript ts-node

I strongly recommend using Microbundle — an opinionated Rollup wrapper — for reasons that will be apparent when we start adapting the package for different environments. While it isn't a common bundler for building web applications, it's an exceptional choice for building libraries.

In package.json, under the scripts section, we'll define our build and test commands:

{
    "scripts": {
        "build": "npm run asbuild && npm run build:node",
        "build:node": "microbundle build --target node --format modern,cjs --raw --no-compress --output build/my-package-node.js --external node:fs/promises",
        "test": "ava test/*.test.ts",
        ...
    },

    ...

Define the entry points to the compiled package, as our users and unit tests will need that information. Remove any existing versions of these entries in package.json, then add the entries below:

{
    "name": "my-package",
    "type": "module",
    "sideEffects": false,
    "types": "./build/index.d.ts",
    "exports": "./build/my-package-node.modern.js",

    ...

Careful readers may notice a small mismatch in our exports path and the --output flag given to Microbundle. That's intentional: the .modern.js suffix is added by Microbundle automatically.

When we build the package, our script will compile the AssemblyScript module and then the TypeScript wrapper to the build/ directory.

npm run build

We now have a compiled JavaScript bundle in ./build/my-package-node.modern.js, type declarations in ./build/index.d.ts, and a WASM binary in ./build/release.wasm. Let's test the code.

Testing the package

NOTICE: As of this writing, Ava needs additional configuration to support ES Modules and TypeScript together. In package.json, add the options below:

{
    "ava": {
        "extensions": {
            "ts": "module"
        },
        "nodeArguments": [
            "--loader=ts-node/esm"
        ]
    },

    ...

We'll write a simple unit test, test/index.test.ts:

// index.test.ts
import test from 'ava';
import { ready, add } from 'my-package';

test('add', async (t) => {
    await ready;
    t.is(add(2, 2), 4, '2 + 2 = 4');
    t.is(add(5, 0), 5, '5 + 0 = 5');
    t.is(add(10, -3), 7, '10 - 3 = 7');
});

Then, run the test.

npm test

With any luck, the test should pass![5]

Compatibility concerns

If you've made it this far, nice work! You may be wondering why this blog post couldn't have been written with half the words and half the dependencies. Here's why.

Our package is now working in a Node.js test environment, but it's not going to run in a web browser, and could easily cause failures in our users' build systems. These environments have incompatible requirements, and our challenge here is to provide builds that work for everyone without creating a maintenance nightmare for ourselves.

Replacing the src/wasm.ts loader with a stub, we're going to create three new loaders and a build target for each:

// src/wasm.ts — Stub, replaced at compile-time.
const wasm = new Uint8Array(0);
export default wasm;
// src/wasm-compat.ts — Legacy bundler WASM loader.
const WASM_BASE64 = '';
const wasm = /* #__PURE__ */ fetch('data:application/wasm;base64' + WASM_BASE64);
export default wasm;
// src/wasm-node.ts — Node.js WASM loader.
import { readFile } from 'node:fs/promises';
const wasmURL = /* #__PURE__ */ new URL('./release.wasm', import.meta.url);
const wasm = /* #__PURE__ */ readFile(wasmURL);
export default wasm;
// src/wasm-default.ts — Web & modern bundler WASM loader.
const wasm = /* #__PURE__ */ fetch(/* #__PURE__ */ new URL('./release.wasm', import.meta.url));
export default wasm;

We'll extend our package.json#exports entries to list each of these options:

{
    "type": "module",
    "sideEffects": false,
    "source": "./src/index.ts",
    "types": "./build/index.d.ts",
    "main": "./build/my-package-node.cjs",
    "module": "./build/my-package-default.modern.js",
    "exports": {
        "compat": {
            "types": "./build/index.d.ts",
            "require": "./build/my-package-compat.cjs",
            "default": "./build/my-package-compat.modern.js"
        },
        "node": {
            "types": "./build/index.d.ts",
            "require": "./build/my-package-node.cjs",
            "default": "./build/my-package-node.modern.js"
        },
        "default": {
            "types": "./build/index.d.ts",
            "require": "./build/my-package-default.cjs",
            "default": "./build/my-package-default.modern.js"
        }
    },

    ...

And finally, update package.json#scripts to create the necessary builds:

{
    "scripts": {
        "build": "npm run asbuild && npm run build:compat && npm run build:node && npm run build:default",
        "build:compat": "microbundle build --target web --format modern,cjs --raw --no-compress --no-sourcemap --output build/my-package-compat.js --external ./release.wasm --alias ./wasm=./wasm-compat --define WASM_BASE64=`base64 -i build/release.wasm`",
        "build:node": "microbundle build --target node --format modern,cjs --raw --no-compress --no-sourcemap --output build/my-package-node.js --external node:fs/promises --alias ./wasm=./wasm-node",
        "build:default": "microbundle build --target web --format modern,cjs --raw --no-compress --output build/my-package-default.js --external ./release.wasm --alias ./wasm=./wasm-default",
        "clean": "rm build/*",
        ...
    },

    ...

NOTICE: If you chose to use JavaScript instead of TypeScript, the --alias flag should include ".js" extensions.

Running "npm run build" now, Microbundle compiles each variation into the build/ folder. Inspect my-package-*.modern.js and notice that each variation is bundled into a single JavaScript file containing the appropriate WASM loader.

Node.js, Web applications, and modern bundlers should find the right version of the package by default. Users with older bundlers should explicitly import from my-package/compat to get a self-contained module, with the WASM binary embedded as a base64 string. This embedding adds +33% file size and some parsing overhead, so it's just a fallback for these older tools.

We now have a fully-functional package, ready for npm, with basic support for all major JavaScript platforms. If the package will require different implementations or functionality on each platform, the same approach can be extended without further duplication.

Complete code from this tutorial is available on GitHub:

The articles below offer a deeper look at writing and optimizing AssemblyScript code:

Thanks for reading, and feel free to reach out with any questions!

1 Not a security concern, but error handling in WASM is sometimes opaque, and tends to increase the size of the WASM binary more than equivalent error handling in JavaScript would require.

2 As of this writing, DOM manipulation or calling external JavaScript dependencies would be common examples.

3 Or JavaScript modules. Add ".js" extensions to your import statements, if so.

4 As much as I wish that support for ES Modules and TypeScript were table stakes for a testing framework, my dear reader, it is not.

5 You may see a warning, "ExperimentalWarning: Custom ESM Loaders is an experimental feature". Node.js support for anything related to ES Modules continues to not be great, and we're including CommonJS fallbacks in our published package to deal with that. We'll ignore this warning from the test suite though, and hope for better support in upcoming Node.js versions.