Compiler plugins

A compiler plugin is an object with a name and a transform(ast, context) function. The compiler calls it once per module with the syntax tree and a context, and the plugin reports findings and edits the module through the context. It runs in the build, in your process, on your machine, and never in the browser.

import type { CompilerPlugin } from "@shadow-js/compiler";

/** Production build: removes every `data-testid` attribute so test hooks never reach users. */
export const stripTestIds: CompilerPlugin = {
  name: "strip-test-ids",
  transform(_ast, context) {
    if (context.dev) return;
    context.walk({
      JSXAttribute(node) {
        if (node.name.type !== "JSXIdentifier" || node.name.name !== "data-testid") return;
        let start = node.start;
        while (start > 0 && /\s/.test(context.source.charAt(start - 1))) start -= 1;
        context.remove({ start, end: node.end });
      },
    });
  },
};

Pass plugins in the plugins option. The edits are applied after transform returns, and the compiler parses the result again, so a plugin cannot hand malformed code to the next stage.

import { compile } from "@shadow-js/compiler";
import { stripTestIds } from "./strip-test-ids";

const source = 'export const Save = () => <button data-testid="save" class="btn">Save</button>;';

console.log(compile(source, { plugins: [stripTestIds] }).code.includes("data-testid"));
console.log(compile(source, { plugins: [stripTestIds], dev: true }).code.includes("data-testid"));

false
true

An absent or empty plugins list is the plain compiler: no tree is parsed for plugins, no map is composed, nothing extra is allocated.

Stages and order#

A plugin runs in one of two stages. enforce: "pre" (the default) runs on the tree of the source, before every compiler pass: JSX is still there, there are no _$ helper names, and useStore calls and derived constants are as written. enforce: "post" runs on the tree of the generated code: JSX is gone, the runtime imports and the hoisted templates are there, and TypeScript syntax outside JSX is still there.

Inside a stage, plugins run in the order of the array, and nothing else decides the order. Each plugin sees the module as the plugins before it left it. A list of [A (post), B, C] runs B, then C, then the compiler, then A, whatever the position of A in the array.

import { type CompilerPlugin, compile } from "@shadow-js/compiler";

function logger(name: string, enforce?: "pre" | "post"): CompilerPlugin {
  return {
    name,
    ...(enforce === undefined ? {} : { enforce }),
    transform(_ast, context) {
      console.log(`${name}:${context.stage}`);
    },
  };
}

compile("export const n = 1;", {
  filename: "n.ts",
  plugins: [logger("a", "post"), logger("b"), logger("c")],
});

b:pre
c:pre
a:post

The internal representation of the compiler is not part of the plugin API. A post plugin reads the generated code, which is the contract between the compiler and the runtime and does not change shape without a major version.

Reading the tree#

context.walk(visitor) visits the tree depth first, in source order, without recursion, so a deeply nested module cannot overflow the stack. The visitor has a function or an { enter, leave } pair for each node type in NodeMap (Identifier, Literal, CallExpression, JSXElement, JSXAttribute and others), typed with the shape of that node, and an untyped enter and leave for every node. An enter that returns false skips the children. context.is(node, "Literal") narrows a node, or an optional child, to its typed shape.

This plugin collects the keys of t("...") calls and reports each one. The keys are the plugin's own state, which it owns:

import type { CompilerPlugin } from "@shadow-js/compiler";

export function i18nExtractor(): CompilerPlugin & { readonly keys: string[] } {
  const keys: string[] = [];
  return {
    name: "i18n-extractor",
    keys,
    transform(_ast, context) {
      context.walk({
        CallExpression(node) {
          if (!context.is(node.callee, "Identifier") || node.callee.name !== "t") return;
          const first = node.arguments[0];
          if (context.is(first, "Literal") && typeof first.value === "string") {
            keys.push(first.value);
            context.report({ code: "I18N_KEY", message: first.value, severity: "info", node: first });
          } else {
            context.report({ code: "I18N_DYNAMIC_KEY", message: "t() needs a literal key.", node });
          }
        },
      });
    },
  };
}

Reporting#

context.report({ code, message, severity, node }) records a finding. severity is "warning" (the default), "info" or "error". Warnings and info come back in result.diagnostics, with the plugin's name in plugin and the position in the original source. An error stops the compilation after transform returns.

import { compile } from "@shadow-js/compiler";
import { i18nExtractor } from "./i18n-extractor";

const source = [
  'export const Card = (p) => <h2>{t("card.title")}</h2>;',
  'export const Save = (p) => <button>{t("card.save")}{t(p.key)}</button>;',
].join("\n");

const extractor = i18nExtractor();
const { diagnostics } = compile(source, { filename: "card.tsx", plugins: [extractor] });

console.log(extractor.keys.join(","));
console.log(diagnostics.filter((d) => d.code === "I18N_KEY").length);
const first = diagnostics[0];
console.log(`${first?.code} plugin=${first?.plugin} line=${first?.line}`);
const dynamic = diagnostics.find((d) => d.code === "I18N_DYNAMIC_KEY");
console.log(`${dynamic?.code} ${dynamic?.severity}`);

card.title,card.save
2
I18N_KEY plugin=i18n-extractor line=1
I18N_DYNAMIC_KEY warning

Editing#

All changes go through edits on the context. An edit names a span of context.source, and the offsets are those of the tree, whatever order the edits are made in.

A range is a node or any { start, end }. Two edits that overlap are an error (PLUGIN_EDIT_CONFLICT); replacing the same span twice keeps the last text; an insert at the edge of a replaced span stays outside it. The nodes a plugin "creates" are source text: context.create builds it with validated names and quoted strings (identifier, string, call, jsxAttribute, jsxText, jsxElement), and text that is not valid is reported with the plugin's name. A pre plugin that inserts JSX gets it compiled like any other JSX.

import { type CompilerPlugin, compile } from "@shadow-js/compiler";

/** Sorts the class names of a literal `class` or `className`, so the same set always reads the same. */
const sortClassNames: CompilerPlugin = {
  name: "sort-class-names",
  transform(_ast, context) {
    context.walk({
      JSXAttribute(node) {
        if (node.name.type !== "JSXIdentifier" || !/^class(Name)?$/.test(node.name.name)) return;
        const literal = node.value;
        if (!context.is(literal, "Literal") || typeof literal.value !== "string") return;
        const sorted = literal.value.split(/\s+/).filter(Boolean).sort().join(" ");
        if (sorted !== literal.value) context.replace(literal, context.create.string(sorted));
      },
    });
  },
};

const code = compile('export const A = () => <p class="wide  box primary" />;', {
  plugins: [sortClassNames],
}).code;
console.log(code.includes('class=\\"box primary wide\\"'));
console.log(!code.includes("wide  box"));

true
true

Imports#

A plugin that adds code usually needs an import, and the imports of the generated module are the contract with the runtime, so a plugin declares every import it adds, in imports, and adds it with context.addImport(from, name), which returns a local name that does not collide with anything in the module. An import that a plugin adds in another way, as text in an edit, must be declared too, or the compilation fails with PLUGIN_UNDECLARED_IMPORT. In the post stage, an edit that removes or changes an import of the runtime module fails with PLUGIN_IMPORT_MODIFIED.

import { type CompilerPlugin, compile } from "@shadow-js/compiler";

const tracking: CompilerPlugin = {
  name: "tracking",
  imports: [{ from: "@my/analytics", names: ["track"] }],
  transform(ast, context) {
    const track = context.addImport("@my/analytics", "track");
    context.insertAfter(ast, `${track}("loaded");\n`);
  },
};

console.log(compile("", { filename: "boot.ts", plugins: [tracking] }).code.trim());

import { track as _$p_track } from "@my/analytics";
_$p_track("loaded");

Errors#

A plugin error is a CompileError with the plugin's name in the message and in diagnostic.plugin, and the position in the original source. That covers an error that transform throws (PLUGIN_ERROR, with the thrown value as cause), a finding reported with severity "error", a plugin that returns a value or a promise (PLUGIN_BAD_RETURN: a plugin is synchronous and returns nothing), edits that overlap (PLUGIN_EDIT_CONFLICT), edits that leave a module that does not parse (PLUGIN_INVALID_OUTPUT), more than 100000 edits in one call (PLUGIN_TOO_MANY_EDITS), and the import errors above.

import { CompileError, type CompilerPlugin, compile } from "@shadow-js/compiler";

const noAlert: CompilerPlugin = {
  name: "no-alert",
  transform(_ast, context) {
    context.walk({
      CallExpression(node) {
        if (context.is(node.callee, "Identifier") && node.callee.name === "alert") {
          context.report({ code: "NO_ALERT", message: "alert() is not allowed.", severity: "error", node });
        }
      },
    });
  },
};

try {
  compile("export const f = () => {\n  alert(1);\n};", { filename: "f.ts", plugins: [noAlert] });
} catch (error) {
  if (error instanceof CompileError) {
    console.log(error.diagnostic.code);
    console.log(error.diagnostic.plugin);
    console.log(`line=${error.diagnostic.line} column=${error.diagnostic.column}`);
    console.log(error.message.includes('plugin "no-alert"'));
  }
}

NO_ALERT
no-alert
line=2 column=2
true

A compile error that a plugin did not cause, such as INVALID_HTML_NESTING on markup a plugin changed, keeps its own code, and its position is traced back to the original source.

Source maps#

Every edit is tracked, so the map points at the module as it was written. A replaced identifier maps to the original, and its original name is kept in names; text after an edit maps to its own place; inserted text has no mapping of its own and resolves to the place it was inserted at. The maps of the pre edits, of the compiler and of the post edits are composed into one, and sourcesContent holds the original text. With sourceMap: false no map is built.

import { type CompilerPlugin, compile } from "@shadow-js/compiler";

const rename: CompilerPlugin = {
  name: "rename",
  transform(_ast, context) {
    context.walk({
      Identifier(node) {
        if (node.name === "foo") context.replace(node, "renamedValue");
      },
    });
  },
};

const source = "const foo = 1;\nexport const answer = foo + 1;\n";
const { map } = compile(source, { filename: "a.ts", plugins: [rename] });
const parsed = JSON.parse(String(map)) as { names: string[]; sourcesContent: string[] };
console.log(parsed.names.join(","));
console.log(parsed.sourcesContent[0] === source);

foo
true

The compiler's own text passes that keep no map of their own (the useStore cell rewrite and the derived-constant pass) keep line numbers; on a line they rewrote, a column can be off.

Rules for plugin authors#

• Deterministic. The same module and options must give the same edits: build caches rely on it. Keep state across modules (the keys of an extractor) in the plugin and reset it yourself.
• Synchronous. transform returns nothing. Read what you need before the build starts.
• Declared. List every import in imports.
• Located. Pass a node to report, so the finding has a line and a column.
• Trusted. A plugin runs with the privileges of the build process. Install plugins the way you install any build tool; the compiler does not load plugins by name and does not sandbox them.