The missing link in JavaScript tools

Lately, my mind has been occupied with finding ways to make tools simpler to work with. Our tools of today present a pretty fractured architecture. The frontend layer has to deal with many languages at once, ranging from custom templating languages, importing CSS to JSX or plain JavaScript. Each tool, the linter, formatter, type checker, test runner and dev server typically demands its own unique plugin system and preprocessing logic to help it understand those workflows.

The template predicament

This makes it difficult to innovate frameworks and integrate them with today's tooling. Take .svelte files as an example. How do we teach our linter to understand them? What about type checking? Everything needs its own plugin. But why?

All of these plugins have in common that they get an input string and turn it back into valid JavaScript. To get type checking in Svelte, Vue or other custom languages to work, the code is transpiled in a way that TypeScript understands. The templating bits are converted to JSX and the rest to standard JS. Both JS and JSX can be type checked by TypeScript. When a type checking error is encountered we check the source maps, if it maps back to user code. If it does, report it, else ignore it.

<script lang="ts">
	export let name: string;
	let count: number = 0;

	function handleClick() {
		count += 1;
	}
</script>

<div>
	<h1>Hello, {name}!</h1>
	<p>You have clicked {count} times.</p>
	<button on:click="{handleClick}">Click me</button>
</div>

<style>
	div {
		border: 1px solid blue;
		padding: 10px;
	}
</style>

Pseudo example of what gets sent to the LSP (real world is different, but it's the same idea):

import { SvelteComponent, init, safe_not_equal } from "svelte/internal";

const __svelte_script = (() => {
	let name: string;
	let count: number = 0;

	function handleClick() {
		count += 1;
	}

	return {
		props: { name },
		data: { count },
		methods: { handleClick },
	};
})();

function render() {
	const { name, count, handleClick } = __svelte_script;

	return (
		<div>
			<h1>Hello, {name}!</h1>
			<p>You have clicked {count} times.</p>
			<button onClick={handleClick}>Click me</button>
		</div>
	);
}

TypeScript can perfectly type check this.

A Virtual File System as the Unifying Layer

Imagine a world where we could modify files before handing them off to the type checker or our linter. Something like a virtual file system of some sorts that everyone could tap into. Runtimes could even use this to run your code directly. You could run your code as if it would be any other script:

• deno foo.svelte
• node foo.svelte

At its heart, this virtual file system is a shared processing pipeline that has quite a lot in common with typical bundler APIs. Because if you think about it, resolveId() and load() + transform() neatly map to file system operations.

Bundler API	File System API
resolveId	follow symlink
load (+ transform)	readFile

The resolveId function would be reinterpreted as resolving symlinks within this virtual file system. This allows for dynamic mapping and aliasing of files. Similarly, the load and transform phases, where code is read and modified, can be thought of as a readFile call.

Cross-Language usage

Lots of tools are in the process of moving some of their pieces to Rust or other languages like GO. We're now in a situation where we need to account for multiple consuming languages. All of that could be solved by talking to the same processing pipeline via IPC.

Conclusion

I'm not sure when or even if such a thing will ever see the light of day. I would welcome it though, because it would make experimenting with new templating languages or other DSL's in the JS ecosystem much easier. I want to be able to take my pre-processing pipeline with me, regardless of which test runner, linter or formatter I want to use.