Speeding up the JavaScript ecosystem - Isolated Declarations

Unbeknownst to many, the new isolatedDeclaration feature shipped in TypeScript 5.5 is much more important than you might realize. It just revolutionized how we package and distribute JavaScript code. You don't need to create *.d.ts files manually anymore by invoking the tsc compiler. "Go to source" (the thing when you do ctrl+click or cmd+click on macOS) actually works now and it leads you right to the TypeScript source code instead of a *.d.ts file or some compiled JavaScript code. And on top of it all it makes publishing packages way faster than ever before.

How did it achieve that? Let's take a step back and assess the current situation of packaging JavaScript code in 2024.

Packaging for npm in 2024 is a mess

Honestly, it's a mess. There is no point in sugar-coating it. A buddy of mine wanted to publish his first every library on npm, but was quickly discouraged when he realized how much work and specialized knowledge that entails. You have to care about CommonJS vs ESM, fiddle with a bunch of settings to make *.d.ts files work and the list goes on. In the end, he aborted the mission and instead just copy and pasted files between his projects around. It was much less of a hassle compared to having to deal with all the packaging woes.

Packaging code should be as easy as the act of uploading files. Packaging and sharing code should be so easy that every developer, no matter their experience level, can do it.

Even for experienced developers, creating npm packages that can be used in every environment is quite a challenge. Mark Erikson from the Redux team wrote a blog post spanning multiple pages about his experience of creating npm packages. The thing is that packaging and sharing code among developers should be easy. Developers shouldn't have to read a dozen blog posts and become experts of the inner workings of all these tools to be able to share code. It should just work.

Various CLI tools emerged to help with this situation, but this just adds another friction point. You got tool A which can fix a part of the problem, then you have tool B which only deals with another piece of the problem and so on. Before you know it, you have to stay on top of which tool is the right one to use, up until it will be replaced by yet another tool. It's bad.

So, how do we fix all of this?

Why generating .d.ts files takes so much time

At the heart of the problem is an architectural one. We only ever ship build artifacts, the compiled JS and the relevant .d.ts files in a package. Shipping the original TypeScript source files would be much better, but barely any tool in the JS ecosystem works with that. The assumption that nothing inside the node_modules folder needs to be compiled is too widely baked in our tools. Shipping the TypeScript sources instead of .d.ts files would also be a performance regression when working with TypeScript.

Parsing .d.ts files is much quicker as they contain only the bits needed for type checking compared to a normal .ts file. A .d.ts file contains no function bodies, or other stuff, just the type definitions that are needed for consuming a module.

// Input: add.ts
const SOME_NUMBER = 10;
export function quickMath(a: number, b: number) {
	return a + b + SOME_NUMBER;
}

// Output: add.d.ts, contains only the bits needed for type checking
export function quickMath(a: number, b: number): number;

Oftentimes, the return type of a function is not known. It has to be inferred by the TypeScript compiler by walking and checking the whole function body first. Inference is a very costly thing to do in terms of performance, especially for complex functions. Therefore the goal of creating .d.ts files is to get rid of all inference, so that the TypeScript compiler only needs to read these files and do no additional work. Type inference is the main reason creating those .d.ts files takes so long. And because this process takes so long, we try to do as much of it ahead of time, when creating the npm package.

All this changes with isolated declarations.

Entering a new era of packaging

The idea behind isolated declarations is that you make it trivial for tools other than the TypeScript compiler to generate those .d.ts files from TypeScript source code. It does it by requiring explicit return types for exported functions and other things. But don't worry, if a type is complex to reason about, the TypeScript language server comes with a handy feature to infer the missing type for you.

By turning the process into a mere syntax stripping one, the time to create .d.ts is very close to 0s. With a specialized Rust-based parser this happens in the blink of an eye even for gigantic projects as that work can now be parallized. It's so fast that you don't even notice it. Without isolated declarations you always had to invoke the tsc compiler and run a full type checking pass.

Since the cost to generate definition files became next to nothing, it's easily fast enough to do on the fly when needed. We can flip the process of publishing on its head: Instead of dealing with all the hassle of building .d.ts files and publishing them to npm ahead of time, you can just upload your TypeScript source code as is, and the definition files are automatically generated on the fly whenever you install a package. Generation of the .d.ts doesn't occur when publishing a package, it happens when you install a package.

This also changes how you declare the available entries for your package. Previously, with npm you had to reference a couple of build artifacts to make this work.

// package.json
{
	"name": "@my-scope/my-package",
	"version": "1.0.0",
	"type": "module",
	"exports": {
		".": {
			"types": "./dist/types/index.d.ts",
			"import": "./dist/index.js"
		},
		"./foo": {
			"types": "./dist/types/foo.d.ts",
			"import": "./dist/foo.js"
		}
	}
}

With a system based on isolated declarations this is much simpler: Just reference your source files directly.

// jsr.json
{
	"name": "@my-scope/my-package",
	"version": "1.0.0",
	"exports": {
		".": "./src/index.ts",
		"./foo": "./src/foo.ts"
	}
}

Pointing the entries of your package to the actual source files feels much more natural as it frees you from worrying about packaging artifacts.

Using it in production

Now of course, talking about things in theory is all good and well, but how does such a thing feel in production? The core idea behind isolated declarations have been floated around various times in the TypeScript issue tracker, but it took some time and many smart minds to finalize. Here at Deno (disclaimer: I work for Deno) we've been following that process from beginning very closely and starting working on our own rust-based version of that. The system that we've been talking about so far in this post, is exactly how packaging in Deno works since early December 2023. It's been used in production by every Deno user for more than half a year by now.

Things like "go to source" in your editor work out of the box and actually move you to the source code rather than some random definition file.

We realized the usefulness of having such a system and made it work for everyone, not just for Deno users too. You can use it with npm, with yarn, with pnpm or even with bun. The JSR registry does this by implementing the npm protocol for installing packages and simply acts a yet another npm registry when talking with these package managers. From the package manager's point of view it's no different than talking to any other private npm registry that you may be already using in your company.

Since npm clients don't have a system in place for generating definition files on the fly, we generate them for the npm tarballs during the publishing process behind the scenes. Therefore the npm tarball comes with the standard transpiled .js files like you're already used to from other projects. The whole npm ecosystem is based around shipping .js files and changing that would break the it. The takeaway here is not to only publish TypeScript to npm.

But if you do use a runtime that can run TypeScript natively and use a registry that can also distribute the original TypeScript sources, then having isolated declarations is a game changer.

Conclusion

Isolated declarations just changed the game of publishing forever. It made it a near zero cost process to generate definition files, which opens the door for generating definition files on the fly. This greatly simplifies the publishing process to a mere upload of source files, which is much quicker to do. Right now, JSR is the first and only registry to take advantage of that and it can be used with any package manager and in any runtime. In a nutshell, JSR is aimed at everyone who feels like life is too short to deal with all packaging woes and just wants to share code. Try out it out and see for yourself.