DX Challenges of TS/JS Projects

Kicking off a blog series about managing TS Projects

Well… It has actually been more than one year since my last blog. Back then I bid farewell to programming for a while, and had plans to switch to Rust development. Fast forward more than a year, I am back working with TypeScript, and haven’t written any production Rust code yet. While I still have high hopes for Rust, it was just more convenient for me to stick to something I have a lot of experience with, while also trying to learn from things that Rust does well. Anyway, I digress.

What I want to do right now is to start with a small blog series, in rubber duck debugging style, to documenting my thoughts and opinions on how to manage JS/TS projects, and maybe in the process coming up with some viable solutions to the problems we face.

Let’s start explaining the challenges we currently face by giving some statistics on our codebase.

According to tokei, we currently have ~4000 source files with >300kLOC, excluding external dependencies, tests and automatically generated code. We are going all-in on TypeScript and use it for all the new code we write.

A tsc invocation gives the following statistics right now:

Files:           3780
Memory used: 1060813K
Total time:    31.56s

Running yarn to install all external dependencies takes ~50 seconds and pre-compiling ~1500 of our source files another ~18 seconds

Our unit and integration test suites run for ~3 minutes each, with another ~12 minutes for our end-to-end tests running in cypress (on CI). BTW, we have a mixed opinion on cypress, mostly related to their inability to upgrade their browser to a version that is more recent than electron 1 *sigh*.

Running a full lint with both eslint and tslint takes ~2 minutes.

And since we are creating both more code, and more tests, things will only get slower.

For local development, I now switched from vim to vscode, with which I am super happy in general. If it were not for the TS language server. I showed some stats above that running the TS typechecker itself takes ~30 seconds and consumes some ~1G of RAM. The language server itself, combined with the typescript-tslint-plugin plugin takes an incredibly long time to start, while consuming in excess of 2G of RAM. In my last blog post I was lamenting the way JS apps waste resources like CPU and RAM. Well today I am using more than 8G of my 16G just to run a browser, IDE, email client and desktop environment, all written in JS *sigh*.

But that isn’t even that much of a problem, if at least it would be stable. Hint: It’s not! While I do love all the code assist features, including the integration with tslint, it frequently takes ages to give me code-completion or even expand snippets. Worst of all, I have reached a level of slowness where switching branches would reliably kill tsserver after a minute due to OOM *sadface*. By now this starts to slow me down significantly.

Also, it kind of bothers me that these separate tools, such as IDE integration, linting (for example on pre-commit) and actually running your code share no state, and thus are doing a lot of redundant work that makes them slow. More on that in a later post.

Apart from these problems that revolve around DX tooling, we also face problems when pushing code into production, related to building / bundling the code and around how to deal with external and internal dependencies.

This is it for now. In the followup posts, I want to go into more detail on how I would like to better manage these challenges.