Honey, I shrunk the node_modules! ...and improved app’s performance in the process. On node module size

When working with Node.js and npm, one of the common worries is npm package size and the size of your Node modules. Are you in the search for a solution? Which library to use? Which tool? Check out my experience and get your module size down!

What you’ll learn

The struggle is real, and believe me, at TSH we’ve been there multiple times. But fear not, something can be done to reduce that size! We’ve created a few simple steps to reduce its size. In this practical overview, I’m going to walk you though these steps, which include:

1. Visualizing dependencies on the graph.
2. Using production flags with your npm installs.
3. Removing unnecessary files.
4. A little something that I like to call: seeking & cleaning.

Just follow me, please…

Our node_modules backstory

Not so long ago, we’ve been working on this British fintech project, where we had to migrate the system to a microservice architecture using Node.js. When we were introduced to the platform, we got two requirements from the client – it had to be efficient and light. Performance is not a problem for Node.js (unless you accidentally block the event loop), however, lightweight can be. The starting image was 32MB, which is a very nice result, but it quickly turned out that the real problem is node_modules.

Contrary to what you may think, the size of the application is important in a couple of areas:

• performance  – an application that is light and runs faster, takes up fewer resources so it works more efficiently during development,
• costs  – some light apps require fewer resources which means that the infrastructure costs drop and you save up a lot of money.

Developers like to simplify their work to do everything faster  – that’s why they include additional libraries. Unfortunately, this results in an increased size of the entire application that can go out of control very quickly.

This is how our adventure with “slimming down” the thing began. So what did we do to shrink them? Reduce the number of dependencies

This one sounds obvious but before we get into more invasive solutions, maybe you should think about the packages you install. Do you really need all of them?

At TSH we are using sites like npm.broofa or npm.anvaka to actually see a whole graph of dependencies for a single package and then we discuss if this is something we’re looking for.

A lot of times I see people installing Jest just for simple unit tests (about 460+ dependencies, +/-60MB) when there are smaller libs that will do exactly the same (Mocha – 115 dependencies, 12MB).

I know what you’re thinking – it’s just a dev dependency. Yeah, it is, yet by reducing the number of modules you also speed up your development machine. Fewer things stored in memory, fewer libs to follow by IDE all of this makes us develop faster. It all comes to simple props and cons. In our case, we started with 700MB of dependencies and after a few exchanges and removals, we ended up with just 256MB.

Use production flag

Another obvious (but sometimes forgotten) method is to use `–production` flag on npm install. Generally speaking, of all places, you don’t want to install unnecessary Node.js modules on production the most.

Using the production flag will skip all of the devDependencies and use production ones only. Trust me it’s worth it. First, you need to separate your devDependencies from other dependencies in the package.json file. When you are building with a docker-composer file, you can instead pass build argument as production to achieve a similar effect. It goes like this:

npm install --production

Most of our projects have seen +-33% reduction of node_modules size after we’ve used it (176MB).

Remove unnecessary files

Have you ever thought about stuff being installed when you type npm install? I’m not talking about the whole tree of dependencies, but a lot of trash that’s inside, down the path name. Docs, tests, markdowns, images, sources, a lot of files that aren’t useful during development at all (tell me when was the last time, you dived into node_modules to read docs, huh?).

At TSH we found two cool libraries – node-prune and ModClean. Both of them have the same purpose to remove everything that is not necessary for a package. ModClean comes with three patterns – safe, caution and danger – and sometimes removes too many files.

That’s why most of the time we use node-prune because it’s not only safer but also it allows us to reduce node_modules by another 30% from the production version.

For better visualisation: we started with 256MB of those modules for dev version. After production we went down to about 176MB and then after node-prune, we are at about 126MB. It’s more than half of initial size!

See also: Dependency injection in Node.js

Seek and clean

Even after using node-prune, there’s still something that can be done. At some point, we started to check the size of each module, to see which ones are the largest. There is a very simple command in case of MacOS and Linux:

It’ll print every module that has a size of at least 1MB. By doing this you will know which modules take up the most space. It was quite helpful because until then we didn’t realize that RxJS / gRPC and AWS aren’t so slim. What’s more, you can use it on a specific module to find which directory is to blame. This allowed us to find that RxJS has a version for ts, es5, es2015 and UMD package, even though we’re using dist directly.

Those additional packages took 7MB from 11MB of the whole package. More than 50%!

The same story was with gRPC (there was a directory of multiple third parties that we didn’t use at all – about 12MB) and AWS (you get a version for web and native bundled – about 10MB). Because of that, we created a special shell script that did the cleaning.

By doing it we went down by another 17%, so from 126MB to about 104MB.

Node module size – the final result

We started with 700MB of dependencies. Then we did some chopping and left only the modules we actually needed. The rest of them were exchanged for smaller modules or even replaced by our custom code. We ended up with 256MB. After that, we used the production flag and made it into 176MB. By using node-prune we could go down to 126MB and then cut it down to 104MB after some additional cleaning.

From 700MB to 104MB! It sounds great, doesn’t it?

Believe it or not, but all of those steps (except production flag) are usable on dev version too, so we tried that as well. We ended up with 161MB for dev version. It’s less than using production flag only.

And just like that, we lost 539 MB in total!

So remember:

Your orca-sized node_modules are not doing any favour to your application – they worsen the performance, slow down the processes, and devour infrastructure (which means you virtually lose a lot of money on additional resources).

Of course, there are also other methods of reducing Node.js modules worth exploring. I highly suggest you take a look at concepts such as ignore path, ignore type, Rust power, pretty bytes, cwd type, path cover options, API nodemodulessize, match type, match path use or dir context options match. All in all, I highly recommend slimming down the modules in your project for the sake undeniable of business benefits.

And if you’d rather like a specialist to take care of it, you can give our Node.js team a shout. Good luck!