In Node.js development, it is a common practice that dependent modules use the same library as their dependency but with different versions in each separate module. Usually, it is not a problem thanks to NPM tree resolution. The NPM will try to minimize the number of installed copies doing deduplication. However, when you have to configure or patch some installed libraries, you are likely to face a problem.
For example, I have a custom logger module that uses the popular "debug" module features, such as pretty print and per-module log configuration. Let's run npm-ls debug.
Have you ever seen something like this below ("debug" versions)?
│ │ └── debug@2.6.9
│ ├─┬ send@0.18.0
│ │ └── debug@2.6.9
│ └─┬ serve-index@1.9.1
│ └── debug@2.6.9
├─┬ @nestjs/platform-express@8.4.7
│ ├─┬ body-parser@1.20.0
│ │ └── debug@2.6.9
│ └─┬ express@4.18.1
│ ├── debug@2.6.9
│ └─┬ finalhandler@1.2.0
│ └── debug@2.6.9 deduped
├─┬ axios-debug-log@0.8.4
│ └── debug@4.3.4 deduped
├── debug@4.3.4
Now, I want to configure the "debug" module to set up custom formatters and my logger as the debug output:
import createDebug from 'debug';
createDebug.formatters.O = function (v) {
// ...
}
createDebug.log = (...args) => {
// my Logger.log();
};
Next, I want to debug how "express" is working in my Node.js app, for example:
> DEBUG=express:* node app.js
Unfortunately, my configuration doesn't work because "express" uses "debug@2.6.9", but I've configured my app for "debug@4.3.4".
How to fix the issue with diffferent debug versions
Here are possible ways to fix this issue:
1. Upgrade dependencies (through "@nestjs/platform-express", for example)
If I'm lucky enough, my upgrade operation will result in some "express" sub-dependency version, with the "debug@4.3.4". However, it's nearly impossible to find a working combination of both modules that use the 4.3.4 version of the "debug". Also, if there are two or more different versions specified in any of the modules, the version of dependency will change again over time. And I should keep in mind that other modules need to be updated too.
2. Change the app's "debug" version (directly or by setting up a range)
I thought this would work but it didn’t in fact. When I downgrade my version to "2.6.9", it becomes the same as the "express" used and deduplicated, and I have yet another module with the "debug@3.2.7". The problem is still here.
3. Try to add an implicit dependency as the explicit to the "package.json"
It can help if there are compatible version ranges between target dependencies, and then the NPM can install and dedupe the version I set exactly in my "package.json". But in my case, some dependencies use exact "debug" versions too, so copies will remain.
4. Configure each instance of the "debug"
For example, this way:
import createDebug from 'debug';
import createDebug_2_6_9_1 from 'express/node_modules/debug';
import createDebug_2_6_9_2 from 'body-parser/node_modules/debug';
function configure(createDebugInstance) { ... }
[createDebug, createDebug_2_6_9_1, createDebug_2_6_9_2].forEach(configure);
This code works, but it is as weird as ineffective and weak. As you see, even the same "debug" version isn’t deduplicated if it doesn't match the project's "package.json". I have to find all versions of "debug" in my app dependencies. If I write a Node.js library, the users of my library should do the same. And the worst thing is that they should do it on each update of their dependencies, as the "debug" dependency version can change or some new version instance may appear in some dependency.
If I wrote a Node.js library this way, this would cause a real headache for my users because they would have to сonfigure each instance of the "debug". And the worst thing is that they should do it on each update of their dependencies, as the "debug" dependency version can change or some new version instance may appear.
5. Use "package.json" field "overrides"
As you see, I should get the same version of the "debug" in each nested dependency of my module. Before it was possible doing tricky manipulations with a custom module, like npm shrinkwrap or npm-force-resolutions. Fortunately, now a better solution exists — the built-in "overrides" config in the "package.json".
Since NPM v8.3 has been released, it is possible to control module versions for each of the nested dependencies in the dependency tree. The field "overrides", which provides this feature, is usable to:
- patch modules that are known to have a security issue (see "npm audit");
- replace an existing nested dependency;
- make sure the same module version is used by all dependencies.
The "overrides" allows the specification of the exact version, as well as a range like "debug@>3.2.7 <=4.3.4". In my case, it is required to have exactly one version, so the final working solution is this in "package.json":
{
"dependencies": {
"debug": "4.3.4"
},
"overrides": {
"debug": "$debug"
}
}
Simple and clean. The "$debug" variable describes how to use the module's dependency version, and it is the recommended way to specify an override.
Let's re-install modules and check npm ls debug again to ensure the version is locked exactly:
│ │ └── debug@4.3.4 deduped
│ └─┬ serve-index@1.9.1
│ └── debug@4.3.4 deduped
├─┬ @nestjs/platform-express@8.4.4
│ ├─┬ body-parser@1.20.0
│ │ └── debug@4.3.4 deduped
│ └─┬ express@4.17.3
│ ├─┬ body-parser@1.19.2
│ │ └── debug@4.3.4 deduped
│ ├── debug@4.3.4 deduped
│ ├─┬ finalhandler@1.1.2
│ │ └── debug@4.3.4 deduped
│ ├─┬ send@0.17.2
│ │ └── debug@4.3.4 deduped
│ └─┬ serve-static@1.14.2
│ └─┬ send@0.17.2
│ └── debug@4.3.4 deduped
├─┬ axios-debug-log@0.8.4
│ └── debug@4.3.4 deduped
├── debug@4.3.4
This procedure may apply to other modules too (in this case, the "body-parser" has copies) until the best result. The described override works because of the nature of the "debug" module, but other modules may have some breaking changes between versions. But it can be overcome, if necessary, by mixing the "overrides" and, possibly, some coding. See details about more complex overrides in npm Docs.
Conclusion
Also, one important note: if you are updating dependencies and you have "package-lock.json" created by NPM, you should delete it before installing, as well as it is better to remove "node_modules" too. As NPM will try to re-create the dependency tree, described in this file, and do the modification over it. And the resulting dependency tree will not be the same as you get on the clean install.
Happy coding!