Thoughts on yarn

Yarn announced itself as a new package manager for JavaScript. In fact is a better npm client but its release suggests that change may be coming to JavaScript package management with issues of security and centralisation up for debate.

Yarn is a better npm client

Yarn announced itself at a new package manager for JavaScript. Eventually the project may look at creating a registry but for now it is simply a better npm client. It is a drop in replacement for the official npm client and offers a global cache and better parallelisation. From my crude tests and anecdotal reports from friends and colleagues it is much faster and more efficient than the standard npm client.

The problems outlined in the announcement are largely related to deployment and scaling the npm client in enterprise settings. Yarn seems to have re-solved some of the problems that the npm client has already addressed. Switch npm-shrinkwrap for a lockfile for example. It seems a shame that there was no aspiration to contribute to the open source npm client but multiple clients are no bad thing.

Given developers want reliability and speed if the yarn project can succeed in improving download speeds and reliability I fully expect it to become the default. More interesting for me is that yarn is an expression that not everything is rosy in the JavaScript package management ecosystem.

Tiny modules exploding

It is indisputable that npm has been a runaway success. The platform has seen explosive growth, has scaled and has become a daily tool for software developers.

150 new authors are now publishing packages every day.

The explosive growth is not surprising given that small modules are favoured over large monolithic code bases but the growth is indisputable and impressive. The npm service has matured to become stable and able to scale.

Tiny modules, big dependency trees

The Node.js ecosystem has promoted writing tiny modules. Along with the upside of tiny reusable modules comes massive dependency trees. Have a look at the yarn dependency graph as an example.

Yarn dependency tree
Yarn dependency tree

I have seen the large dependency trees cause build servers to lock up or fail which is frustrating when you are trying to ship code. Furthermore package versioning can lead to brittle builds if you do not specify exact versions or use npm shrinkwrap.

With so many tiny userland modules there are issues of modules authors not following semantic versioning or just removing packages. One such example is left-pad a 52 line module for left padding a string. This module was removed from the registry by the author breaking deployment pipelines and a number of major projects that depended on it. The package was later restored and the fallout documented by npm. This incident led to many running their own npm cache to mitigate against deployments being disrupted.

Being centralised is a problem

The history of npm is one moving from a hobby project and one of the original Node.js package managers. Remember kiwi anyone? npm eventually became the default choice and coupled with Isaac Schlueter taking over as the benevolent dictator of the Node.js project it became official.

The npm project then took $2.6 Million in seed funding and became a for-profit organisation whilst maintaining an open source approach. This has been universally good for the Node.js ecosystem but has resulted in one for-profit organisation owning the hosting of the registry. The npm team have done a fantastic job in scaling the registry and maintaining an excellent service but I can’t help feeling that the release of yarn is an expression that this model is less than perfect.

Far better for me would be to have a decentralised registry that allows replication and mirroring around the world similar to the way major Linux distributions run their registries. This reduces reliance one on single registry provider and reduces issues of scale. Packages are verified by checksums and signing so files can be hosted and downloaded from anywhere. Far from ruining npm’s business model I feel there would still be opportunities to offer private registries and generate revenue. Perhaps we are too far down the road now though.

Security is still a problem

Whilst npm does perform sha1sum checking packages are unsigned. Often authors will declare dependencies with a greater than syntax. Using something like ~1.2.3 will install anything greater than 1.2.3 and less than 1.3.0. As such it would be quite feasible for an npm author account to be compromised and for a new package version to be released that is less than 1.3.0. This version would proliferate around project installs, build servers and production sites as npm install is run.

A package rimfafall was released to highlight this issue but the conclusion was ‘You are responsible for what you require’. With dependencies trees reaching to tens of thousands of modules this simply isn’t feasible.

Whilst signing packages has been discussed it has not progressed so package installers are still vulnerable to an npm account being compromised.

Understanding licensing is a problem

A while back I was asked to perform an audit on a relatively small project for an large enterprise who were interested in understanding the Open Source licences used in a project. There are packages available to output this information. Let us suppose that an organisation requires that no GPL licenses are used in a project. Given the epic dependency trees of projects it is very hard to know this and unless you lock versions of packages you install this can change as you run npm install.

Conclusion

Currently yarn is just a better npm client. The registry it uses is just a proxy to the main npm registry but the fact it does not point to the npm registry directly suggests that there may be some aspiration to include reworking the registry in the project.

The announcement of yarn is an expression of the maturity of JavaScript, Node.js and npm. Seen as a better npm client it offers a drop in replacement to resolve a number of issues experienced at scale. Seen as an expression that npm is not perfect it opens up potential flux and innovation in JavaScript package management. I would love to see package signing and a distributed registry addressed. Until then security and a single organisation owning the distribution of packages remain problems waiting to happen.

Tags

Can you help make this article better? You can edit it here and send me a pull request.

See Also