5

I use yarn to install packages directly from the companies' GitLab:

yarn add git+ssh://<user>@<host>:<repo>

For first-level dependencies, I use yarn --pure-lockfile to reconstruct my node_modules according to the yarn.lock.

However, for second level dependencies yarn seems to always install the latest version.

So let's say I depend on A which is tested with a specific version of B. In A's package.json I don't specify the version, but it is contained in the yarn.lock.

When I now install package A yarn will get the latest version of B despite the entry in A/yarn.lock

I know that I might resolve this by passing a specific version in A/package.json (at least I think).

But is there an option to tell yarn to look at the yarn.lock of dependencies?

hong4rc
  • 3,999
  • 4
  • 21
  • 40
user4344
  • 193
  • 1
  • 10
  • We have a similar thing. We have a build tool that we distribute as a library. The amount of dependencies (most of them transitive) is huge. – EECOLOR Jun 20 '18 at 22:15

1 Answers1

13

TLDR:

When you install dependencies in your application, only your own yarn.lock file is respected. Lockfiles within your dependencies will be ignored. Reference

Let's get some things cleared first:

  1. --pure-lockfile is same as normal yarn install except that it won't generate a yarn.lock file or update one if present.
  2. Yarn always reads from the yarn.lock by default for resolving dependencies while installing unless supplied with --no-lockfile. So, there is no need to tell it to read from yarn.lock.

What is yarn.lock used for?

yarn.lock is used for resolving what version should be fetched given the semver version of a module in package.json. It is not used to determine what semver version should a module be resolved to. That is simply not its use-case.

As mentioned in yarn DOCS: In order to get consistent installs across machines, Yarn needs more information than the dependencies you configure in your package json.. Yarn needs to store exactly which versions of each dependency were installed.

To do this Yarn uses a yarn.lock file in the root of your project.

So, for resolving semver version of a dependency, yarn always depends on package.json. For a given semver version, yarn checks the yarn.lock file to see what version should it fetch. This is what makes yarn Deterministic (Same tecknique is used by npm which uses npm-shrinkwrap.json).

Example: Semver Versions like ^1.2.4 can resolve to any version number which is >= 1.2.3 and < 2.0.0. Without yarn, npm would install 1.2.4 in one machine but 1.9.9 in some other machine, depending on the latest version present at the time of install. This is the problem that yarn solves using yarn.lock.

The semver version is determined by the package.json file. The yarn.lock file is only a lookup for which version and commit hash to install for the given semver version number.

How does yarn resolve version of a module given its semver version?

Suppose currently our yarn.lock file looks like this:

bluebird@2.9.6:
  version "2.9.6"
  resolved "https://<...>/bluebird-2.9.6.tgz#1fc3a6b1685267dc121b5ec89b32ce069d81ab7d"

bluebird@^2.9.30:
  version "2.11.0"
  resolved "https://<...>/bluebird-2.11.0.tgz#534b9033c022c9579c56ba3b3e5a5caafbb650e1"
...
myModule@5.1.0:
  version "5.1.0"
  resolved "https://<...>/moduleA-5.1.0.tgz#ce97130858add59d616ee80675383b0c127290a0"
  dependencies:
    bluebird "^1.0.0"
  1. If package.json has bluebird: "^2.9.30", yarn looks for an entry bluebird@^2.9.30 in the lockfile. It is present and hence is resolved to version=2.11.0.
  2. If package.json has bluebird: "^2.9.0", yarn looks for an entry bluebird@^2.9.0 in the lockfile. It is not present. Suppose the latest stable version which satisfies semver criteria is 2.13.0, then yarn adds an entry for bluebird@^2.9.0, resolved to 2.13.0. While resolving version for a given semver version of bluebird, it does not matter what entry is present for bluebird in moduleA's dependencies in the lockfile.

Semver Version is not affected by what entries are present in the dependencies map for a module in yarn.lock file.

So, if package.json has bluebird: "", yarn looks for an entry bluebird@ in the lockfile but is unable to find it. Hence, it resolves bluebird: "" to the latest version, suppose 3.5.0. Now, yarn will add an entry for bluebird@ resolved to 3.5.0.

bluebird@:
  version "3.5.0"
  resolved "https://<...>/bluebird-3.5.0.tgz#d9551f9de98f1fcda1e683d17ee91a0602ee2eb9"

Form now on whenever yarn encounters {bluebird: ""}, it will find an entry for bluebird@ in the lockfile and hence will always resolve it to 3.5.0.

Solution to your problem

To resolve B: "" to version say 1.0.0, you need to have an entry for B@ in yarn.lock resolved to 1.0.0. Once, yarn.lock has an entry for B@, all the successive installs will always fetch version 1.0.0 for B="".

Following are the steps you need to take to accomplish the same:

Approach 1 (Recommended)

If you want B to resolve to latest version:

  1. Add B:"" in A's package.json
  2. Run yarn install. This will add an entry for B@ resolved to latest version.
  3. Push yarn.lock file.
  4. Form now on, whoever runs yarn install will get the same version.

Approach 2

If you want B to have an older version: (Highly Not Recommended)

  1. Add B: 1.0.0 in A's package.json.
  2. Run yarn install. This will add an entry B@1.0.0 in the lockfile.
  3. Add B@ alongside B@1.0.0 in yarn.lock. B@, B@1.0.0: ...
  4. Change B's version to "" in A's package.json.
  5. Push yarn.lock file.
  6. Form now on, whoever runs yarn install will get the B's version as 1.0.0.

This approach is highly dangerous as you can break something easily. Your yarn.lock file should always be managed by yarn.

Approach 3 (Recommended)

If you want B to stay at 1.0.0

  1. Fix B's version to 1.0.0 in A's package.json.
  2. Run yarn install. This will add an entry B@1.0.0 in the lockfile.
  3. Push yarn.lock file
  4. Form now on, whoever runs yarn install will get the B's version as 1.0.0.

Edit: Using the yarn.lock file present in the dependencies

If you check this doc:, they have clearly mentioned that yarn will use only the top level yarn.lock file and ignore the lock files present in the dependencies.

There is currently no way of locking down second level dependencies using yarn.lock present in them. I don’t see any need for it. In fact the creators of yarn explain here why that is the case. The reasons being:

  1. The versions to be installed for second level dependencies can be captured well by the top-level yarn.lock file, as I have explained above.
  2. You would never be able to update the versions of sub-dependencies in your own application when using them directly because they would be locked by other yarn.lock files. You can verify this point by my explanation of how yarn resolves dependencies.
  3. Yarn would never be able to fold (de-duplicate) dependencies so that compatible version ranges only install a single version.

Also, as in your use-case, if A has a dependency B which works only with version 1.0.0, A’s package.json should have version mentioned for B as 1.0.0 and not “”. You can always fix your top-level yarn.lock to add an entry for B@ resolved to 1.0.0 but it is not recommended to manually fix a yarn.lock file as I have mentioned above.

Hope this helped! Please ping me in the comments for any doubts.

yeshashah
  • 1,536
  • 11
  • 19
  • First of all thank you for you very nice answer. This is however no solution that is usable in the real world. The actual problem is that we have a library (A) where we could specify a dependency (B) with a fixed version (1.0.0), B itself however also has dependencies that might not be locked down. – EECOLOR Jun 23 '18 at 19:27
  • Oops, accidental send... Anyway. We tested A in isolation, this means it has a yarn.lock file which contains all transitive dependencies that we are sure it works with. Now we want to use A with the versions of all of its dependencies as specified in its lock file. – EECOLOR Jun 23 '18 at 19:29
  • 1
    "I don’t see any need for it." - as soon as one of your transitive dependencies breaks the semver rules you will see the need. The only language (that I know of) where there would not be a need is Elm which has a compiler that checks semver. In one of our libraries we have a lock file of almost 10.000 lines, even if the chance is very small, breakage will happen. – EECOLOR Jun 24 '18 at 08:29
  • I understand your point. But take a look at this: https://yarnpkg.com/blog/2016/11/24/lockfiles-for-all/ According to James Kyle, When you install dependencies in your application, only your own yarn.lock file is respected. Lockfiles within your dependencies will be ignored. – yeshashah Jun 24 '18 at 10:51
  • Have also added an edit in my answer to reflect these understandings. Hope this helps. – yeshashah Jun 24 '18 at 11:04
  • Thank you for your time and dedication! If you answer the original question: "But is there an option to tell yarn to look at the yarn.lock of dependencies?" with "no" in the TLDR I can award the bounty to the answer :-) – EECOLOR Jun 24 '18 at 15:32
  • Hope this helped @EECOLOR. I have updated the TLDR section to answer the original question. – yeshashah Jun 24 '18 at 15:55
  • "The versions to be installed for second level dependencies can be captured well by the top-level yarn.lock file, as I have explained above." Yes, and at the same time you say you're not supposed to edit yarn.lock by hand. So if the wrong second level dependency version is captured, there's no "allowed" way to fix it. – Frans May 30 '21 at 12:06