9

I maintain a JavaScript library that is published on the npm registry and it has lots of dependencies. It gets difficult to keep track of what part of the code depends on what external packages.

Unfortunately neither lerna, yarn's workspaces, npm link, or npm's local path dependency declaration help. (I explain why after the example.)

I want to be able to break down the dependencies list declared in package.json by extracting some of the dependencies into new "sub-packages".

So, instead of having the following dependency list

// ~/code/example-lib/package.json
{
  "name": "example-lib",
  "dependencies": {
    "lodash": "*",
    "request": "*",
    "chalk": "*",
    "bluebird": "*",
    "mz": "*",
    "moment": "*",
    "socket.io": "*",
    "socket.io-client": "*",
    "react": "*",
    "react-dom": "*"
  }
}

I want to extract some of the dependencies into a new local package example-lib-subpackage. With local I mean that example-lib-subpackage is only meant to be consumed by example-lib.

example-lib-subpackage's dependency list would be;

// ~/code/example-lib/packages/example-lib-subpackage/package.json
{
  "name": "example-lib-subpackage",
  "dependencies": {
    "lodash": "*",
    "request": "*",
    "bluebird": "*",
    "moment": "*",
    "socket.io-client": "*",
    "react": "*",
    "react-dom": "*"
  }
}

and example-lib's dependency list would then be considerably reduced to;

// ~/code/example-lib/package.json
{
  "name": "example-lib",
  "dependencies": {
    "chalk": "*",
    "example-lib-subpackage": "./packages/example-lib-subpackage",
    "mz": "*",
    "socket.io": "*"
  }
}

Note how example-lib now depends on the local package example-lib-subpackage;

  ...
  "name": "example-lib",
  "dependencies": {
  ...
    "example-lib-subpackage": "./packages/example-lib-subpackage",
  ...

Has anyone achieved this? It would be super convenient.

Note that lerna and yarn's workspaces feature only help if you are ok with publishing the local packages to the npm registry. But in my case publishing the local package example-lib-subpackage to the npm registry doesn't make sense.

Also, npm link and npm's local path dependency feature only work for packages that aren't published but example-lib needs to be on the npm registry.

Local paths [...] should not be used when publishing packages to the public registry.

Quote from https://docs.npmjs.com/files/package.json#local-paths

tk421
  • 5,775
  • 6
  • 23
  • 34
brillout
  • 7,804
  • 11
  • 72
  • 84
  • Why aren't you using [`devDependencies`](https://docs.npmjs.com/files/package.json#devdependencies)? That looks basically like what you're trying to do. – Patrick Roberts Nov 13 '17 at 18:08
  • @PatrickRoberts Because these aren't devDependencies and they should be installed when the user installs the package. – brillout Nov 13 '17 at 18:32
  • If those are the packages required to build the dst of your code, then they are `devDependencies`. Otherwise what you're trying to do, as prevented by `npm` intentionally, is considered an antipattern. – Patrick Roberts Nov 13 '17 at 21:04
  • @PatrickRoberts Ok I see where you come from. The `dependencies` is listing building libs such as babel and webpack because `buidnserve` is itself a building library. Hence the `buildnserve` user would add `buildnserve` to `devDependencies`. This is what we want; It is the user of the library that decides whether the lib is a "real" dependency or only a dependency when developing the code. – brillout Nov 14 '17 at 15:34
  • Ah, thank you for mentioning that. That probably should have been mentioned in your question, since that's relevant to how you should approach your dependency organization. Perhaps you could publish a github repository and link your build code to the repository without publishing it independently on npm? There's not really a way to get around the local linking except by making the code you want to subdivide publicly available in one way or another, as far as I can see. – Patrick Roberts Nov 14 '17 at 18:48
  • Ok I now see my example to be confusing as well. I changed the example. Thanks! I don't think there is a npm built-in way of doing it either. But maybe there is a tool similar to lerna but based on npm's local path feature and that glues the subpackage's dependency list to the parent's one upon npm publish – brillout Nov 14 '17 at 20:52

2 Answers2

5

Since package.json is just a JS object, you might extend it before publishing to NPM.

On prepublish:

  • update package version
  • remove package.json local example-lib-subpackage dependency
  • extend example-lib dependencies with the one declared in example-lib-subpackage
  • optionally run a few tests over your updated package.json
  • publish
  • revert original dependency object
  • commit new version

PouchDb follows a vaguely similar approach described more in detail here and here.

Andrea Carraro
  • 9,731
  • 5
  • 33
  • 57
  • Could be a fun project to implement a lib doing that, let's see I find the time to do it. As for PouchDb, they seem to come from a very different place and have different goals. – brillout Nov 28 '17 at 21:48
  • Of course It does. But in opinion the team produced a priceless documentation and examples about how to handle shared dependencies between parent and sub-packages in monorepo projects. – Andrea Carraro Nov 28 '17 at 22:00
3

I was thinking you could use a build tool to maintain multiple package.jsons and have them compile down to the real one - but you'd be fighting the platform the whole way. You'd have to have your own CLI for installing, and it would be a mess.

You say:

Note that lerna and yarn's workspaces feature only help if you are ok with publishing the local packages to the npm registry. But in my case publishing the local package example-lib-subpackage to the npm registry doesn't make sense.

I don't think you are going to find a solution that makes perfect sense (if you're going down the route of trying to do completely nonstandard things with npm), and I am curious why you are ruling out breaking out example-lib-subpackage into its own repo - that seems like the obvious solution.

Sam H.
  • 4,091
  • 3
  • 26
  • 34
  • `compile down to the real one` Yes thought about that as well and to use npm's local dependency feature for declaring the dependencies. It would avoid your concern of `fighting the platform the whole way`. @toomuchdesign's answer elaborates quite nicely on that. – brillout Nov 28 '17 at 21:38
  • Maybe I'm missing something, but when the team wants to install new package or update, aren't you stuck manually editing json file? – Sam H. Nov 29 '17 at 02:25
  • The trick is to use https://docs.npmjs.com/files/package.json#local-paths for developing. And upon publishing you'd then resolve all local paths to the dependencies of the local packages. Npm would not see any local paths anymore and `Local paths [...] should not be used when publishing packages to the public registry.` wouldn't hold anymore. – brillout Nov 29 '17 at 08:05