35

I'm writing a number of pieces of code (for internal use) using node.js and want to store the modules (packaged up for npm) in a package repository for each distribution to the various machines they will be installed on.

Ideally, I'd like a solution similar to Debian's apt repositories in which I can run a private repository server and configure npm to use a list of repositories to install from (When installing "foo", if "foo" is known by my private server install it from there, otherwise install it from the public server).

However, it looks like the npm registry configuration key only accepts a single URL.

Is there a way to achieve what I want?

The closest I've been able to find have been:

  • Mirroring the public repository locally and adding my packages on top of it… but I don't want to keep that amount of data (2.5G and still downloading) replicated on AWS.
  • Hosting all my packages in git repositories and installing from there (which is more of a hassle).
  • Hosting static packages on HTTP (as far as I can tell, this would prevent me from automatically getting "the latest version". I suppose I could do something with symlinks, but that is still less flexible than git, requires full URLs (which need to be kept up to date), and doesn't give a searchable repository.
Quentin
  • 914,110
  • 126
  • 1,211
  • 1,335
  • 2
    I haven't personally done this, but my understanding is that you can set up a simple CouchDB database, copy only the design documents, (not a full replication) from the NPM registry database and make sure to use various [registry config options](https://npmjs.org/doc/registry.html) when you are dealing with your own internal modules. – Dominic Barnes Jan 30 '13 at 17:14
  • 1
    @DominicBarnes — That sounds ideal … any pointers to documentation on how to perform such a partial replication would be most welcome! – Quentin Jan 30 '13 at 17:18
  • 1
    Check out [this answer](http://stackoverflow.com/questions/7309617/how-to-replicate-design-documents-only), I've been curious about doing something like this for a while so let me know if I can help out. – Dominic Barnes Jan 30 '13 at 17:24

7 Answers7

37

I just set this up for my work. Here's what I did:

  1. Setup empty NPM registry: I followed the instructions from this fork of npmjs.org, which adds much improved documentation.

  2. Setup Kappa: I used Kappa, a great npm proxy from Paypal. (I'm guessing they have a very similar use case to most people who want a private repository; this was exactly what I wanted).

  3. Setup npm_lazy (optional): I wanted a nice cache of frequently used packages in case npmjs.org went down, so I added npm_lazy in front of the whole thing, as a caching layer.

Whole thing took two days(ish) to get up and running. As a side note, if you're worried about people pushing to the public registry by accident, I recommend adding this to your package.json:

"publishConfig": { "registry": "http://my-registry.example.com" },

This really is just a bit of paranoia; once you setup your npm to point to your Kappa/npm_lazy instance, Kappa handles publishing to your private repository for you.

Note: Kappa will only every publish to the first repository in it's config. If you need to publish to both your private registry, and the public, you will need to work out your own solution.

Chris Case
  • 236
  • 5
  • 9
  • I've experimented with creating a full mirror as well as this approach. I'm a fan of https://github.com/terinjokes/docker-npmjs as a way to deploy most of this, and would like to add the npm_lazy integration soon. Just be sure to back up your private repos if you publish them onto a (temporary) docker instance.. – Nate Mar 10 '14 at 08:11
7

In your package.json, you can use any url that points to a valid npm packed module. I use an s3 with a bucket name that is hard to guess.

npm pack
s3cmd put *.tgz s3://path-to-your bucket

S3 is just an example, you could use any mean that can place a file on a web server, it can even be protected via basic auth.

Pascal Belloncle
  • 11,184
  • 3
  • 56
  • 56
  • 1
    With this approach, won't I be forced to specify the exact version of the package each time? This seems to have the limitations of the git approach, but loses the ability to get "the latest version" *or* a specific version. I should have been clearer about my desire for sensible handling of versions in the question. – Quentin Jan 31 '13 at 10:15
  • you can rename the tar ball to have no version, or on a system with symlink, you can symlink to the latest version. Then again, I like to know which version I'm using so the prod system doesn't get the wrong version unexpectedly. – Pascal Belloncle Jan 31 '13 at 19:59
3

I believe Paypal's Kappa project would suit your need.

Here is an article describing Paypal's Kraken project and how Kappa fits in.

I understand it wasn't available at the time of Quentin's question, but perhaps this will be useful for others that come along here.

user605331
  • 3,718
  • 4
  • 33
  • 60
3

npm-registry-client GitHub issue #42 lists several ways how to create your own repository mirror. namely:

Overall it seems to me that you can get best answers by searching through issues in repositories owned by https://github.com/npm or by asking your question there

xmojmr
  • 8,073
  • 5
  • 31
  • 54
1

Based only on listening to a recent episode of NodeUp (#37?), I think you may want to have a look at irisnpm. From what I remember it's a service which gives you a merged set of the public modules and your own private modules.

David Weldon
  • 63,632
  • 11
  • 148
  • 146
  • The service sounds almost ideal (the only thing I don't like is that it is a service and not something I can just install) … however it appears to be in extreme beta form at the moment and having signed into persona, when I click the *account* link to request an account, I get told that I need to sign in with persona (and I tried two different browsers). – Quentin Jan 31 '13 at 10:10
0

As Dominic Barnes suggested, we can replicate only _design documents (CouchDB table schemes)

How to replicate design documents only?

However, it needs to check if some data needed.

Community
  • 1
  • 1
Paul Verest
  • 60,022
  • 51
  • 208
  • 332
0

You could replicate the modules you need and then write a proxy server which looks for a module in your replication. If a module doesn't exist, it could pipe the request to NPM and return the result from there.

pvorb
  • 7,157
  • 7
  • 47
  • 74