Transparently install npm packages offline, from a custom filesystem directory

Question

^{Editor's note: The question's original title was "Use npm install to install node modules stored on a local directory", which made the desire to transparently redefine the installation source less obvious. Therefore, some existing answers suggest solutions based on modifying the installation process.}

I know this is a simple thing, but I'm quite new to anything in this area so after searching around and constantly finding answers that weren't really what I want I figured I'd just ask directly.

I currently have a process that is run in directory FOO that calls npm install. Directory FOO contains a package.json and a npm-shrinkwrap.json file to specify the modules (bluebird, extend, and mysql in this case but it doesn't really matter) and versions. This all works perfectly fine.

But now instead of reaching out to the internet to get the modules I want to have them stored in local directory BAR and have the process in foo use npm to install them from there. I can not store them permanently in FOO but I can in BAR for reasons outside my control. I know this is relatively simple but I can't seem to get the right set of commands down. Thanks for the help.

Answer 1

^{Note: This answer originally suggested only redefining the cache location. While that works in principle, npm still tries to contact the network for each package, causing excessive delays.}

I assume your intent is to transparently change the installation source: in other words: you don't want to change your package, you simply want to call npm install as before, but have the packages be installed from your custom filesystem location, offline (without the need for an Internet connection).

There are two pieces to the puzzle:

• Redefine npm's cache filesystem location (where previously downloaded packages are cached) to point to your custom location:

  • Note that cached packages are stored in a specific way: the package.json file is stored in subfolder package, and the zipped package as a whole as package.tgz. It's easiest to copy packages from your existing cache to your custom location, or to simply install additionally needed ones while you have an Internet connection, which caches them automatically.

  • For transparent use (npm install can be called as usual):

    • By setting the configuration item globally:
      • npm config set cache '/path/to/BAR'
      • Note that this will take effect for all npm operations, persistently.
    • Via an environment variable (which can be scoped to a script or even a single command):
      • export npm_config_cache='/path/to/BAR'
      • npm_config_cache='path/to/BAR' npm install
  • Ad-hoc use, via a command-line option:
    • npm install --cache /path/to/BAR

• Force npm to use cached packages:

  • Currently, that requires a workaround via the cache-min configuration item.
    • A more direct feature, such as via an --offline switch has been a feature request for years - see https://github.com/npm/npm/issues/2568
  • The trick is to set cache-min to a very high value, so that all packages in the cache are considered fresh and served from there:
    • For transparent use (npm install can be called as usual):
      • By setting the configuration item globally:
        • npm config set cache-min 9999999999
        • Note that this will take effect for all npm operations, persistently.
      • Via an environment variable (which can be scoped to a script or even a single command):
        • export npm_config_cache_min=9999999999
        • npm_config_cache_min=9999999999 npm install
    • Ad-hoc use, via a command-line option:
      • npm install --cache-min 9999999999

Assuming you've set cache-min globally or through an environment variable, running npm install should now serve the packages straight from your custom cache location.

Caveats:

• This assumes that all packages your npm install needs are available in your custom location; trying to install a package that isn't in the cache will obviously fail without an Internet connection.

• Conversely, if you do have Internet access but want to prevent npm from using it to fetch packages - which it still will attempt if a package is not found in the cache - you must change the registry configuration item to something invalid so as to force the online installation attempt to fail; e.g.:

  • export npm_config_registry=http://example.org
  • Note that the URL must exist to avoid delays while npm tries to connect to it; while you could set the value to something syntactically invalid (e.g., none), npm will then issue a warning on every use.

---

Sample bash script:

#!/usr/bin/env bash

# Set environment variables that set npm configuration items to:
#  - redefine the location of the cache folder
#  - make npm look in the cache only (assuming the packages are there)
# Note that by doing this inside a script the changes will only take effect
# in the script and NOT persist.
export npm_config_cache='/path/to/BAR' npm_config_cache_min=9999999999

# Now cd to your package and invoke `npm install` as usual.
cd '/path/to/project'
npm install

Answer 2

You might want to try npm link. You could:

1. download the dependency
2. run npm link from the dependency's directory
3. run npm link mycrazydependency from you project

Detail here: https://docs.npmjs.com/cli/link

Answer 3

If a shrink wrap file is present then package.json is ignored. What you need to do is change the URL they are being retrieved from using a find and replace operation like sed .... However I'm not sure changing the URL to a file:/// syntax is valid but give it a go.