11

I am trying to cache a command line tool needed for my build process. The tool is made out of NodeJS. The build succeeds, but I need it to run faster.

The relevant parts of my circle.yml look like this :

dependencies:    
  post:
    - npm -g list
    - if [ $(npm -g list | grep -c starrynight) -lt 1 ]; then npm install -g starrynight; else echo "StarryNight seems to be cached"; fi

test:
  override:
    - npm -g list
    - starrynight run-tests --framework nightwatch

The second npm -g list shows starrynight available for use, but the first one shows that it is not being cached.

echo $(npm prefix -g)

. . . gets me . . . 

/home/ubuntu/nvm/v0.10.33

. . . so I am assuming CircleCI doesn't cache anything installed globally into nvm.

Nothing I have tried gets me my message, "StarryNight seems to be cached".

How can I cache starrynight?

Martin Bramwell
  • 2,003
  • 2
  • 19
  • 35

2 Answers2

6

Ok, I figured this out. Thanks to Hirokuni Kim of CircleCI for pointing me in the right direction.

The relevant bits of the new circle.yml looks like this :

machine:
  node:
    version: 0.10.33

dependencies:
  cache_directories:
    - ~/nvm/v0.10.33/lib/node_modules/starrynight
    - ~/nvm/v0.10.33/bin/starrynight
  pre:
    - if [ ! -e ~/nvm/v0.10.33/bin/starrynight ]; then npm install -g starrynight; else echo "Starrynight seems to be cached"; fi;

Hirokuni suggested caching ~/nvm but cache retrieval took as long as the build, since it restores every available version of nodejs.

I had tried previously to cache just ~/nvm/v0.10.33/lib/node_modules/starrynight on its own, without realizing that the sister 'directory' bin/starrynight is actually an essential symlink to the entry point of the module.

My working assumption is that NodeJS modules run from the command line through a series of symbolic references, probably as follows. . .

npm install -g starrynight creates two new artifacts:

  • an environment alias for npm named starrynight
  • a symlink in the ${prefix}/bin directory, which points to the entry point file, starrynight.js specified with the bin key in package.json.

When the user types starrynight as a CLI command the shell interprets it as an alias for npm and executes it. npm examines $0, gets starrynight, and starts up nodejs with the symlink ${prefix}/bin/starrynight as the module to execute. That symlink refers to ~/nvm/v0.10.33/lib/node_modules/starrynight where the real action takes place.

In short, it is necessary to cache both ${prefix}/lib/node_modules/xxx and ${prefix}/bin/xxx

Martin Bramwell
  • 2,003
  • 2
  • 19
  • 35
2

The path in the accepted answer doesn't seem to work anymore. I've checked on the build instance that now global npm packages are in 

/opt/circleci/nodejs/<version>

so I've added to the circle.yml the following:

- "/opt/circleci/nodejs/v4.3.2/lib/node_modules"
- "/opt/circleci/nodejs/v4.3.2/bin"
Aides
  • 467
  • 2
  • 5
  • 16