3

Our azure pipeline randomly fails during the npm ci task.

The relevant steps from the job are:

steps:
  - task: NodeTool@0
    displayName: Set node version to 12.x
    inputs:
      versionSpec: '12.x'
  - task: Npm@1
    inputs:
      command: 'ci'

with a

  vmImageName: 'ubuntu-latest'

The pipeline log ends with a

8524 silly extract arrify@2.0.1 extracted to /home/vsts/work/1/s/node_modules/google-auth-library/node_modules/arrify (364ms)
8525 http fetch GET 200 https://pkgs.dev.azure.com/OurOrganization/OurProject/_packaging/develop/npm/registry/typeorm 22160ms
8526 silly extract jws@4.0.0 extracted to /home/vsts/work/1/s/node_modules/google-auth-library/node_modules/jws (465ms)
8527 silly extractTree jwa@2.0.0 -> /home/vsts/work/1/s/node_modules/google-auth-library/node_modules/jwa
8528 silly tarball trying jwa@2.0.0 by hash: sha512-jrZ2Qx916EA+fq9cEAeCROWPTfCwi1IVHqT2tapuqLEVVDKFDENFw1oL+MwrTvH6msKxsd1YTDVw6uKEcsrLEA==
8529 verbose teardown shutting down workers.
8530 info teardown Done in 0s
8531 verbose stack SyntaxError: Unexpected end of JSON input while parsing near '...ueryBuilder [here](./'
8531 verbose stack     at JSON.parse (<anonymous>)
8531 verbose stack     at parseJson (/opt/hostedtoolcache/node/12.22.9/x64/lib/node_modules/npm/node_modules/json-parse-better-errors/index.js:7:17)
8531 verbose stack     at /opt/hostedtoolcache/node/12.22.9/x64/lib/node_modules/npm/node_modules/node-fetch-npm/src/body.js:96:50
8531 verbose stack     at runMicrotasks (<anonymous>)
8531 verbose stack     at processTicksAndRejections (internal/process/task_queues.js:97:5)
8532 verbose cwd /home/vsts/work/1/s
8533 verbose Linux 5.11.0-1025-azure
8534 verbose argv "/opt/hostedtoolcache/node/12.22.9/x64/bin/node" "/opt/hostedtoolcache/node/12.22.9/x64/bin/npm" "ci"
8535 verbose node v12.22.9
8536 verbose npm  v6.14.15
8537 error Unexpected end of JSON input while parsing near '...ueryBuilder [here](./'
8538 verbose exit [ 1, true ]9
8536 verbose npm  v6.14.15
8537 error Unexpected end of JSON input while parsing near '...ueryBuilder [here](./'
8538 verbose exit [ 1, true ]

or a

9117 silly extract parse5@4.0.0 extracted to /home/vsts/work/1/s/node_modules/parse5 (1427ms)
9118 silly extract esprima@3.1.3 extracted to /home/vsts/work/1/s/node_modules/escodegen/node_modules/esprima (337ms)
9119 http fetch GET 200 https://pkgs.dev.azure.com/OurOrganization/OurProject/_packaging/develop/npm/registry/typescript 23176ms
9120 verbose teardown shutting down workers.
9121 info teardown Done in 0s
9122 verbose stack SyntaxError: Unexpected end of JSON input while parsing near '...criptlang.org/","auth'
9122 verbose stack     at JSON.parse (<anonymous>)
9122 verbose stack     at parseJson (/opt/hostedtoolcache/node/12.22.9/x64/lib/node_modules/npm/node_modules/json-parse-better-errors/index.js:7:17)
9122 verbose stack     at /opt/hostedtoolcache/node/12.22.9/x64/lib/node_modules/npm/node_modules/node-fetch-npm/src/body.js:96:50
9122 verbose stack     at runMicrotasks (<anonymous>)
9122 verbose stack     at processTicksAndRejections (internal/process/task_queues.js:97:5)
9123 verbose cwd /home/vsts/work/1/s
9124 verbose Linux 5.11.0-1025-azure
9125 verbose argv "/opt/hostedtoolcache/node/12.22.9/x64/bin/node" "/opt/hostedtoolcache/node/12.22.9/x64/bin/npm" "ci"
9126 verbose node v12.22.9
9127 verbose npm  v6.14.15
9128 error Unexpected end of JSON input while parsing near '...criptlang.org/","auth'
9129 verbose exit [ 1, true ]

As you can see, the content after "Unexpected end of JSON input while parsing near" changes, everything else stays the same. Other issues say the issues can be solved using npm cache clean --force. I suspect the problem occurs because of some files which aren't reset correctly between different runs. The error mentions /opt/hostedtoolcache, which could be a directory in which the node executable is cached on hosted workers.

The issue appears sometimes, restarting the job without changing anything else sometimes helps. The crash-rate is about 50%. We are using Azure DevOps artifacts as an npm registry for all packages.

Our .npmrc:

registry=https://pkgs.dev.azure.com/OurOrganization/OurProject/_packaging/develop/npm/registry/ 
                        
always-auth=true
allinonemovie
  • 652
  • 1
  • 6
  • 20
  • So, have you tried to add this command for caches cleaning in the Pipeline? It might be done using a [Power Shell task](https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/powershell?view=azure-devops). Also, more details on caching for `npm` can be found [in this documentation](https://learn.microsoft.com/en-us/azure/devops/pipelines/release/caching?view=azure-devops#nodejsnpm) – Dmitry Kravtsov Jan 18 '22 at 05:50
  • @DmitryKravtsov Sorry for the late reply, I wanted to test it out over some time: No it doesn't fix the issue. Here is the log of the cache clean command: ```/opt/hostedtoolcache/node/12.22.10/x64/bin/npm cache clean --force npm WARN using --force I sure hope you know what you are doing. Finishing: Npm``` – allinonemovie Feb 16 '22 at 10:31
  • 1
    I just wanted to add that we use a local build server through Azure Dev Ops, and we still get this same issue (randomly failing of npm ci exactly as prescribed above). Previously, we had npm install, and NEVER had this issue, but needed to switch to npm ci after getting burned by bad releases upstream. – Frog Pr1nce Mar 08 '22 at 18:32

0 Answers0