AWS CodeBuild does not work with Yarn Workspaces

Question

I'm using Yarn Workspaces in my repository and also using AWS CodeBuild to build my packages. When build starts, CodeBuild takes 60 seconds to install all packages and I'd want to avoid this time caching node_modules folder.

When I add:

cache:
  paths:
    - 'node_modules/**/*'

to my buildspec file and enable LOCAL_CUSTOM_CACHE, I receive this error:

error An unexpected error occurred: "EEXIST: file already exists, mkdir '/codebuild/output/src637134264/src/git-codecommit.us-east-2.amazonaws.com/v1/repos/MY_REPOSITORY/node_modules/@packages/configs'".

Is there a way to remove this error configuring AWS CodeBuild or Yarn?

My buildspec file:

version: 0.2
phases:
  install:
    commands:
      - npm install -g yarn
      - git config --global credential.helper '!aws codecommit credential-helper $@'
      - git config --global credential.UseHttpPath true
      - yarn
  pre_build:
    commands:
      - git rev-parse HEAD
      - git pull origin master
  build:
    commands:
      - yarn run build
      - yarn run deploy
  post_build:
    commands:
      - echo 'Finished.'
cache:
  paths:
    - 'node_modules/**/*'

Thank you!

Update 1:

The folder /codebuild/output/src637134264/src/git-codecommit.us-east-2.amazonaws.com/v1/repos/MY_REPOSITORY/node_modules/@packages/configs was being attempted to be created by Yarn, with the command - yarn at install phase. This folder is one of my repository packages, called @packages/config. When I run yarn on my computer, Yarn creates folders linking my packages as described here. An example of how my node_modules structure is on my computer:

node_modules/
|-- ...
|-- @packages/
|   |-- configs/
|   |-- myPackageA/
|   |-- myPackageB/
|-- ...

Thanks for reporting the issue. We'll take a look. Was the mkdir being attempted by your build logic or CodeBuild? (I assume this is CodeBuild based on your buildspec). — Subin Mathew, Apr 28 '19 at 16:52
@SubinMathew thank you for your reply. The command `- yarn` at the install phase attempted to create that folder, which is one of my packages. I've updated my answer providing more details. — Pedro Arantes, Apr 28 '19 at 19:54
We have made some improvements to keep the file path implicitly static. Could you give this a try again and let us know if you run into the same issue? — Subin Mathew, Jun 07 '19 at 06:15
@SubinMathew I've tried to run CodeBuild with the same configuration and I've received the same error: ` Error: EEXIST: file already exists, mkdir '/codebuild/output/src046905303/src/git-codecommit.us-east-1.amazonaws.com/v1/repos/MY_REPOSITORY/node_modules/@packages/appsync-backend'`. Do I need to change some configuration? — Pedro Arantes, Jun 08 '19 at 13:50
@jogold - can you please post on AWS forums @ https://forums.aws.amazon.com/forum.jspa?forumID=230? We need your account Id and buildARN. We can get the account id when you post on the AWS forums. — Subin Mathew, Sep 12 '19 at 13:49
@PedroArantes did you solve this issue? I'm getting the same error. — ian, Dec 28 '19 at 08:16
https://github.com/aws-samples/aws-codebuild-samples/issues/8 — jpswade, Sep 11 '20 at 11:09

score 4 · Answer 1 · answered Mar 18 '21 at 08:05

I was having the exact same issue ("EEXIST: file already exists, mkdir"), I ended up using S3 cache and it worked pretty well. Note: for some reason the first upload to S3 took way (10 minutes) too long, the others went fine.

Before:

[5/5] Building fresh packages...
--
Done in 60.28s.

After:

[5/5] Building fresh packages...
--
Done in 6.64s.

If you already have your project configured you can edit the cache accessing the Project -> Edit -> Artifacts -> Additional configuration.

My buildspec.yml is as follows:

version: 0.2

phases:
  install:
    runtime-versions:
       nodejs: 14
  build:
    commands:
      - yarn config set cache-folder /root/.yarn-cache
      - yarn install --frozen-lockfile
      - ...other build commands go here

cache:
  paths:
    - '/root/.yarn-cache/**/*'
    - 'node_modules/**/*'
    # This third entry is only if you're using monorepos (under the packages folder)
    # - 'packages/**/node_modules/**/*'

If you use NPM you'd do something similar, with slightly different commands:

version: 0.2

phases:
  install:
    runtime-versions:
       nodejs: 14
  build:
    commands:
      - npm config -g set prefer-offline true
      - npm config -g set cache /root/.npm
      - npm ci
      - ...other build commands go here

cache:
  paths:
    - '/root/.npm-cache/**/*'
    - 'node_modules/**/*'
    # This third entry is only if you're using monorepos (under the packages folder)
    # - 'packages/**/node_modules/**/*'

Kudos to: https://mechanicalrock.github.io/2019/02/03/monorepos-aws-codebuild.html

score 0 · Answer 2 · answered Apr 26 '23 at 10:27

In case anyone are still suffering this error in 2023, here are some of my observations:

If you want to use any sort of Local Cache (docker layer, source cache), it won't be compatible with specified path (which is used for custom cache) in the buildspec, e.g.

cache:
  paths:
    - 'node_modules'

You will need to remove this from your buildspec.

AWS CodeBuild does not work with Yarn Workspaces

2 Answers2

Linked