Github monorepo as source for AWS CodePipeline

Question

We use the monorepo approach for storing our source in github.

Is it currently possible to have CodePipeline trigger only on a commit to a particular subfolder.

This is something that is currently possible with TeamCity by setting a filter on the Source Repository but I've seen no example of this with CodePipeline.

Answer 1

We worked on exactly that issue. In short: It's possible with a little work around. You can set up a GitHub Webhook as source for a single AWS CodeBuild project. The project allows you to configure filters (file paths) on events pushed to that Webhook. Additionally, you add an S3 bucket and push the code referenced by the filtered event to that bucket. The bucket can be included as source stage for your CodePipeline.

We have documented our solution in a blog post: https://blog.codecentric.de/en/2019/05/codebuild-trigger-pipeline/

Answer 2

I tried to use CodeBuild as suggested in other answers, but it ended more complicated than I expected for my setup.

I ended up with a solution using GitHub Actions and source branches per stack.

1) I created a branch per stack

• pipelines/api
• pipelines/data

2) Then I created a GitHub Action per stack

These actions are only being triggered on main pushes and are filtered based on the changed files.

If an api matching change then merge into pipelines/api, similarly for the data stack and pipelines/data

# .github/workflows/sync-pipelines-api-yaml

name: Sync pipelines/api
on:
  push:
    branches:
      - main
    paths:
      - "package-lock.json"
      - "lib/api/**"
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
        run: |
          git switch -c main
          git remote set-url origin "https://$GITHUB_ACTOR:$GITHUB_TOKEN@github.com/$GITHUB_REPOSITORY"
          git push origin main:pipelines/api --force

# .github/workflows/sync-pipelines-data-yaml

name: Sync pipelines/data
on:
  push:
    branches:
      - main
    paths:
      - "package-lock.json"
      - "lib/data/**"
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
        run: |
          git switch -c main
          git remote set-url origin "https://$GITHUB_ACTOR:$GITHUB_TOKEN@github.com/$GITHUB_REPOSITORY"
          git push origin main:pipelines/data --force

3) Then, I changed the source branch for each CodePipeline accordingly

// lib/api/stack.ts

const sourceAction = new actions.GitHubSourceAction({
  owner,
  repo,
  branch: "pipelines/api"
});

// lib/data/stack.ts

const sourceAction = new actions.GitHubSourceAction({
  owner,
  repo,
  branch: "pipelines/data"
});

---

Some benefits of this setup:

• If I want to deploy all stacks, I can push directly to each branch bypassing the stack action
• Very powerful filtering options
• Easy to test and change. CodeBuild/CodePipeline test cycle can take some time
• Minimal CodePipeline change

Answer 3

Looks like you answered your own question ;). The answer is not yet, but as a workaround, you can parse the list of changed files and derive the list of changed subfolders from that: https://forums.aws.amazon.com/thread.jspa?messageID=811027&#811027/.

Answer 4

We have written an article on a workaround for deploying a GitHub MonoRepo using CodeBuild and included an example repository.

Answer 5

Normally CodePipeline monitors the git repo and then downloads the source and triggers CodeBuild.

I worked around this issue by instead having CodeBuild monitor the git repo, as that allows you to restrict the changes to a subdirectory within the repo. Once it builds it uploads the artifacts to S3, then you can have the CodePipeline trigger on the S3 artifact upload and continue with the process.

It's a bit untidy but providing the first step in your pipeline is a single CodeBuild one then it should be easy enough to do.