Can AWS CodePipeline track multiple feature branches and run tests on each?

Question

With Bitbucket and Bamboo, I was able to have Bamboo track every feature branch and run tests on each so that, at the pull request time, I was able to see if the branch passed its unit tests.

With AWS CodePipeline, I can't tell if I am able to track each feature branch and have tests run on them before merging.

Is this possible? If so, please point me to documentation.

Answer 1

I was looking for a solution to exactly this problem. Eventually I settled on having a CodeBuild, which can be triggered off of a branch regex, begin the pipeline by pushing an archive to a specific S3 key. In my case, I also had that CodeBuild do my full build/test process, but you could also configure the CodeBuild to only pull the code and push it the the S3 key that triggers your CodePipeline.

Here's part of an example CodeBuild config matching 2 branches:

Then I set the CodeBuild artifact to go to a single key in a single bucket.

Then I setup a CodePipeline with an Amazon S3 source pointing to the same key/bucket.

Answer 2

CodePipeline is not the right tool for you. Create a separate, standalone CodeBuild project. It will work very much like other 3rd party CI services such as Travis.

There are a number of ways you can connect this "CI stage" to a continuous delivery pipeline, which is what CodePipeline is for:

1. The simplest is just to use CodeBuild as a gate for pushing changes to the pipeline source branch, typically master.
2. You can also push artifacts created in CodeBuild to ECR or S3 and trigger a pipeline from those events.
3. If you want to get fancy, use some other jiggery pokery eg. Events, SQS and Lambda.

I had the same confusion as CodeBuild and CodePipeline are tightly interlinked but also separate tools. CodePipeline does use CodeBuild but each has its own Git connector that works differently.

Answer 3

Further review shows that with Cloudformation, you can pick the branch that CodePipeline tracks,

AWS CodeCommit (CodeCommit)

• PollForSourceChanges¹ (Optional)
• RepositoryName (Required)
• BranchName (Required)

You can see an example of complete template but the CodePipeline stage looks like,

Name: CheckoutSourceTemplate
ActionTypeId:
  Category: Source
  Owner: AWS
  Version: 1
  Provider: CodeCommit
Configuration:
  PollForSourceChanges: True
  RepositoryName: !GetAtt [PipelineRepo, Name]
  BranchName: master
OutputArtifacts:
  - Name: TemplateSource
RunOrder: 1

With CodeCommit Repo's, you can create Triggers that can use these triggers to launch a Lambda function,

You can configure Lambda functions by creating the trigger in the Lambda console as part of the function. This is the simplest method, as triggers created in the Lambda console automatically include the permissions required for AWS CodeCommit to invoke the Lambda function. If you create the trigger in AWS CodeCommit, you must include a policy to allow AWS CodeCommit to invoke the function. For more information, see Create a Trigger for an Existing Lambda Function and Example 2: Create a Policy for AWS Lambda Integration.

So what could happen is to set up a CloudFormation template like above to track the master branch. Then have CodeCommit trigger on repository changes and call a Lambda function that uses Boto3 to get_pipeline to retrieve the master branch pipeline.

Then using either update_pipeline or create_pipeline to add a stage to the existing master branch pipeline or create an entirely new pipeline that tracks the additional branches desired.

This way, the CodePipeline can track the feature branches in a useful way.

Answer 4

Multiple codebuild projects to build multiple branches, drop your artifacts to s3 and pickup with codepipeline from there.

What i'm trying to achieve is: Checkout a different branch under the same codebuild project so as to be able to to a diff check and generate packages for sitecore items. i.e. release\branch <=> master => then generate package of latest items only, release\branch project, then Zip.

Answer 5

I have achieved what you want to do by creating ephemeral pipelines, I have a codebuild that looks for branches on the repo which are not main and is triggered on PR.

When that codebuild runs, it extracts the branch name and then creates a brand new code pipeline on the fly using the branch it now knows about, this is done with terraform. When the change is merged to master I then destroy the ephemeral environment and the ephemeral pipeline.

I could not think of a neater solution to creating a multi-branch pipeline.

Answer 6

Alternatively what can be done is to have a dedicated branch to trigger the pipeline (ex: pipeline branch). Once you want to test code from a branch, you can change the reference of the pipeline branch to your custom branch (ex: feature-branch) such that the pipeline branch will basically "point" to the latest commit of the custom branch and will trigger the code pipeline execution.

Also what we can do is, by taking this a step further, we can create a custom GitHub action, that will trigger the above branch reference change within Github to fully automate the pipeline trigger based on the developer's need.