8

Python is my preferred language but any supported by Lambda will do. -- All AWS Architecture --

I have Prod, Beta, and Gamma branches and corresponding folders in S3. I am looking for a method to have Lambda respond to a CodeCommit trigger and based on the Branch that triggered it, clone the repo and place the files in the appropriate S3 folder.

  • S3://Example-Folder/Application/Prod
  • S3://Example-Folder/Application/Beta
  • S3://Example-Folder/Application/Gamma

I tried to utilize GitPython but it does not work because Lambda does not have Git installed on the base Lambda AMI and GitPython depends on it.

I also looked through the Boto3 docs and there are only custodial tasks available; it is not able to return the project files.

Thank you for the help!

Shawn
  • 561
  • 1
  • 5
  • 12

3 Answers3

5

The latest version of the boto3 codecommit includes the methods get_differences and get_blob. You can get all the content of a codecommit repository using these two methods (at least, if you are not interested in the retaining the .git history).

The script below takes all the content of the master branch and adds it to a tar file. Afterwards you could upload it to s3 as you please. You can run this as a lambda function, which can be invoked when you push to codecommit.

This works with the current lambda python 3.6 environment. botocore==1.5.89 boto3==1.4.4

import boto3
import pathlib
import tarfile
import io
import sys


def get_differences(repository_name, branch="master"):
    response = codecommit.get_differences(
        repositoryName=repository_name,
        afterCommitSpecifier=branch,
    )
    differences = []
    while "nextToken" in response:
        response = codecommit.get_differences(
            repositoryName=repository_name,
            afterCommitSpecifier=branch,
            nextToken=response["nextToken"]
        )
        differences += response.get("differences", [])
    else:
        differences += response["differences"]
    return differences


if __name__ == "__main__":
    repository_name = sys.argv[1]
    codecommit = boto3.client("codecommit")
    repository_path = pathlib.Path(repository_name)
    buf = io.BytesIO()
    with tarfile.open(None, mode="w:gz", fileobj=buf) as tar:
        for difference in get_differences(repository_name):
            blobid = difference["afterBlob"]["blobId"]
            path = difference["afterBlob"]["path"]
            mode = difference["afterBlob"]["mode"]  # noqa
            blob = codecommit.get_blob(
                repositoryName=repository_name, blobId=blobid)
            tarinfo = tarfile.TarInfo(str(repository_path / path))
            tarinfo.size = len(blob["content"])
            tar.addfile(tarinfo, io.BytesIO(blob["content"]))
    tarobject = buf.getvalue()
    # save to s3
Sam vL
  • 81
  • 1
  • 6
1

Looks like LambCI does exactly you want.

enter image description here

jaym
  • 1,253
  • 13
  • 18
1

Unfortunately, currently CodeCommit doesn’t have an API to upload the repository to S3 bucket. However, if you are open to trying out CodePipeline, You can configure AWS CodePipeline to use a branch in an AWS CodeCommit repository as the source stage for your code. In this way, when you make changes to your selected tracking branch in CodePipeline, an archive of the repository at the tip of that branch will be delivered to your CodePipelie bucket. For more information about CodePipeline, please refer to following link: http://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-simple-codecommit.html

Hsiao-Tung Chen
  • 316
  • 3
  • 3