6

We have a python docker image which needs to build/publish (CI/CD) into AWS container registry. At the moment AWS does not support for running docker tasks using docker hub private repositories, therefore we have to use ECR instead of docker hub.

Our CI/CD pipeline uses docker build and push tasks. Docker authentication is done via a Service Endpoint in the VSTS project.

There are few steps we should follow to setup a VSTS service endpoint for ECR. This required to execute AWS CLI command (locally or cloud) to get a user and password for docker client to login, it looks like;

aws ecr get-login --no-include-email

Above command outputs a docker login command with a username (AWS) and a password (token).

The issue with this approach is access token will last only for 12 hours. Therefore CI/CD task requires updating the Service Endpoint every 12 hours, otherwise build fail with unauthorised token exception.

Other option we have is to run some shell commands to execute aws get-login command and run docker build/push commands in the same context. This option required installing aws cli into build agent (we are using public linux agent). In addition shell command involves awkward task configuration with environment/variables. Otherwise we will be exposing aws application id and secret in the build steps.

Could you please advice if you have solved VSTS CI/CD pipeline using docker with AWS ecr?

Thanks, Mahi

Rosario Capuno
  • 11
  • 6
mahifernando
  • 314
  • 2
  • 8

2 Answers2

13

After lot of research, trial and error I found an answer to my own question.

AWS provides an extension to VSTS with build tasks and Service Endpoints. You need to configure AWS service endpoint using an account number, application ID, and secret. Then, in your build/release definition;

  1. build docker image using out of the box docker build task, or shell/bash command (for an example; docker build -t your:tag . )

  2. Then add another build step to push image into AWS registry, for this you can use AWS extension task (Amazon Elastic Container Registry Push Image). Amazon Elastic Container Registry Push Image build task will generate token and login docker client every time you run this build definition. You don't have to worry about updating username/token every 12 hours, AWS extension build task will do that for you.

build docker image

Amazon Elastic Container Registry Push Image

mahifernando
  • 314
  • 2
  • 8
  • Have you figured out how to push 2 tags within the same step? I want to push tag: latest and 1.x.x to the same image – odannyc Dec 07 '18 at 21:22
  • How do I pull from the registry? The image I'm building is based on an image in my AWS ECR. Any ideas? – PilotBob May 03 '19 at 21:35
  • Pushing your image into ECR shouldn't be part of the release definition?? you may have different enviornments. In that case how would you share the image settings like name and tag from the build to release definition? – Gonza May 14 '20 at 17:24
2

You are looking for this

Amazon ECR Docker Credential Helper AWS documentation This is where Amazon ECR Docker Credential Helper makes it easy for developers to use ECR without the need to use docker login or write logic to refresh tokens and provide transparent access to ECR repositories.

Credential Helper helps developers in a continuous development environment to automate the authentication process to ECR repositories without having to regenerate tokens every 12 hours. In addition, Credential Helper also provides token caching under the hood so you don’t have to worry about getting throttled or writing additional logic

Aniket Chopade
  • 801
  • 5
  • 12
  • Thanks for taking time, this is not doable in Visual Studio Team Services build definition. As I mentioned in the question, docker build step required "registry" service endpoint. Which is configured using username/password (token). Using a credential helper in service endpoints not supported. – mahifernando Jul 16 '18 at 14:54