7

I have an ECS managed EC2 instance running in a VPC (in one of the private subnets). When trying to run a task on this instance it doesn't seem to be able to pull the image. As far as I can make out from the documentation there is no special configuration needed for the ECS agent to pull the image from the repo.

Looking at the Docker logs I repeatedly see the following:

level=error msg="Download failed, retrying: dial tcp 54.231.17.81:443: i/o timeout"

The ecs-agent logs repeatedly show me that the image is not downloading:

Pulling image module="TaskEngine" image="REDACTED.dkr.ecr.us-east-1.amazonaws.com/REDACTED:latest" status="Retrying in 19 seconds"

It eventually tries to run image, but obviously fails and exits. Giving me the message below in the Cluster Tasks tab:

STOPPED (Essential container in task exited)

This error has been occurring with both amzn-ami-2016.03.e and amzn-ami-2016.03.d AMIs

Is there any specific configuration or networking rules that I need to apply to be able to pull from ECR?

Any help here would be greatly appreciated.

As a side note, the instance does have access to the internet (pinging google.com works fine), and when I try to pull an image from Docker Hub, it also works fine.

Ryan McClarnon
  • 203
  • 2
  • 9
  • is IAM configured correctly? – Marc Young Jul 14 '16 at 14:41
  • Yes I'm using the default AmazonEC2ContainerServiceforEC2Role policy attached to a role I've named "ecsInstanceRole" This role is attached to each EC2 instance – Ryan McClarnon Jul 14 '16 at 16:03
  • can you pull the image from bash prompt on the instance? – Shibashis Jul 14 '16 at 23:36
  • Not without further procedures as the recommended AMI comes without aws-cli installed. Hence "docker pull" will not be authorized. – Ryan McClarnon Jul 18 '16 at 09:06
  • 1
    you can authorize without the AWS cli. do aws ecr get-login somewhere else and run the output on the ecs instance to try manually pulling. – Marc Young Jul 20 '16 at 22:03
  • Thanks for the info Marc, I think we have chosen DockerHub for now, but if we choose to switch back I'll definitely take your advice and try it out. – Ryan McClarnon Jul 22 '16 at 08:46
  • Hi @RyanMcClarnon I am also facing this issue, did you switch to AWS ECR? Did you manage to solve this? – Jeremie Jan 27 '18 at 20:16
  • Also running into this isssue. I have to login to the instance and manually pull the image after logging in with docker. – Caleb Aug 01 '18 at 11:32

1 Answers1

2

To download image from ECR, Container Instance needs access to ECR/S3 endpoints.

If your subnet is private you have to either use PrivateLink feature or have to use NAT gateway to reach to ECR endpoints.

If you choose to use PrivateLink, this includes:

  1. Creating the VPC Endpoint for Amazon ECR
  2. Creating the Amazon S3 Gateway Endpoint

If you choose to use NatGateway, route all traffic to NATGateway and whitelist AWS IP ranges.

Reference Link: https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-endpoints.html

Mangal
  • 607
  • 4
  • 8