Cannot authenticate to Docker in Elastic Beanstalk through S3

Question

http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_image.html#docker-singlecontainer-dockerrun-privaterepo

Following the instructions here to connect to a private docker hub container from Elastic Beanstalk, but it stubbornly refuses to work. It seems like when calling docker login in Docker 1.12 the resulting file has no email property, but it sounds like aws expects it so I create a file called dockercfg.json that looks like this:

{
    "https://index.docker.io/v1/": {
        "auth": "Y2...Fz",
        "email": "c...n@gmail.com"
    }
}

The relevant piece of my Dockerrun.aws.json file looks like this:

  "Authentication": {
    "Bucket": "elasticbeanstalk-us-west-2-9...4",
    "Key": "dockercfg.json"
  },

And I have the file uploaded at the root of the S3 bucket. Why do I still get errors that say Error: image c...6/w...t:23 not found. Check snapshot logs for details. I am sure the names are right and that this would work if it was a public repository. The full error is below. I am deploying from GitHub with Circle CI if it makes a difference, happy to provide any other information needed.

INFO: Deploying new version to instance(s).                         
WARN: Failed to pull Docker image c...6/w...t:23, retrying...
ERROR: Failed to pull Docker image c...6/w...t:23: Pulling repository docker.io/c...6/w...t
Error: image c...6/w...t:23 not found. Check snapshot logs for details.
ERROR: [Instance: i-06b66f5121d8d23c3] Command failed on instance. Return code: 1 Output: (TRUNCATED)...b-project
Error: image c...6/w...t:23 not found
Failed to pull Docker image c...6/w...t:23: Pulling repository docker.io/c...6/w...t
Error: image c...6/w...t:23 not found. Check snapshot logs for details. 
Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/03build.sh failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
INFO: Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
ERROR: Unsuccessful command execution on instance id(s) 'i-06b66f5121d8d23c3'. Aborting the operation.
ERROR: Failed to deploy application.                                

ERROR: Failed to deploy application.

EDIT: Here's the full Dockerrun file. Note that %BUILD_NUM% is just an int, I can verify that works.

{
  "AWSEBDockerrunVersion": "1",
  "Authentication": {
    "Bucket": "elasticbeanstalk-us-west-2-9...4",
    "Key": "dockercfg.json"
  },
  "Image": {
    "Name": "c...6/w...t:%BUILD_NUM%",
    "Update": "true"
  },
  "Ports": [
    {
      "ContainerPort": "8080"
    }
  ]
}

EDIT: Also, I have verified that this works if I make this Docker Hub container public.

Everything seems to be correct. Could you confirm that EB has all required IAM permissions to access that S3 bucket? — sap1ens, Feb 12 '17 at 04:18
Double checked, the user I'm using the access key for has full S3 access and full Elastic Beanstalk access — CamJohnson26, Feb 12 '17 at 04:31

score 6 · Accepted Answer · answered Feb 19 '17 at 20:15

OK, let's do this;

Looking at the same doc page,

With Docker version 1.6.2 and earlier, the docker login command creates the authentication file in ~/.dockercfg in the following format:

{
  "server" :
  {
    "auth" : "auth_token",
    "email" : "email"
  }
}

You already got this part correct I see. Please double check the cases below one by one;

1) Are you hosting the S3 bucket in the same region?

The Amazon S3 bucket must be hosted in the same region as the environment that is using it. Elastic Beanstalk cannot download files from an Amazon S3 bucket hosted in other regions.

2) Have you checked the required permissions?

Grant permissions for the s3:GetObject operation to the IAM role in the instance profile. For details, see Managing Elastic Beanstalk Instance Profiles.

3) Have you got your S3 bucket info in your config file? (I think you got this too)

Include the Amazon S3 bucket information in the Authentication (v1) or authentication (v2) parameter in your Dockerrun.aws.json file.

Can't see your permissions or your env region, so please double check those. If that does not work, i'd upgrade to Docker 1.7+ if possible and use the corresponding ~/.docker/config.json style.

Thanks for the suggestions, sadly I made the stupid mistake of having 2 Dockerrun.aws.json files and editing the wrong one, since I was getting auth issues no matter what I never actually double checked that. All resolved now. — CamJohnson26, Feb 21 '17 at 17:14

Velu · Answer 2 · 2023-02-25T17:20:33.863

Depending on your Docker version, this file is saved as either ~/.dockercfg or *~/.docker/config.json

cat ~/.docker/config.json

Output:

{
  "auths": {
    "https://index.docker.io/v1/": {
      "auth": "zq212MzEXAMPLE7o6T25Dk0i"
    }
  }
}

Important:

Newer versions of Docker create a configuration file as shown above with an outer auths object. The Amazon ECS agent only supports dockercfg authentication data that is in the below format, without the auths object. If you have the jq utility installed, you can extract this data with the following command:

cat ~/.docker/config.json | jq .auths

Output:

{
  "https://index.docker.io/v1/": {
    "auth": "zq212MzEXAMPLE7o6T25Dk0i",
    "email": "email@example.com"
  }
}

Create a file called my-dockercfg using the above content.
Upload the file into the S3 bucket with the specified key(my-dockercfg) in the Dockerrun.aws.json file.

{
  "AWSEBDockerrunVersion": 2,
  "authentication": {
    "bucket": "elasticbeanstalk-us-west-2-618148269374",
    "key": "my-dockercfg"
  }
}

Cannot authenticate to Docker in Elastic Beanstalk through S3

2 Answers2