Boto is unable to access bucket inside ECS container which have correct IAM roles (but Boto3 can)

Question

I have ECS container and I have attached an IAM policy like below:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Action": [
                "s3:List*",
                "s3:Get*"
            ],
            "Resource": "arn:aws:s3:::my_s3_bucket/*"
        }
    ]
}

and have both boto and boto3 installed in it.

I am able to list bucket using boto3 but not by boto. Please see code below:

import boto3
s3_conn = boto3.client('s3')
s3_conn.list_objects(Bucket='my_s3_bucket')

'Owner': {u'DisplayName': 'shoppymcgee', u'ID': 'adf3425700e4f995d8773a8b********'}, u'Size': 116399950}, {u'LastModified': datetime.datetime(2013, 5, 18, 6, 35, 6, tzinfo=tzlocal()), u'ETag': '"2b4a4d60458cde1685c93dabf98c6e19"', u'StorageClass': 'STANDARD', u'Key': u'2013/05/18/SLPH_201305180605_eligible-product-feed.txt', u'Owner': {u'DisplayName': 'shoppymcgee', u'ID': 'adf3425700e4f995d8773a8be6b0df09d06751f3274d8be5e8ae04761a5eef09'},

import boto
conn = boto.connect_s3()
print conn
S3Connection:s3.amazonaws.com
mybucket = conn.get_bucket('my_s3_bucket')


Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/boto/s3/connection.py", line 509, in get_bucket
    return self.head_bucket(bucket_name, headers=headers)
  File "/usr/local/lib/python2.7/site-packages/boto/s3/connection.py", line 528, in head_bucket
    response = self.make_request('HEAD', bucket_name, headers=headers)
  File "/usr/local/lib/python2.7/site-packages/boto/s3/connection.py", line 671, in make_request
    retry_handler=retry_handler
  File "/usr/local/lib/python2.7/site-packages/boto/connection.py", line 1071, in make_request
    retry_handler=retry_handler)
  File "/usr/local/lib/python2.7/site-packages/boto/connection.py", line 943, in _mexe
    request.body, request.headers)

Version of boto - boto==2.48.0

Version of boto3 and botocore - botocore==1.7.41 and boto3==1.4.7

Sam · Accepted Answer · 2018-05-20T20:24:10.843

Boto does not support the AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variable, which is what gives containers/tasks the ability to use the task-specific IAM role.

If you do a search on GitHub for that environment variable in Boto's repository, you'll come up with no code hits and an open issue asking for it to be implemented - https://github.com/boto/boto/search?q=AWS_CONTAINER_CREDENTIALS_RELATIVE_URI

Until support is added (if it will be at all given boto's maintenance state), the only way to use boto is to call the metadata service manually on curl 169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI, retrieve the credentials and manually pass them to boto (be careful of the expiry on the temporary credentials though).

Or migrate to boto3.

Boto is unable to access bucket inside ECS container which have correct IAM roles (but Boto3 can)

1 Answers1