boto3: Is aws_session_token necessary when trying to list objects in a bucket?

Question

I'm writing a function currently to fetch objects from a bucket using boto3. While connecting, I have to connect like this:

session = boto3.session.Session(
    aws_access_key_id="XXXXXXXXXX",
    aws_secret_access_key="YYYYYYYYYYYYYY")

or like this:

client = boto3.client('s3',aws_access_key_id="XXXXXXXXXXXX",
aws_secret_access_key="YYYYYYYYYYYYYYY"

i.e, there are only two arguments with me: aws_access_key_id and aws_secret_access_key. And using a shared credential file is not an option. Though I am able to connect without the aws_session_token, whenever I run:

for obj in first_bucket.objects.filter(Prefix="my filter"):
        print(obj.key)
        response = obj.key

this error is shown:

botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ListObjectsV2 operation: The AWS Access Key
Id you provided does not exist in our records.

When I went through the credentials documentation here and went through a few other questions on SO, there are only suggestions to use aws_session_token when other credentials are temporary. There is also mention that sessions can be created by boto3 itself and that there's no need for the programmer to handle it. In that case, why am I being forced to add a aws_session_token mandatorily when accessing objects? Is aws_session_token something that is mandatory when objects are being accessed?

Where do the access and secret key come from, are they the access credentials of an IAM User? Does not sound like it. — luk2302, Sep 02 '21 at 11:24

score 0 · Answer 1 · answered Sep 03 '21 at 13:41

This was happening because the aws_access_key_id and aws_secret_access_key I had were temporary credentials. I was able to use only two of those parameters when I got credentials from another account. John's comment above made me think about the credentials I have and so I tried using credentials from another account and it worked.

boto3: Is aws_session_token necessary when trying to list objects in a bucket?

1 Answers1