AWS CLI listing S3 buckets gives SignatureDoesNotMatch error using IAM user credentials

Question

I am using AWS CLI on Ubuntu 16.04 LTS, I am trying to list all buckets. In aws configure I have input the IAM user access key and IAM user secret key. This IAM user has permissions to list buckets and can list buckets in the console. But using AWS CLI with these keys and running the command aws s3 ls, it is giving me this error:

A client error (SignatureDoesNotMatch) occurred when calling the ListBuckets operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.

I have created a policy to list buckets for this particular IAM user also.

I want to perform further sync operations and make all files public operations via a shell script using this IAM user credentials and do not want to use root credentials.

Based on the error, the secret you're using is wrong. Most likely, a copy/paste error. Try revoking that access key and create new keys. — Matt Houser, Jun 19 '16 at 02:42
The other thing it could be is trying to use signature V2 in a region that only supports V4 (like eu-central-1). — Matt Houser, Jun 19 '16 at 02:44
Thank you sir, by changing the keys I was able to solve the problem. Could you please tell — Piyush dhore, Jun 19 '16 at 03:21

schmudu · Answer 1 · 2018-12-03T23:52:20.840

Found my issue. I had old AWS keys in my environment variables. If you have environment variables named

AWS_SECRET_ACCESS_KEY
AWS_ACCESS_KEY_ID

the awscli will use those values instead of what is provided via ~/.aws/credentials.

Try running printenv | grep AWS and verify that those values aren't set. If so then just run a

unset AWS_SECRET_ACCESS_KEY
unset AWS_ACCESS_KEY_ID

and you should be good to go.

score 20 · Answer 2 · answered May 19 '20 at 05:23

In my case, this was due to incorrect aws_secret_access_key.

To check, open the file ~/.aws/credentials by typing:

cat ~/.aws/credentials

The content should be something like below:

[default]
aws_access_key_id = xxx
aws_secret_access_key = xx

See if the aws_access_key_id & aws_secret_access_key matches your credentials. If it doesn't, edit and save changes.

p/s: If you don't remember your aws_secret_access_key, generate a new key and secret by going to aws console --> your name --> My Security Credentials.

Then click 'Create access key':

Take note that you can only have two access keys at a time.

You can use directly `aws configure` and add the correct credentials with the proper permissions created by IAM — Tenma Kenzo, Apr 03 '23 at 11:17

score 7 · Answer 3 · edited May 30 '19 at 07:25

7

Can happen even when the machine time is not in sync with the NT server.

sudo ntpdate ntp.ubuntu.com helped me solve this problem.

edited May 30 '19 at 07:25

tomerpacific

4,704
13
34
52

answered May 30 '19 at 05:37

Aditya

71
1
1

1

A frequent issue on Mondays after changing to the summer/winter time – Jorge P. Oct 27 '19 at 11:50

score 6 · Answer 4 · answered Oct 24 '19 at 04:32

It means that your AWS security credentials got expired. Simply creating new credentials will work.

Go to your AWS account -> My security credentials
click on Create New Access Key. Make a note of access key id and secret access key
Run aws configure and enter new credentials

score 5 · Answer 5 · answered Jan 10 '17 at 14:43

5

This error is because of incorrect aws s3 access key/secret key.

answered Jan 10 '17 at 14:43

Ananth

787
5
18

score 3 · Answer 6 · answered May 24 '19 at 18:38

The issue was with the AWS credentials, i copied the secret from excel file into txt file and some how few of the special character stripped away. Make sure to copy it properly.

Also try restarting the machine and make sure AWS is not set in environment variable, you can check it with printenv | grep 'AWS'

score 3 · Answer 7 · edited Feb 15 '21 at 15:04

3

Fixed it by using --endpoint-url option as stated here: https://github.com/aws/aws-cli/issues/4922

Looks like it is related to the fact that the VM I was on was in a different region than the bucket.

edited Feb 15 '21 at 15:04

RobC

22,977
20
73
80

answered Feb 15 '21 at 13:59

Suresh

627
6
16

score 2 · Answer 8 · answered Apr 02 '18 at 21:11

2

In my case, I had encryption but was sending the the size the file was before being encrypted. If you got this error and your secret and key are correct, it's worth double checking your md5, mimetype, size, and other attributes

answered Apr 02 '18 at 21:11

mowwwalker

16,634
25
104
157

2

An interesting possibility, thanks for suggesting something that may not have been obvious to a new user – Zerodf Jul 12 '18 at 21:40

score 2 · Answer 9 · answered Sep 26 '18 at 11:47

2

Just check the time on the system on which you are running. make sure it is updated

answered Sep 26 '18 at 11:47

gamechanger17

4,025
6
22
38

score 2 · Answer 10 · edited Jun 02 '19 at 12:45

2

Please switch to root user.

In my case I was accessing this command aws s3 ls from standard user and its giving below error:

"AWS CLI listing S3 buckets gives SignatureDoesNotMatch error using IAM user credentials"

then switched to root user by using sudo su command and tried accessing aws s3 ls command it listed s3 bucket names.

edited Jun 02 '19 at 12:45

Brian Tompsett - 汤莱恩

5,753
72
57
129

answered Jun 02 '19 at 12:01

user11589664

39
1

Not sure why this was downvoted, because it helped me. I ran the command with sudo and it worked. – Caleb Pitman Jan 26 '20 at 18:57

score 2 · Answer 11 · answered Aug 25 '21 at 14:50

2

This worked for me (I was trying to download a large file):

aws configure set default.s3.multipart_threshold 1000MB

answered Aug 25 '21 at 14:50

Zubin G

21
1

Kuldip Mori · Answer 12 · 2022-03-21T06:20:35.587

2

if you are facing this Error also check the key are the same or different.

in a short time, the best solution is to recreate a new key and change in code with the OLD key working in my case.

After the changes for check use this command.

$ aws configure

-> access key

-> secret key

-> region

-> output formate

$ aws s3 ls

if you can get a list of buckets is available in s3 so the problem solved

edited Mar 21 '22 at 06:20

answered Mar 21 '22 at 06:14

Kuldip Mori

161
1
1
8

score 2 · Answer 13 · answered Jan 04 '23 at 08:10

2

My credential have both "+/" and I recreate one with only "+" works. https://stackoverflow.com/a/74102924/19478714

answered Jan 04 '23 at 08:10

Lion

21
1

score 1 · Answer 14 · answered Aug 20 '19 at 15:22

1

For me the problem was the / in my secret_key to get round it pasted the secret key into a .dat file on my pc copied it to the server pg the .dat file on the server and copied the secret key when using aws configure

answered Aug 20 '19 at 15:22

Chris

11
1

I also experienced this issue on a new key that contained a `/` in the secret. I found it easier to invalidate and generate a new one. – crypdick Oct 26 '20 at 23:05

score 1 · Answer 15 · edited May 09 '21 at 13:26

Newbie to Boto3,Python and AWS automation here.

I got the error

"A client error (SignatureDoesNotMatch) occurred when calling the CreateBucket operation: The request signature we calculated does not match the signature you provided. Check your key and signing method." when attempting to programmatically add an S3 bucket to my AWS account.

I use Jupyter as my IDE and spent a lot of time attempting to fix this issue. What I found is that this is related to the default region that is entered into the "config" file in .aws, for some reason the default region was "us-west-2" in my config file and the bucket I was attempting to add was in us-east-2.

I have seen some solutions attempt to correct this with environmental variables but I believe this solution is much simpler.

score 1 · Answer 16 · answered May 19 '21 at 07:17

1

Using single quotes (') instead of using qutoes (") when exporting the AWS_SECRET_ACCESS_KEY, solved this problem for me using "aws s3 cp" commands.

answered May 19 '21 at 07:17

carrotcakeslayer

809
2
9
33

score 1 · Answer 17 · answered Jun 30 '21 at 03:41

1

in my case, i have some typo on the region name. once fixed it, all works fine.

Set the Region name to None first.. then change it back correctly.

Should be good.

answered Jun 30 '21 at 03:41

sajy2k

21
3

score 1 · Answer 18 · answered Jul 09 '21 at 11:02

My case is different, yesterday these configured keys in my new Mac is working fine but today is not working. I tried comparing with the working old windows system configuration and both looks same. I couldn't understand, I copy pasted from windows to mac again then it's working fine, seems to be some invisible characters might have added.

score 1 · Answer 19 · answered Apr 18 '22 at 22:41

1

I had the same error on Windows using PuTTY. The fix was to enclose the aws_secret_access_key in the ~/.aws/credentials file with double quotes ". I think it's because the / in the aws_secret_access_key causes some issues on Windows.

answered Apr 18 '22 at 22:41

aikind

98
1
2
9

score 1 · Answer 20 · answered Jun 17 '22 at 11:55

I ran the following command via pipeline in BitBucket:

aws s3 sync --delete ./public s3://abc123.net/ --exclude "*.css" --exclude "*.js" --exclude "*.json"  --exclude "*.html" --exclude "*.svg" --exclude "*.ttf" --exclude "*.eot" --exclude "*.png" --exclude  "*.jpg" --exclude "*.gif" --exclude "*.webp" --exclude "*.woff2"

I got the following error:

fatal error: An error occurred (SignatureDoesNotMatch) when calling the ListObjectsV2 operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.

I fixed it by updating the AWS Access Key ID and AWS Secret Access Key in the deployment variables.

JimShapedCoding · Answer 21 · 2022-07-12T12:59:44.947

1

For Windows: I encountered this error since my tz was configured -3 hours than the correct tz.

If you double-checked that ~/.aws/credentials matches to what has been configured in the aws configure, sync the time zone this way:

Search for Change the Date and Time in Windows Start menu -> Turn on the Set time zone automatically -> Click on Sync Now

edited Jul 12 '22 at 12:59

answered Jul 12 '22 at 12:33

JimShapedCoding

849
1
6
17

score 1 · Answer 22 · answered Jul 24 '22 at 14:20

I solved this issue by entering the following commands in the terminal:

aws configure set aws_secret_key "": This command resets the AWS Secret key
aws configure: This command restarts the AWS configuration. Follow the prompts and input the details gotten from the AWS user as seen on the screenshot attached.

score 1 · Answer 23 · answered Aug 27 '22 at 13:16

I also had this error from The CircleCi Graphic User interface.

It can be caused by the difference of AWS environment from in CircleCI with the environment you have in your terminal when your terminal is the trigger of the .yml file to github till CircleCi.

If you are on the track of auto deployment with a connection of Github and CircleCi with AWS check the variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY if are the same as the ones you entered in your local machine.

The command: AWS configure, will help you to know at least the 4 last characters of your keys and then you can see if there are the ones you have in AWS and CircleCi.

score 1 · Answer 24 · answered Oct 17 '22 at 20:32

This is an issue that has been around at least since 2014.

It has be reproduced on a wide range of aws versions and python versions.
It has been reproduced on a wide range of operating systems.

Even though many people assume that it is linked to the use of / and + characters, it seems unlikely given that

Some keys with both / and + characters work on one system but not on another.
Some systems that don't support a specific key with / and + characters do support freshly generated keys that also contain both / and + characters.

So, there certainly has to be more to it.

Moving back and forth between a working and non-working key consistently breaks and repairs it. Meaning that it's not just some unrelated issue that gets fixed while fiddling around with the CLI tool.

Generating a new key has always instantly fixed the problem for me. So, that's definitely the way to get around it.

score 0 · Answer 25 · answered Aug 17 '23 at 10:16

0

If you're using old access key, check whether it has cli access, if no modify it or create new access key with aws cli access.

answered Aug 17 '23 at 10:16

Sivashankar

493
1
8
17

AWS CLI listing S3 buckets gives SignatureDoesNotMatch error using IAM user credentials

25 Answers25

Linked