Boto3 not generating correct signed-url

Question

I've a use case where I use lambda function to generate signed URL to upload to an S3 bucket, I also set the metadata values when generating signed URL, my boto3 version is boto3==1.18.35. Previously when I generate the signed-url to upload to the bucket the URL looks like this:

https://bucket-name.s3.amazonaws.com/scanned-file-list/cf389880-09ff-4301-8fa7-b4054941685b/6919de95-b795-4cac-a2d3-f88ed87a0d08.zip?AWSAccessKeyId=ASIAVK6XU35LOIUAABGC&Signature=xxxx%3D&content-type=application%2Fx-zip-compressed&x-amz-meta-scan_id=6919de95-b795-4cac-a2d3-f88ed87a0d08&x-amz-meta-collector_id=2e8672a1-72fd-41cc-99df-1ae3c581d31a&x-amz-security-token=xxxx&Expires=1641318176

But now the URL looks like this:

https://bucket-name.s3.amazonaws.com/scanned-file-list/f479e304-a2e4-47e7-b1c8-058e3012edac/3d349bab-c814-4aa7-b227-6ef86dd4b0a7.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIA2BIILAZ55MATXAGA%2F20220105%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20220105T001950Z&X-Amz-Expires=36000&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-meta-collector_id%3Bx-amz-meta-scan_id&X-Amz-Security-Token=xxxxx&X-Amz-Signature=xxxx

Notice the URL it generates now does not have the correct value for metadata information i.e. x-amz-meta-collector_id and x-amz-meta-scan_id.

The I'm using to generate signed-url is:

bucket_name = os.environ['S3_UPLOADS_BUCKET_NAME']
metadata = {
    'scan_id': scan_id,
    'collector_id': collector_id
}

params = {
    'Bucket': bucket_name,
    'Key': path + file_obj['fileName'],
    'ContentType': file_obj.get('contentType') or '',
    'Metadata': metadata
}

logger.info('metadata used for generating URL: ' + str(metadata))

s3 = boto3.client('s3')
presigned_url = s3.generate_presigned_url('put_object', Params=params, ExpiresIn=36000)
logger.info(f'Presigned URL: {presigned_url}')

return presigned_url

Because of the change in the URL, I'm getting a SignatureDidNotMatch error, Thanks for the help in advance!

WatchMyApps Lab · Accepted Answer · 2022-04-20T07:39:35.887

1

The problem is on the AWS servers, the URL generated from us-west-2 is different from the URL generated in ap-south-1.

More: The signed-url generated from a lambda deployed in the ap-south-1 region, and the X-Amz-Signature-Version was automatically being added to the URL, but when I deploy the same lambda in a different region i.e. us-west-2, I get a different format of signed-url which in my case was the correct one!

edited Apr 20 '22 at 07:39

answered Jan 05 '22 at 22:31

WatchMyApps Lab

113
1
7

1

As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Jan 06 '22 at 00:13
Can you provide the solution to this issue? – neilharia7 Jul 28 '22 at 12:19

score 0 · Answer 2 · answered Sep 01 '23 at 18:41

The same issue happens to me as well. I created the presigned url like this, with some metadata.

Part of the pre-signed-url looks like this.

....Amz-Expires=3600&X-Amz-SignedHeaders=host%3Bx-amz-meta-document_ext%3Bx-amz-meta-document_id%3Bx-amz-meta-document_type%3Bx-amz-meta-user_id&X-Amz-Security....

What you really need to do is, without touching the parameters, set the headers with correct keys and values.

key should be - x-amz-meta-{metadata key}
value should be - {metadata value}

Example:

In my case I had to set following header keys

x-amz-meta-document_ext
x-amz-meta-document_id
x-amz-meta-document_type
x-amz-meta-user_id

with the correct header values which I set on the metadata dictionary while I was creating the presigned URL.

postman

Boto3 not generating correct signed-url

2 Answers2