0

I understood that the requests library internally encodes the URL similar to urllib.parse.quote(). Earlier this used to be configurable in the request with config={'encode_uri': False} but this configuration has been discontinued.

I need to put to an AWS S3 presigned URL that contains a signature. when I use requests.put() to the URL as received, I get 403 response with SignatureDoesNotMatch.

The signature part of the URL given to requests.put() is

Bf%2BBufRIxITziSJGuAq9cYFbxBM%3D

The server sees it as:

Bf+BufRIxITziSJGuAq9cYFbxBM=

Is this related to the requests library encoding the URL and probably converting the signature part as above? If so, is there anyway to prevent it and get it to use the URL as passed?

Jake Chander
  • 1
  • 1
  • We should probably back up a bit and understand what, exactly, you are trying to do. The S3 API only supports `GET`, `HEAD`, `PUT`, and `DELETE` requests using pre-signed URLs... not `POST`. I can think of no exceptions, but I could be mistaken. `POST` (browser-based object upload) requests use a web form to convey the policy, signature, and other values, not a pre-signed URL. Additionally, your signature looks correct and there is no obvious reason why you would want to avoid this. In SigV2 signatures, `+` `/` `=` are correctly escaped as `%2B` `%2F` `%3E`. – Michael - sqlbot Apr 21 '18 at 20:12
  • You're right Michael, it should have been put not post and I have updated that. About the signature, AWS does not appear to expect the escaped characters – Jake Chander Apr 22 '18 at 04:33

1 Answers1

1

Override the encoding function

import requests, urllib

class NoQuotedCommasSession(requests.Session):
    def send(self, *a, **kw):
        # a[0] is prepared request
        a[0].url = a[0].url.replace(urllib.quote(","), ",")
        return requests.Session.send(self, *a, **kw)

s = NoQuotedCommasSession()
s.get("http://somesite.com/an,url,with,commas,that,won't,be,encoded.")