34

I am using python requests package to get results from a API and the URL contains + sign in it. but when I use requests.get, the request is failing as the API is not able to understand + sign. how ever if I replace + sign with %2B (URI Encoding) the request is successful.

Is there way to encode these characters, so that I encode the URL while passing it to the requests package

Error: test user@gmail.com does not exist
API : https://example.com/test+user@gmail.com
user3435964
  • 663
  • 1
  • 11
  • 23
  • 1
    Take a look at [urllib.parse](https://docs.python.org/3/library/urllib.parse.html). In particular, see `urlencode`, but it's a good idea to see what else that module has to offer. – PM 2Ring Oct 17 '17 at 06:04

2 Answers2

73

You can use requests.utils.quote (which is just a link to urllib.parse.quote) to convert your text to url encoded format.

>>> import requests
>>> requests.utils.quote('test+user@gmail.com')
'test%2Buser%40gmail.com'
Boris Verkhovskiy
  • 14,854
  • 11
  • 100
  • 103
MohitC
  • 4,541
  • 2
  • 34
  • 55
  • 7
    This solution is answering the question but little bit not correct. `'https://example.com?callback_url=' + requests.utils.quote('https://callback.com')` will return `'https://example.com?callback_url=https%3A//callback.com'` (`//` didn't escaped). This happens because of default `safe='/'` argument. `requests.utils.quote(some_string, safe='')` will escape all specific symbols. – rzlvmp Apr 07 '22 at 08:48
1

In Python 3+, one can URL-encode any string using the urllib.parse.quote() (alternatively, you could use requests.utils.quote(), which is essentially using the same quote() function from urllib.parse module). This function is intended for quoting the path section of a URL, not the whole URL. It will essentially replace any special characters in a string using the %xx escape. In Python 2.x, the quote() function can be accessed directly from the urllib package, i.e., urllib.quote().

Note that the quote() function considers / characters safe by default. Hence, in case you are passing another URL to the path section (or the query string) of the URL and would like to encode / characters as well, you could do so by setting the safe parameter to '', i.e., an empty string.

Example

import requests
from urllib.parse import quote 

base_url = 'https://example.com/'
path_param = 'https://raw.githubusercontent.com/ultralytics/yolov5/master/data/images/zidane.jpg'
url = base_url + quote(path_param, safe='')
r = requests.get(url)
print(r.request.url)
Chris
  • 18,724
  • 6
  • 46
  • 80