Querystring Array Parameters in Python using Requests

Question

I have been trying to figure out how to use python-requests to send a request that the url looks like:

http://example.com/api/add.json?name='hello'&data[]='hello'&data[]='world'

Normally I can build a dictionary and do:

data = {'name': 'hello', 'data': 'world'}
response = requests.get('http://example.com/api/add.json', params=data)

That works fine for most everything that I do. However, I have hit the url structure from above, and I am not sure how to do that in python without manually building strings. I can do that, but would rather not.

Is there something in the requests library I am missing or some python feature I am unaware of?

Also what do you even call that type of parameter so I can better google it?

Answer 1

All you need to do is putting it on a list and making the key as list like string:

data = {'name': 'hello', 'data[]': ['hello', 'world']}
response = requests.get('http://example.com/api/add.json', params=data)

Answer 2

What u are doing is correct only. The resultant url is same what u are expecting.

>>> payload = {'name': 'hello', 'data': 'hello'}
>>> r = requests.get("http://example.com/api/params", params=payload)

u can see the resultant url:

>>> print(r.url)
http://example.com/api/params?name=hello&data=hello

According to url format:

In particular, encoding the query string uses the following rules:

• Letters (A–Z and a–z), numbers (0–9) and the characters .,-,~ and _ are left as-is
• SPACE is encoded as + or %20
• All other characters are encoded as %HH hex representation with any non-ASCII characters first encoded as UTF-8 (or other specified encoding)

So array[] will not be as expected and will be automatically replaced according to the rules:

If you build a url like :

`Build URL: http://example.com/api/add.json?name='hello'&data[]='hello'&data[]='world'`

OutPut will be:

>>> payload = {'name': 'hello', "data[]": 'hello','data[]':'world'}
>>> r = requests.get("http://example.com/api/params", params=payload)
>>> r.url
u'http://example.com/api/params?data%5B%5D=world&name=hello'

This is because Duplication will be replaced by the last value of the key in url and data[] will be replaced by data%5B%5D.

If data%5B%5D is not the problem(If server is able to parse it correctly),then u can go ahead with it.

Source Link

Answer 3

One solution if using the requests module is not compulsory, is using the urllib/urllib2 combination:

payload = [('name', 'hello'), ('data[]', ('hello', 'world'))]
params = urllib.urlencode(payload, doseq=True)
sampleRequest = urllib2.Request('http://example.com/api/add.json?' + params)
response = urllib2.urlopen(sampleRequest)

Its a little more verbose and uses the doseq(uence) trick to encode the url parameters but I had used it when I did not know about the requests module.

For the requests module the answer provided by @Tomer should work.

Answer 4

Some api-servers expect json-array as value in the url query string. The requests params doesn't create json array as value for parameters.

The way I fixed this on a similar problem was to use urllib.parse.urlencode to encode the query string, add it to the url and pass it to requests

e.g.

from urllib.parse import urlencode
query_str = urlencode(params)
url = "?" + query_str
response = requests.get(url, params={}, headers=headers)

Answer 5

The solution is simply using the famous function: urlencode

>>> import urllib.parse
>>> params = {'q': 'Python URL encoding', 'as_sitesearch': 'www.urlencoder.io'}
>>> urllib.parse.urlencode(params)
'q=Python+URL+encoding&as_sitesearch=www.urlencoder.io'