77

Python requests is a good module to ease my web REST API access programming, I usually do like below

import json
url = 'https://api.github.com/some/endpoint'
payload = {'some': 'data'}
headers = {'Content-type': 'application/json', 'Accept': 'application/json'}

r = requests.post(url, data=json.dumps(payload), headers=headers)

And when there is error comes out, I want to see what happen behind it. Constructing the curl command to reproduce in command line is the common way, since this is the standard way which is most described in RESP API document

try:
    r = requests.post(url, data=json.dumps(payload), headers=headers)
except Exception as ex:
    print "try to use curl command below to reproduce"
    print curl_request(url,"POST",headers,payload)

It will be nice I can generate curl command sample for this request, see good example in libcloud's debug, I can't find a simple way to construct, below are the method I want to create by myself.

# below code is just pseudo code, not correct 
def curl_request(url,method,headers,payloads):
    # construct curl sample from requests' structure
    # $ curl -v -H "Accept: application/json" -H "Content-type: application/json" 
    # -d '{"some":"data"}' 
    # -X POST https://api.github.com/some/endpoint
    request = "curl -v "
    for header in headers:
        print header
        request = request + '-H "' + header + ": " + headers[header] + '" '
    for payload in payloads:
        request = request + '-d {} "' + payload + ": " + payloads[payload] + '" '         
    request = request + "-X %s %s" % (method,url)
    return request

It will also be nice if we have method in requests already

Below are the final solution get the answer, works for me. Show it here for your reference

def curl_request(url,method,headers,payloads):
    # construct the curl command from request
    command = "curl -v -H {headers} {data} -X {method} {uri}"
    data = "" 
    if payloads:
        payload_list = ['"{0}":"{1}"'.format(k,v) for k,v in payloads.items()]
        data = " -d '{" + ", ".join(payload_list) + "}'"
    header_list = ['"{0}: {1}"'.format(k, v) for k, v in headers.items()]
    header = " -H ".join(header_list)
    print command.format(method=method, headers=header, data=data, uri=url)
Larry Cai
  • 55,923
  • 34
  • 110
  • 156

2 Answers2

111

You can also use curlify to do this.

$ pip install curlify
...
import curlify
print(curlify.to_curl(r.request))  # r is the response object from the requests library.
Kerem
  • 1,494
  • 2
  • 16
  • 27
Daniel Gerber
  • 3,226
  • 3
  • 25
  • 32
  • 3
    Looks like if something like a timeout occurs, curlify won't work (since there's no response object) - any suggestions? – cdm Aug 07 '19 at 15:24
  • 3
    @cdm you could use it on a prepared request that you get from the request object by calling the prepare() method. You need to construct the Request object with a constructor rather than one of the methods that prepares the request and then sends it. – jonseymour Jun 02 '21 at 07:06
74

This method existed in requests once upon a time but it is far from being remotely relevant to the module. You could create a function that takes a response and inspects its request attribute.

The request attribute is a PreparedRequest object so it has headers, and body attributes. The body is what you pass to curl with -d and the headers can be generated as you did above. Finally you'll want to pluck off the url attribute from the request object and send that. The hooks don't matter to you unless you're doing something with a custom authentication handler.

req = response.request

command = "curl -X {method} -H {headers} -d '{data}' '{uri}'"
method = req.method
uri = req.url
data = req.body
headers = ['"{0}: {1}"'.format(k, v) for k, v in req.headers.items()]
headers = " -H ".join(headers)
return command.format(method=method, headers=headers, data=data, uri=uri)

That should work. Your data will be properly formatted whether it is as multipart/form-data or anything else.

Nickolay
  • 31,095
  • 13
  • 107
  • 185
Ian Stapleton Cordasco
  • 26,944
  • 4
  • 67
  • 72
  • thank, this is what I want to do in sample code, but it turns to be complicated, since I need to handle different type of data, headers[header] above could be integer like, can u paste your sample codes for me to reference ? – Larry Cai Jul 30 '13 at 00:55
  • If you're passing integers to requests and not getting exceptions, I don't know what you're doing. I can put some example code into my answer though and it'll work on python versions that requests work on (i.e. 2.6+) – Ian Stapleton Cordasco Jul 30 '13 at 02:12
  • Thank, your sample codes is good enough for me so far, and I updated the question with the codes I used in reality. – Larry Cai Jul 30 '13 at 08:59
  • What you're using is highly misleading for someone who doesn't understand what problem you're solving. It seems to me you want to JSON encode ALL data that you're dealing with when in reality a request could quite possibly be encoded in another way. If you want you should add your own answer instead and point to that with an explanation of why you're doing what you are doing. – Ian Stapleton Cordasco Jul 30 '13 at 17:57
  • Funny that you're using the GitHub API as the example. Have you tried [github3.py](https://github.com/sigmavirus24/github3.py)? – Ian Stapleton Cordasco Jul 31 '13 at 02:09
  • I just copied the sample from requests web, actually I use internal web service. Good to see github3.py, maybe I can try when I play with github – Larry Cai Jul 31 '13 at 02:20
  • 2
    shouldn't command be like: 'curl -X {method} -H "{headers}" -d "{data}" {uri}' (notice I added " and -d) – nivcaner Jun 08 '14 at 06:56
  • instead of quoting `"{headers}"` we have to quote each individual header. I edited the answer to do that. – Nickolay Jan 07 '17 at 19:36
  • @Nickolay `'"{0}: {1}"'` would add backslash \ to each `"` for escaping and curl command will end up filled with \, thus `"'{0}: {1}'"` would not. – Mike K Aug 11 '18 at 01:17
  • 1
    This does not work if you are using `params` to Request. – Robin Lavallée Jul 04 '20 at 20:25
  • I think you are doubling the -H flag (one in the main string and others in the join()) – Ijaz Ur Rahim Jul 22 '22 at 06:09
  • 1
    @Ijaz Ur Rahim Nope, he doesn’t. `join` adds the ` -H ` prefix _in between_ the list elements which would result in the first element not being prefixed. Hence @Nickolay’s code is correct. – Tom E Mar 07 '23 at 10:28
  • @TomE my bad, yes that's the case. – Ijaz Ur Rahim Mar 08 '23 at 16:54