how to translate this code in curl using bash to request in python

Question

I'm trying to translate a bash script that works with curl to send a code remotely to a mega Arduino, using an ESP01 as a programmer with the ESP-link firmware, the bash script that does that work, summarized in a few lines, is this:

   #! /bin/bash

   # first make a post to reset the arduino 
   curl -m 10 -s -w '%{http_code}' -XPOST 
   http://192.168.4.1/pgmmega/sync
   sleep 0.5 
   # make a GET to Sync with it and wait for the sync
   curl -s http://192.168.4.1/pgmmega/sync
   sleep 0.1
   #send the .hex file 
   curl -m 20 -s -g -d @/tmp/arduino_build_274266/SMI_6_0_1.ino.hex 
   http://192.168.4.1/pgmmega/upload

And my code in python3 is:

    #! /bin/python3
    import requests
    import time 
    urlsync = 'http://192.168.4.1/pgmmega/sync' 
    urlupload = 'http://192.168.4.1/pgmmega/upload' 

    file = {'upload_file': 
    open('/home/wander/app_llamadores/SMI_6_0_1.ino.hex', 'rb')}


    rsync = requests.post(urlsync, stream=True, timeout = 5)
    time.sleep(0.4)
    print(rsync1.status_code) #204
    rsync1 = requests.get(urlsync)

    print(rsync1.status_code) #200 

    rupload = requests.post(urlupload, files=file, timeout=20)

    print(rupload.status_code)

I tried with that code, and although the synchronization part seems to work fine, when I send the .hex file, I get this ..

    Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", 
    line 387, in _make_request
    six.raise_from(e, None)
    File "<string>", line 3, in raise_from
    File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", 
    line 383, in _make_request
    httplib_response = conn.getresponse()
    File "/usr/lib/python3.6/http/client.py", line 1331, in 
    getresponse
    response.begin()
    File "/usr/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
    File "/usr/lib/python3.6/http/client.py", line 258, in 
    _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
    File "/usr/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
    socket.timeout: timed out

    During handling of the above exception, another exception 
    occurred:

    Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/requests/adapters.py", line 
    440, in send
    timeout=timeout
    File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", 
    line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
    File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", 
    line 357, in increment
    raise six.reraise(type(error), error, _stacktrace)
    File "/usr/lib/python3/dist-packages/six.py", line 693, in 
    reraise
    raise value
    File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", 
    line 601, in urlopen
    chunked=chunked)
    File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", 
    line 389, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
    File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", 
    line 309, in _raise_timeout
    raise ReadTimeoutError(self, url, "Read timed out. (read 
    timeout=%s)" % timeout_value)
    urllib3.exceptions.ReadTimeoutError: 
    HTTPConnectionPool(host='192.168.4.1', port=80): Read timed  
    out. (read timeout=20)

    During handling of the above exception, another exception 
    occurred:

    Traceback (most recent call last):
    File "./CargaFirmware.py", line 17, in <module>
    rupload = requests.post(urlupload, files=file, stream=True, 
    timeout=20)
    File "/usr/lib/python3/dist-packages/requests/api.py", line 112, 
    in post
    return request('post', url, data=data, json=json, **kwargs)
    File "/usr/lib/python3/dist-packages/requests/api.py", line 58, 
    in request
    return session.request(method=method, url=url, **kwargs)
    File "/usr/lib/python3/dist-packages/requests/sessions.py", line 
    520, in request
    resp = self.send(prep, **send_kwargs)
    File "/usr/lib/python3/dist-packages/requests/sessions.py", line 
    630, in send
    r = adapter.send(request, **kwargs)
    File "/usr/lib/python3/dist-packages/requests/adapters.py", line 
    521, in send
    raise ReadTimeout(e, request=request)
    requests.exceptions.ReadTimeout: 
    HTTPConnectionPool(host='192.168.4.1', port=80): Read timed  
    out. (read timeout=20)

It never interrupts the load, but after a few seconds the microcontroller stops waiting for the code, and restarts, the code must be loaded before 20 seconds after the synchronization. — Wander Jose, Feb 19 '19 at 15:39

score 0 · Answer 1 · answered Feb 19 '19 at 21:09

Your curl and requests commands are not equivalent.

With curl you are HTTP POSTing data directly in the request body. With requests you are (implicitly) creating MultiPart Encoded form data, with upload_file field set to the contents of the SMI_6_0_1.ino.hex file.

Simple way to see what is happening under the hood is to prepare request, is to prepare and inspect request without sending:

from requests import Request

url = "http://192.168.4.1/pgmmega/sync"
filename = "SMI_6_0_1.ino.hex"

req = Request("POST", url, files={"upload_file": open(filename, "rb")}).prepare()
print(vars(req)["body"])
# b'--052baa09bbc2f6ab52a2c9f8f2b99b3a\r\nContent-Disposition: form-data; name="upload_file"; filename="SMI_6_0_1.ino.hex"\r\n\r\n\r\n--052baa09bbc2f6ab52a2c9f8f2b99b3a--\r\n'

print(vars(req)["headers"]["Content-Type"])
# multipart/form-data; boundary=052baa09bbc2f6ab52a2c9f8f2b99b3a

and

req = Request("POST", url, data=open(filename, "rb")).prepare()
print(vars(req)["body"])
# <_io.BufferedReader name='SMI_6_0_1.ino.hex'>

print(vars(req)["headers"]["Content-Type"])
# KeyError: 'content-type'

So most probably requests are sending a file in format that cannot be read by target server. In this case, switch to requests.post(url, data=...) should fix the problem.

This is not that uncommon problem. There is a web page, where you can convert curl to requests on your own - https://curl.trillworks.com/

Thank you very much, it is Exactly what happened, the comment was very helpful, I did not understand the subject very well, then I saw the response of the Microcontroller and I was able to see that it responded with "Error in request format", due to the explained in your commentary. — Wander Jose, Feb 20 '19 at 13:57

how to translate this code in curl using bash to request in python

1 Answers1