python:requests, how do I post MULTIPART/FORM-DATA and stream large file?

Question

I need to POST a .wav file in MULTIPART/FORM-DATA.

my script so far is :

import requests
import json
import wave 

def get_binwave(filename):

    w = wave.open(filename, "rb")
    binary_data = w.readframes(w.getnframes())
    w.close()
    return binary_data


payload = {
    "operating_mode":"accurate",
    "model":{
       "name":"code"
    },
    "channels":{
        "first":{
            "format": "audio_format",
            "result_format": "lattice"
         }
     }
}

multiple_files = [
    ("json","application/json",json.dumps(payload)),
    ("first","audio/wave",str(get_binwave("c.wav")))]
r = requests.post("http://localhost:8080", files=multiple_files)

I'm facing two problems:

1. The .wav file binary is too big, so I'm guessing I need to stream it?

2. The the server expects the boundary to be = "xxx---------------xxx". How do I set it ?

How do I do all of this properly?

Answer 1

Requests does not actually stream multipart/form-data uploads (but it will be landing there soon). Until then, install requests-toolbelt from PyPI. To use it, your script would look like

import requests
import json
from requests_toolbelt.multipart import encoder


payload = {
    "operating_mode":"accurate",
    "model":{
       "name":"code"
    },
    "channels":{
        "first":{
            "format": "audio_format",
            "result_format": "lattice"
         }
     }
}

multiple_files = [
    ("json", "application/json", json.dumps(payload)),
    ("first", "audio/wave", open("c.wav", "rb"),
]
multipart_encoder = encoder.MultipartEncoder(
    fields=multiple_files,
    boundary="xxx---------------xxx",
)
r = requests.post("http://localhost:8080",
                  data=multipart_encoder,
                  headers={'Content-Type': multipart_encoder.content_type})

For more documentation about that library and the MultipartEncoder, see http://toolbelt.readthedocs.io/en/latest/uploading-data.html#streaming-multipart-data-encoder