Searching through Stackoverflow I found a lot of answers how to download a file with Python (may it be using urllib2, requests etc), but only one for using compression (httplib2). Sadly this answer is not using streams.
How can I download a large binary file with Python using compression and streams?
What I currently use is to slow and looks the following:
def get_large_file_a(url, file, length): # using requests
response = requests.get(url, stream=True)
with open(file, 'wb') as f:
for data in tqdm(
response.iter_content(),
total=length,
unit='B',
unit_scale=True,
):
f.write(data)
