Download files from a website using python

Question

I am new to Python and I have a requirement to download multiple csv-files from a website authenticated using username and password.

I wrote the below piece of code to download a single file but unfortunately the contents in the downloaded file are not same as in the original file.

Could you please let me know what I am doing wrong here and how to achieve this.

import requests
import shutil
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
url="https:xxxxxxxxxxxxxxxxxxxx.aspx/20-02-2019 124316CampaignExport.csv" 
r = requests.get(url, auth=('username', 'Password'), 
verify=False,stream=True)
r.raw.decode_content = True
with open("D:/20-02-2019 124316CampaignExport.csv", 'wb') as f:
    shutil.copyfileobj(r.raw, f)

Why `raw`? What will be in case `shutil.copyfileobj(io.BytesIO(r.content))` ? (don't forget `import io` for this) — Alex Yu, Feb 20 '19 at 03:22

score 2 · Answer 1 · answered Feb 20 '19 at 03:22

The following code worked for me (only indenting the last line):

import requests
import shutil
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
url="linkToDownload" 
r = requests.get(url, auth=('username', 'Password'), 
verify=False,stream=True)
r.raw.decode_content = True
with open("filename", 'wb') as f:
    shutil.copyfileobj(r.raw, f)

This means the problem is stemming from your URL or authentication rather than the python code itself.

Your URL has a space in it, which is likely causing an error. I can't confirm for sure as I don't have your URL. If you have write-access to it, try renaming it with a "_" insetead of a space.

Download files from a website using python

1 Answers1