1

I want to do the following if the folder doesn't exist then create it, but if I execute my script (second time) obviously will be already so I need to remove the folder and download the file inside, but my current script overwrites the location, and demo becomes a file, how can I do this?

import os, shutil, wget

base_path = os.path.dirname(os.path.abspath(__file__))
directory = os.path.join(base_path, 'demo')
# check for extraction directories existence
if not os.path.isdir(directory):
    os.makedirs(directory)
else:
    if os.path.exists(directory) and os.path.isdir(directory):
        shutil.rmtree(directory)
    #os.makedirs(directory)

remote_location = 'https://github.com/facebookresearch/SING/blob/master/sing/nsynth/examples.json.gz?raw=true'
try:
    wget.download(remote_location, out=directory)
except:
    pass
JLeno46
  • 1,186
  • 2
  • 14
  • 29
jahjh21a
  • 89
  • 11
  • Checking if a directory exists, then trying to create it if it does not, creates a race condition: some other process could create it *after* `isdir` returns `False`, but before you call `makdirs`. A better approach is to call `makedirs`, but ignore the exception raised if it already exists. – chepner Dec 03 '20 at 20:33
  • Added how to download abd also how to read it to json – Prayson W. Daniel Dec 05 '20 at 08:56

2 Answers2

2

Use pathlib when working with path and folders

from pathlib import Path
import requests

DIR_PATH = Path(__file__).parent / "demo"

# create dir_path if it does not exist
Path(DIR_PATH).mkdir(parents=True, exist_ok=True)

URL = "https://github.com/facebookresearch/SING/blob/master/sing/nsynth/examples.json.gz?raw=true"

response = requests.get(URL, stream=True)
with open(f"{DIR_PATH}/example.json.gz", "wb") as h:
    for data in response.iter_content():
        h.write(data)

Explanation:

Path(__file__).parent returns directory (parent) to which the python script is called. With pathlib / is used to as you would in linux. We add "demo" and created it if it does not exist.

Using requests, we get the file and place it to our folder using streaming.

To read the file, we will unzip it and load it to json

import json
import gzip
from pathlib import Path


DIR_PATH = Path(__file__).parent / "demo"
with gzip.open(f"{DIR_PATH}/example.json.gz", 'rb') as gz:
    json_data = json.load(gz)
Prayson W. Daniel
  • 14,191
  • 4
  • 51
  • 57
-1
file_name = "new_file.ext" # Needs A file name!
# Otherwise we'll just overwrite the directory
if not os.path.exists(directory):
    os.makedirs(directory) # Create folder if it doesn't exist

file_location = os.path.join(directory, file_name) # Create the actual file location from our file name
remote_location = 'https://github.com/facebookresearch/SING/blob/master/sing/nsynth/examples.json.gz?raw=true'
try:
    wget.download(remote_location, out=file_location)
except:
    pass
TheLazyScripter
  • 2,541
  • 1
  • 10
  • 19
  • Don't check if the directory exists; *try* to create it, and ignore the exception raised if it already does. – chepner Dec 03 '20 at 20:34
  • The value of checking if it exists first would be to allow for some type of naming incrementation on directory name or even a chance to cancel the download. However in most cases I agree – TheLazyScripter Dec 03 '20 at 20:36
  • this is exactly I have. I want to create if not exist > download, and if exist folder/file , remove with shutil.rmtree(path) , and recreate again – jahjh21a Dec 03 '20 at 20:36