0

I know this is very simple, but I need some directions.

I have a Jupyter Notebook that I used to run on my local Linux machine. The notebook has some deep learning training code that imports dataset, processing and training and stuff.

In my local machine I have my dataset located at

'/home/USERNAME/Workspace/Final Year Project/input'

This input folder has two sub folders train and test When I run the notebook on my local machine it runs perfectly, but my system has some limitations so I chose to use Google Colab instead.

But the main issue I am facing is how to import the same dataset in Colab? Like I know it could be done using Google Drive but how?

Currently I am loading my dataset in my numpy array using the file path

If I upload my dataset to Google Drive how could I use this file path?

For example to get the training data I use the below function which takes the file path as parameter

# Get training data
def get_X_data(path, output_shape=(None, None)):
    '''
    Loads images from path/{id}/images/{id}.png into a numpy array
    '''
    img_paths = ['{0}/{1}/images/{1}.png'.format(path, id) for id in os.listdir(path)]
    X_data = np.array([skimage.transform.resize(skimage.io.imread(path)[:,:,:3], output_shape=output_shape, mode='constant', preserve_range=True) for path in img_paths], dtype=np.uint8)  #take only 3 channels/bands

    return X_data
X_train = get_X_data(train_path, output_shape=(img_height,img_width))

Any help would be really appreciated. Thank You.

Jon Doe
  • 103
  • 2
  • 5

2 Answers2

1

An easy way to sync your files to Drive on macOS or Window is to install the Drive sync client -- https://www.google.com/drive/download/

Then, in Colab, you can mount your Drive files using the following command:

from google.colab import drive
drive.mount('/content/drive')

Afterward, your files will appear in the path /content/drive/My Drive and in the file browser like so:

enter image description here

Bob Smith
  • 36,107
  • 11
  • 98
  • 91
0

You have to use this code there after uploading your dataset in Drive

  • Make sure that this chunk of code is copied in another cell before execution of the code cells you have
  • First run this code in a cell !pip install -U -q PyDrive

Then run this part of code

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials

auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)
file_id = 'REPLACE_WITH_YOUR_FILE_ID'
downloaded = drive.CreateFile({'id': file_id})
print('Downloaded content "{}"'.format(downloaded.GetContentString()))

Note: For every file in Google drive you will have a file_id that is get the shareable link of that file and last part of that link you will get a file id. A file ID looks like: laggVyWshwcyP6kEI-y_W3P8D26sz.

  • Replace this file_id in the code
Maheshwar Kuchana
  • 92
  • 8