1

I have two copies of a 400MB dataset file in my personal computer and in my Google drive. I want to play with the dataset with the programming language Julia on the Google Colab Jupyter notebook. I found a working code piece that changes the default Colab runtime type from Python 3 to Julia 1.3.1. If you run the following code in a code cell, and then reload the Colab page, the runtime type becomes Julia:

%%shell
if ! command -v julia 2>&1 > /dev/null
then
    wget 'https://julialang-s3.julialang.org/bin/linux/x64/1.3/julia-1.3.1-linux-x86_64.tar.gz' \
        -O /tmp/julia.tar.gz
    tar -x -f /tmp/julia.tar.gz -C /usr/local --strip-components 1
    rm /tmp/julia.tar.gz
fi
julia -e 'using Pkg; pkg"add Plots; add PyPlot; add IJulia; add Knet;"'
julia -e 'using Pkg; pkg"build Knet;"'

When the runtime type becomes Julia, clicking on the Mount Drive button returns the following error message:

Mounting your Google Drive is only available on hosted Python runtimes.

When I try to mount the drive during the Python runtime type, then converting the runtime type to Julia, Colab clears everything including the mounted drive. So, this method does not work, too.

When I try to upload the dataset to Colab from my computer, everything starts smoothly. However, each time that I try to upload the dataset from my computer in place of mounting the drive, I face one of these two problems: Either the upload process fails or Colab stops the Julia runtime due to inactivity (how can I start being active without my dataset). When the upload process stops without uploading the file completely, the yellow-green circle on the bottom left part of the page which indicates the percentage of the task that is completed becomes completely red. It gives no error message except this red circle. When I download the uploaded (not complete) file to my computer, I see that it is only around 20MB (the original file was 400MB). Therefore, I can understand that the upload process has failed.

The same question has been asked here before. However, the answer suggests mounting the drive in Python runtime and changing the runtime type after that. This does not work for me because when the runtime changes, everything goes away as I stated above.

By the way, my dataset cannot be found anywhere else. So, sample datasets folder does not work.

So, how can I use my dataset on Google Colab with Julia?

Community
  • 1
  • 1
Serhat Soyer
  • 39
  • 6

1 Answers1

0

If the dataset is not top secret, you can share it publicly and use gdown command to download it

run(`gdown --id 1-7dVdjCIZIxh8hHJnGTK-RA1-jL1tor4`)

Here 1-7dV...or4 is the file_id taken from the shared URL.

korakot
  • 37,818
  • 16
  • 123
  • 144